Contextual Multi-Armed Bandits for Non-Stationary Heterogeneous Mobile Edge Computing

Base station (BS) selection for task offloading in Mobile Edge Computing (MEC) is a challenging problem due to the dynamic nature of MEC systems. The wireless channel as well as the load of BSs are stochastic quantities that can change in a statistically non-stationary fashion. Moreover, the computation capabilities of the BSs are heterogeneous. As the dynamic behaviour of a MEC system is, in practical scenarios, not known in advance, deciding where to offload has to be done under uncertainty about the MEC system and considering its non-stationary and heterogeneous characteristics. This paper investigates latency minimization in MEC with heterogeneous BSs. In order to meet low latency demands, a mobile unit (MU) has to quickly identify the best BS for offloading different computation tasks while facing uncertainty about the non-stationary system dynamics. To solve this problem, we propose a novel piece-wise stationary contextual Multi-Armed Bandit (MAB) algorithm that treats different task types as context and detects non-stationary changes in the BSs' performance. With the use of extensive simulations, we show that our proposed approach outperforms state-of-the-art algorithms, as it quickly adapts to changes in the MEC system and exhibits no penalty during stationary phases.