Investigation of Q-learning applied to DVFS management of a System-on-Chip

This paper presents a new Q-learning based strategy to manage Dynamic Voltage Frequency Scaling (DVFS) on a system on chip (SoC) such that the energy consumption is reduced. We address software applications with throughput constraints. The proposed Q-learning formulation has two main advantages: it has a reduced state space to limit the overhead and it embeds a new reward function tailored for throughput-constrained applications. The DVFS manager is evaluated on a test chip executing an HMAX object recognition application. We perform an experimental investigation of the main Q-learning parameters. The results suggest that the proposed method reduces the energy consumed with up to 44% at the cost of occasionally increasing the number of throughput violations, when compared to two state-of-the-art feedback controllers that address the same application domain.