Energy-efficient Ferroelectric-FET-based Agent with Memory Trace for Enhanced Reinforcement Learning

In this work, for the first time, we algorithmically merge the memory trace effect into activation function in neural network for reinforcement learning, and a novel ferroelectric FET (FeFET) based agent with both activation function and plastic weight is proposed and experimentally demonstrated. By exploiting the physics of short-time scale dynamics of depolarization process induced by depolarization field, the activation function with memory trace effect can be emulated in one FeFET at ultra-low operating voltage of 0.5V. In addition, the plastic weight with multilevel states can be realized by the long-time scale retention of spontaneous polarization in FeFET simultaneously. Moreover, based on the proposed FeFET-based agent, reinforcement learning is demonstrated with convergence rate boost and high energy efficiency. This work provides a promising highly-integrated agent solution for RL system.