In-sensor nonlinear convolutional processing based on hybrid MTJ/CMOS arrays

In-sensor computing implemented by novel neuromorphic devices has been regarded as the potential technology to break the acquisition wall. Moreover, the nonlinear convolution inspired by the biological neural system outperforms the traditional linear convolution. Therefore, realizing the in-sensor nonlinear convolutional processing with the intrinsic nonlinearity of novel neuromorphic devices would be exciting and hardware friendly. Here, a new type of in-sensor nonlinear convolutional processing architecture is proposed based on STT-MTJ (spin transfer torque-magnetic tunnel junction) devices and simple peripheral circuits. The nonlinear dynamic characters of the STT-MTJ device are regulated by bias currents and magnetic fields. Meanwhile, the hybrid STT-MTJ/CMOS (complementary metal oxide semiconductor) circuit exhibits nonlinear dependence of the output voltage on bias currents and magnetic fields. In this way, a nonlinear convolutional unit is developed by the inherent property of hardware. Based on the in-sensor nonlinear convolutional unit, a convolutional network is simulated to perform the classification task, and a high accuracy is realized. This work indicates that the in-sensor nonlinear convolution offers a promising way to develop in-sensor computing at the edge intelligence devices.