Experimental demonstration of a robust training method for strongly defective neuromorphic hardware
CoRR(2023)
摘要
The increasing scale of neural networks needed to support more complex
applications has led to an increasing requirement for area- and
energy-efficient hardware. One route to meeting the budget for these
applications is to circumvent the von Neumann bottleneck by performing
computation in or near memory. An inevitability of transferring neural networks
onto hardware is that non-idealities such as device-to-device variations or
poor device yield impact performance. Methods such as hardware-aware training,
where substrate non-idealities are incorporated during network training, are
one way to recover performance at the cost of solution generality. In this
work, we demonstrate inference on hardware neural networks consisting of 20,000
magnetic tunnel junction arrays integrated on a complementary
metal-oxide-semiconductor chips that closely resembles market-ready spin
transfer-torque magnetoresistive random access memory technology. Using 36
dies, each containing a crossbar array with its own non-idealities, we show
that even a small number of defects in physically mapped networks significantly
degrades the performance of networks trained without defects and show that, at
the cost of generality, hardware-aware training accounting for specific defects
on each die can recover to comparable performance with ideal networks. We then
demonstrate a robust training method that extends hardware-aware training to
statistics-aware training, producing network weights that perform well on most
defective dies regardless of their specific defect locations. When evaluated on
the 36 physical dies, statistics-aware trained solutions can achieve a mean
misclassification error on the MNIST dataset that differs from the
software-baseline by only 2 %. This statistics-aware training method could be
generalized to networks with many layers that are mapped to hardware suited for
industry-ready applications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要