FTT-NAS: Discovering Fault-Tolerant Neural Architecture
2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)(2020)
摘要
With the fast evolvement of deep-learning specific embedded computing systems, applications powered by deep learning are moving from the cloud to the edge. When deploying NNs onto the edge devices under complex environments, there are various types of possible faults: soft errors caused by atmospheric neutrons and radioactive impurities, voltage instability, aging, temperature variations, and malicious attackers. Thus the safety risk of deploying neural networks at edge computing devices in safety-critic applications is now drawing much attention. In this paper, we implement the random bit-flip, Gaussian, and Salt-and-Pepper fault models and establish a multi-objective fault-tolerant neural architecture search framework. On top of the NAS framework, we propose Fault-Tolerant Neural Architecture Search (FT-NAS) to automatically discover convolutional neural network (CNN) architectures that are reliable to various faults in nowadays edge devices. Then we incorporate fault-tolerant training (FTT) in the search process to achieve better results, which we called FTT-NAS. Experiments show that the discovered architecture FT-NAS-Net and FTT-NAS-Net outperform other hand-designed baseline architectures (58.1%/86.6% VS. 10.0%/52.2%), with comparable FLOPs and less parameters. What is more, the architectures trained under a single fault model can also defend against other faults. By inspecting the discovered architecture, we find that there are redundant connections learned to protect the sensitive paths. This insight can guide future fault-tolerant neural architecture design, and we verify it by a modification on ResNet-20-ResNet-M.
更多查看译文
关键词
fault-tolerant training,FTT-NAS-Net outperform,hand-designed baseline architectures,single fault model,future fault-tolerant,embedded computing systems,deep learning,atmospheric neutrons,radioactive impurities,neural networks,edge computing devices,safety-critic applications,random bit-flip,NAS framework,convolutional neural network architectures,fault-tolerant neural architecture search,FT-NAS-Net architecture
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络