Confidential Execution of Deep Learning Inference at the Untrusted Edge with ARM TrustZone.

This paper proposes a new confidential deep learning (DL) inference system with ARM TrustZone to provide confidentiality and integrity of DL models and data in an untrusted edge device with limited memory. Although ARM TrustZone supplies a strong, hardware-supported trusted execution environment for protecting sensitive code and data in an edge device against adversaries, resource limitations in typical edge devices have raised significant challenges for protecting on-device DL requiring large memory consumption without sacrificing the security and accuracy of the model. The proposed solution addresses this challenge without modifying the protected DL model, thereby preserving the original prediction accuracy. Comprehensive experiments using different DL architectures and datasets demonstrate that inference services for large and complex DL models can be deployed in edge devices with TrustZone with limited trusted memory, ensuring data confidentiality and preserving the original model's prediction exactness.