A Performance Analysis of Deep Neural Network Models on an Edge Tensor Processing Unit

Machine learning on edge devices, embedded systems at the boundaries of computer networks, can provide real-time insight into data-driven problems in many application areas. Further, hardware-based machine learning accelerators, like the Edge tensor processing unit (TPU), offer the promise of saving time and energy on edge computations. By analyzing the performance of machine learning models on edge hardware, we can better understand when and how to apply machine learning on these systems. We analyze the characteristics of models that benefit from the Edge TPU and also demonstrate cases in which a low-powered, mobile CPU will outperform the TPU. We also compare the energy usage of the Edge TPU with a mobile CPU.