Tiny Deep Ensemble: Uncertainty Estimation in Edge AI Accelerators via Ensembling Normalization Layers with Shared Weights
CoRR(2024)
Abstract
The applications of artificial intelligence (AI) are rapidly evolving, and
they are also commonly used in safety-critical domains, such as autonomous
driving and medical diagnosis, where functional safety is paramount. In
AI-driven systems, uncertainty estimation allows the user to avoid
overconfidence predictions and achieve functional safety. Therefore, the
robustness and reliability of model predictions can be improved. However,
conventional uncertainty estimation methods, such as the deep ensemble method,
impose high computation and, accordingly, hardware (latency and energy)
overhead because they require the storage and processing of multiple models.
Alternatively, Monte Carlo dropout (MC-dropout) methods, although having low
memory overhead, necessitate numerous (∼ 100) forward passes, leading to
high computational overhead and latency. Thus, these approaches are not
suitable for battery-powered edge devices with limited computing and memory
resources. In this paper, we propose the Tiny-Deep Ensemble approach, a
low-cost approach for uncertainty estimation on edge devices. In our approach,
only normalization layers are ensembled M times, with all ensemble members
sharing common weights and biases, leading to a significant decrease in storage
requirements and latency. Moreover, our approach requires only one forward pass
in a hardware architecture that allows batch processing for inference and
uncertainty estimation. Furthermore, it has approximately the same memory
overhead compared to a single model. Therefore, latency and memory overhead are
reduced by a factor of up to ∼ M×. Nevertheless, our method does not
compromise accuracy, with an increase in inference accuracy of up to ∼ 1%
and a reduction in RMSE of 17.17% in various benchmark datasets, tasks, and
state-of-the-art architectures.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined