CrowdNAS: A Crowd-guided Neural Architecture Searching Approach to Disaster Damage Assessment

AbstractDisaster damage assessment (DDA) has emerged as an important application in disaster response and management, which aims to assess the damage severity of an affected area by leveraging AI (e.g., deep learning) techniques to examine the imagery data posted on social media during a disaster event. In this paper, we focus on a crowd-guided neural architecture searching (NAS) problem in DDA applications. Our goal is to leverage human intelligence from crowdsourcing systems to guide the discovery of the optimal neural network architecture in the design space to achieve the desirable damage assessment performance. Our work is motivated by the limitation that the deep neural network architectures in current DDA solutions are mainly designed by AI experts, which is known to be both time-consuming and error-prone. Two critical technical challenges exist in solving our problem: i) it is challenging to design a manageable NAS space for crowd-based solutions; ii) it is non-trivial to transfer the imperfect crowd knowledge to effective decisions in identifying the optimal neural network architecture of a DDA application. To address the above challenges, we develop CrowdNAS, a crowd-guided NAS framework that develops novel techniques inspired by AI, crowdsourcing, and estimation theory to address the NAS problem. The evaluation results from two real-world DDA applications show that CrowdNAS consistently outperforms the state-of-the-art AI-only, crowd-AI, and NAS baselines by achieving the highest classification accuracy in the damage assessment while maintaining a low computational cost under various evaluation scenarios.