Towards a ‘resource cadastre’ for a circular economy – Urban-scale building material detection using street view imagery and computer vision

The lack of data on existing buildings hinders efforts towards repair, reuse, and recycling of materials, which are crucial for mitigating the climate crisis. Manual acquisition of building data is complex and time-consuming, but combining street-level imagery with computer vision could significantly scale-up building materials documentation. We formulate the problem of building facade material detection as a multi-label classification task and present a method using GIS and street view imagery with just a few hundred annotated samples and a fine-tuned image classification model. Our method shows strong performance with macro-averaged F1 scores of 0.91 for Tokyo, 0.91 for NYC, 0.96 for Zurich, and 0.93 for the merged dataset. By utilizing open-access and non-proprietary data, our method can be scaled-up step by step to a global level. We make our in the wild dataset publicly available as the Urban Resource Cadastre Repository to encourage future work on automatic building material detection.