The eurosdr rpas benchmark: open dataset description and summary of key results

In 2021 EuroSDR initiated a benchmark study with the aim to evaluate the geometric quality of real-world survey data generated from state-of-practice commercial Remotely Piloted Aircraft System (RPAS) photogrammetry (including DJI P4 RTK and DJI P1) and lidar (including DJI L1 and Riegl MiniVUX). The particular benchmark focus was on achievable data quality from real-world network configurations in the absence of ground control, on-the-fly Real Time Kinematic (RTK) corrections, and/or local GNSS base station information. Successive custom datasets were released to registered benchmark participants who submitted individual outputs that were independently evaluated against reference surveys. Without the inclusion of any supporting ground information, DJI P4 RTK and DJI P1 RPAS solutions were found to deliver m- and dm-level accuracies, respectively, in both plan and height. RTK solutions were found to provide cm-level precisions and accuracies, with some outliers. The introduction of ground control points resulted in similar planimetric accuracy to the RTK solutions, but with slight improvements in height. In terms of lidar datasets, the Riegl MiniVUX solution, using corrections from a local base station, was found to provide smaller discrepancies than the DJI L1 RTK solution, when independently compared against terrestrial laser scanning surveys. This paper provides various quality statistics and demonstrates multiple ways of assessing the geometric quality of RPAS data. The EuroSDR RPAS benchmark datasets are now openly available online in order to support and facilitate further investigation by the community.