A cloud-based data processing and visualization pipeline for the fibre roll-out in Germany

To support the roll-out of fibre broadband Internet in Germany, Deutsche Telekom has set itself the goal of connecting more than 2.5 million households per year to FTTH (Fibre to the Home). However, planning and approval processes have been very complex and time-consuming in the past due to high communication overhead between stakeholders, missing automation, and lack of information about planning areas. Telekom addresses this problem by collecting large amounts of geospatial data (3D point clouds and 360(degrees) panorama images), which can be used to automatically find suitable routes for fibre optic lines, to determine possible locations for distribution cabinets, as well as to build a 3D visualization helping to create detailed plans and to present them to decision makers. This speeds up planning tremendously, but processing this data and creating the visualization in a short time requires automation. In this systems paper, we present a data processing platform that we have built and operated together with Telekom over the course of the last six and a half years. The platform makes use of the cloud to manage Big Data in a scalable and elastic manner. It builds upon research results from us, specifically a scientific workflow management system to automate processing, as well as Fibre3D, a web-based tool that planners can use to display the processed data and to perform fine-planning. Besides the technological aspects, this paper also describes a practical use case that shows how the platform and Fibre3D help Telekom speed up the planning and approval process. We also summarize lessons learned and give recommendations for the design of systems similar to ours. With the presented technology, Telekom has been able to already connect more than 8 million households to FTTH and expects to even improve on this in the future. We consider our collaboration, therefore, an example of how well knowledge and technology transfer between research and industry can work, and, at the same time, what impact it can have on society.