Hierarchical mussel farm reconstruction from video with object tracking

As the mussel farming industry expands, particularly in regions such as New Zealand, there is a growing need for advanced monitoring and management solutions to ensure sustainability and operational efficiency. The current reliance on manual infrequent observation of aquaculture structures limits farmers' ability to monitor them in real time. Addressing these challenges, large-scale 3D reconstruction provides a practical solution by facilitating the creation of replicable depictions of mussel farm scenes and floatation buoys derived from recorded video, supporting off-site evaluation and enabling precise decision making. This paper introduces a novel approach to enhance the visualisation and monitoring of mussel farm floatation through a hierarchical reconstruction process. In contrast to earlier studies, we focus on recovering not only the overall environment (background) but also the finer details of key elements such as buoys to create a comprehensive representation of mussel farm geometry and appearance. We propose to segment the scene into the background and granular object instances and reconstruct them separately in a multi-stage process. The initial 3D scene reconstruction is performed using the Structure-from-Motion (SfM) technique, leveraging video footage captured by a vessel-mounted camera. This coarse reconstruction serves as the foundation for subsequent fine-grained enhancements. To recover finer details, object tracking is applied and the trajectories obtained are then conjunct with geometry triangulation to determine the real-world positions of individual buoys. A multiple-scale denoising method, grounded in dominant direction correlation, is implemented to eliminate non-reliable tracking objects and reduce reconstruction artifacts, ensuring the accuracy of the final results. This hierarchical reconstruction approach contributes to the advancement of mussel farm management by offering a powerful technology for comprehensive visualisation, enabling farmers to make informed decisions based on a detailed understanding of the mussel farm's geometry and dynamics.