In-plane Strain Analysis by Correlating Geometry and Visual Data Through a Gradient-Based Surface Reconstruction

Abnormalities in tissue can be detected and analyzed by evaluating mechanical properties, such as strain and stiffness. While current sensor systems are effective in measuring longitudinal properties perpendicular to the measurement sensor, identifying in-plane deformation remains a significant challenge. To address this issue, this paper presents a novel method for reconstructing in-plane deformation of observed tissue surfaces using a fringe projection sensor specifically designed for measuring tissue deformations. The method employs the latest techniques from computer vision, such as differentiable rendering, to formulate the in-plane reconstruction as a differentiable optimization problem. This enables the use of gradient-based solvers for an efficient and effective optimization of the problem optimum. Depth information and image information are combined using landmark correspondences between the respective image observations of the undeformed and deformed scenes. By comparing the reconstructed pre- and post-deformation geometry, the in-plane deformation can be revealed through the analysis of relative variations between the corresponding models' geometries. The proposed reconstruction pipeline is validated on an experimental setup, and the potential for intraoperative applications is discussed.