Multi-frame optimisation for active stereo with inverse renderingto obtain consistent shape and projector-camera posesfor 3D endoscopic system

3D measurement of inside human body becomes important and several techniques have been proposed. Among them, ultra-small pattern projector with off-the-shelf endoscope is promising because of stability, accuracy and cost-effectiveness. One drawback of the system is that since 3D shape is reconstructed for each frame while endoscope head is moving, they are not registered. In addition, shapes are distorted independently, since the position of the projector cannot be fixed to the head. In this paper, we propose a novel bundle adjust (BA) algorithm to solve both distortion and shape registration problems for a 3D endoscope system based on pattern projection. By using the proposed method, multiple measurements that potentially have scale inconsistencies and shape distortions are registered, with simultaneous correction of inter-frame inconsistencies. The effectiveness of the technique is proved for measurements of concave shapes made of day, phantom models, and a real stomach of a pig.