Registration of nasopharyngoscopic video with treatment planning ct scans

Purpose: We present a method that allows one to generate data from nasopharyngoscopy videos into a form suitable for use in radiation treatment planning, as well as for review of the data itself as a 3D object. Methods and Materials: The accurate transfer of data from endoscopic video frames to the planning CT requires an explicit endoscopic video/CT registration. Our approach to accomplishing this goal is to first reconstruct an accurate 3D model of the anatomy from the 2D video data as seen on multiple video frames. That reconstruction, which we call an endoscopogram, can then be deformably registered to the CT scan. To generate the endoscopogram we first compute a sparse 3D point cloud by tracking corresponding feature points from one video frame to another using standard structure-from-motion (SfM) techniques adapted to the pharyngeal region. We then use a second technology, shape-from-shading (SFS), to produce highquality images from each video frame. Uncorrected, these SFS images contain dense information and in many places accurate local curvature, but they lack global spatial fidelity. Using our newly developed techniques, we can correct the SFS images using global information from the sparse SFS-generated point cloud as a constraint. We then register several of these corrected SFS images to each other to generate a 3D image that displays the entire surface area seen in the whole video. Results: We have successfully computed a spatially accurate endoscopogram (shown below) and have registered it with the planning CT scan with an average accuracy of about 3-5 mms. Conclusions: The technologies developed by our working group will allow accurate transfer of data from endoscopy images directly onto the planning CT scan as well as review of the video as a 3D object.