First results of the ex vivo application and analysis with neural networks in hyperspectral imaging (HSI) of squamous cell carcinoma of head and neck

Surgical therapy of head and neck tumors aims at complete resection of the tumor with adherence to safety margins. In this study, hyperspectral imaging (HSI) is used to differentiate ex vivo tumor from healthy tissue based on their spectroscopic characteristics. Acquired data will be used to train a neural network which can differentiate tumor tissue from healthy ones. Resected tumor samples from 13 patients with squamous cell carcinoma were photographed ex vivo with an HSI camera. HSI data acquisition was supplemented with a high-resolution color (RGB) image using the integrated camera. In the RGB image, the areas of tumor, tumor margin, healthy mucosa, and musculature were annotated. The RGB and HSI images were registered and divided into tiles with a resolution of 40x40 pixels with 104 wavelengths. The neural network was trained and tested using the tiles. A total of 4666 test tiles were selected from 14 annotated photographs. Unfocused and artifact-overlaid tiles were excluded. 3280 tiles were used for neural network training. For the remaining tiles, the annotated RGB images were compared with the neural network predictions. Here, the neural network showed an accuracy of 95%. The training and testing performed on the hyperspectral imaging data show promising results with respect to the differentiation accuracy of the neural network. The accuracy of the neural network prediction depends on the quality and quantity of the image data provided. The goal is an in vivo application in clinically acceptable acquisition time.