Hyperbot-A Benchmarking Testbed For Acquisition Of Robot-Centric Hyperspectral Scene And In-Hand Object Data

Robots will benefit from identifying novel objects in their environments through multi-modal sensing capabilities. The overarching goal of this research is to accelerate multi-modal sensor data collection for general-purpose robots to infer material properties of objects they interact with. To this end, we designed a benchmarking testbed to enable a robot manipulator to perceive spectral and spatial characteristics of scene items. Our design includes the use of a push broom Visible to Near Infrared (VNIR) hyperspectral camera, co-aligned with a depth camera. This system enables the robot to process and segment spectral characteristics of items in a larger spatial scene. For more targeted item manipulation, we integrate a VNIR spectrometer into the fingertips of a gripper. By acquiring spectral signatures both at a distance and at grasp time, the robot can quickly correlate data from the two sensors, each of which contain distinct quantum efficiencies and noise. Our approach to this challenge is a step towards using spectral data for enhanced grasp selection in cluttered environments and automated ground-truthing of hyperspectral sensor data. This paper describes our approach to the design of this benchmarking testbed. The project code and material list are located here: https://github.com/RIVeR-Lab/HyperBot.