3D semantic map-based shared control for smart wheelchair

The previous perception and control system of smart wheelchairs normally doesn't distinguish different objects and treats all objects as obstacles. Consequently it is hard to realize the object related navigation tasks such as furniture docking or door passage with interference from the obstacle avoidance behavior. In this article, a local 3D semantic map is built online using a low-cost RGB-D camera, which provides the semantic and geometrical data of the recognized objects to the shared control modules for user intention estimation, target selection, motion control, as well as parameters adjusting of weight optimization for addressing different target. With the object information provided by 3D semantic map, our control system can choose different behaviors according to user intention to implement object related navigation. A smart wheelchair prototype equipped with a Kinect is developed and tested in real environment. The experiments showed that the 3D semantic map-based shared control can effectively enhance the smart wheelchair's mobility, and improve the collaboration between the user and the smart wheelchair.