Using touch sensor and vision feedback to adapt skewering strategy for robust assistive feeding

Assistive feeding using a robotic arm can help people become more independent and reduce caregiver burden. The ability to skewer different types of food is a key requirement for assistive feeding. State-of-the-art methods have shown promising results by learning food skewering strategies based on vision and force-torque sensor feedback. However, force-torque sensors are expensive and have a lengthy and complicated fabrication process. In this work, we demonstrate how MWCNT/PDMS-based tactile sensor arrays, developed by Leong Research Group, which are much cheaper and easy to fabricate, can be used for learning food skewering strategies. We first show how the sensors can be calibrated to be sensitive to different food textures. We then create a touch sensor and vision feedback dataset for skewering different foods and show that a neural network trained on this dataset can learn food skewering strategies with much better success than naive skewering strategies and is comparable to the skewering strategies learned using force-torque sensors. Overall, our system presents a much cheaper alternative for assistive feeding while providing similar accuracy.