Camera Configuration Models for Machine Vision Based Force Estimation in Robot-Assisted Soft Body Manipulation

Robotic proxies can enable precise, repeatable and intelligent medical or therapeutic procedures involving direct physical contact with human bodies. Some of these interactions include but are not limited to palpation, ultrasound imaging, and delivery of medication or treatment through magnetically anchored systems. Contact forces with the deformable manipulation target are oftentimes task critical. In the absence of direct force measurement or accurate torque sensing, other sensing modalities like vision may be used. This paper explores the use of multiple streaming camera views of deformable object manipulation to infer joint-level torques. Combinations of eye-in-hand, eye-to-hand and robot pose and velocities from soft body manipulation tasks are used as inputs to a modular recurrent neural network structure to estimate joint torques. Results are promising and indicate that with the trained models that even in the absence of robot state information, the use of multiple camera viewpoints can provide comparable results. These developments can bolster vision-based force estimation for systems that provide care in lieu of a human or teleoperatively extend expert skill to otherwise inaccessible areas or populations.