Tactile-based Object Retrieval from Granular Media

We introduce GEOTACT, a robotic manipulation method capable of retrievingobjects buried in granular media. This is a challenging task due to the need tointeract with granular media, and doing so based exclusively on tactilefeedback, since a buried object can be completely hidden from vision. Tactilefeedback is in itself challenging in this context, due to ubiquitous contactwith the surrounding media, and the inherent noise level induced by the tactilereadings. To address these challenges, we use a learning method trainedend-to-end with simulated sensor noise. We show that our problem formulationleads to the natural emergence of learned pushing behaviors that themanipulator uses to reduce uncertainty and funnel the object to a stable graspdespite spurious and noisy tactile readings. We also introduce a trainingcurriculum that enables learning these behaviors in simulation, followed byzero-shot transfer to real hardware. To the best of our knowledge, GEOTACT isthe first method to reliably retrieve a number of different objects from agranular environment, doing so on real hardware and with integrated tactilesensing. Videos and additional information can be found athttps://jxu.ai/geotact.