Stick Roller: A Case Study on Efficiently Learning Precise Tactile Dynamics for In-hand Manipulation

Abstract In-hand manipulation is challenging in robotics due to the intricate contact dynamics and high degrees of control freedom. Precise manipulation with high accuracy often requires tactile perception, which adds further complexity to the system. This paper addresses these challenges in perception and control and proposes a framework for learning high-resolution tactile dynamics on real hardware that is feasible and scalable. Specifically, we use a case study on manipulating a small stick using the Allegro hand equipped with the Digit vision-based tactile sensor to demonstrate the framework's effectiveness. The framework includes an action space reduction module, tactile perception module, and learning with uncertainty module, all designed to operate in low data regimes. With minimal manually collected data, our learned contact dynamics model achieves grasp stability, sub-millimeter precision, and zero-shot generalizability across novel objects. The proposed framework demonstrates promising results for enabling precise in-hand manipulation with tactile feedback on real hardware.