Super Intendo: Semantic Robot Programming from Multiple Demonstrations for taskable robots.

When an end-user instructs a taskable robot on a new task, it is important for the robot to learn the user’s intention for the task. Knowing the user’s intention, represented as desired goal conditions, allows the robot to generalize across variations of the learned task seen at execution time. However, it has proven challenging to learn goal conditions due to the large, noisy, and complex space of goal conditions expressed by human users. This paper introduces Semantic Robot Programming with Multiple Demonstrations (SRP-MD) to learn a generative model of latent end-user task goal conditions from multiple end-user demonstrations in a shared workspace. By learning a generative model of the goal conditions, SRP-MD generalizes to task instances even when the quantity of objects to be arranged is not in the training set or novel object instances are included. At test time, a new goal is pulled from the learned generative model given the objects present in the initial scene. The efficacy of SRP-MD as a step toward taskable robots is shown on a Fetch robot learning and executing bin packing tasks in a simulated environment with grocery items.