Cloud-Based Probabilistic Knowledge Services for Instruction Interpretation

As the tasks of autonomous manipulation robots get more complex, the tasking of the robots using natural-language instructions becomes more important. Executing such instructions in the way they are meant often requires robots to infer missing, and disambiguate given information using lots of common and commonsense knowledge. In this work, we report on Probabilistic Action Cores (Prac) (Nyga and Beetz, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012) – a framework for learning of and reasoning about action-specific probabilistic knowledge bases that can be learned from hand-labeled instructions to address this problem. In Prac, knowledge about actions and objects is compactly represented by first-order probabilistic models, which are used to learn a joint probability distribution over the ways in which instructions for a given action verb are formulated. These joint probability distributions are then used to compute the plan instantiation that has the highest probability of producing the intended action given the natural language instruction. Formulating plan interpretation as a conditional probability is a promising approach because we can at the same time infer the plan that is most appropriate for performing the instruction, the refinement of the parameters of the plan on the basis of the information given in the instruction, and automatically fill in missing parameters by inferring their most probable value from the distribution. Prac has been implemented as a web-based online service on the cloud-robotics platform openEASE [7].