Observation and Execution.

Assistive robotic systems in household or industrial production environments get more and more capable of performing also complex tasks which previously only humans were able to do. As robots are often equipped with two arms and hands, similar manipulations can be executed. The robust programming of such devices with a very large number of degrees of freedom (DOFs) compared with single industrial robot arms however is laborious if done joint-wise. Two major directions to overcome this problem have been previously proposed. The programming by demonstration (PbD) approach, where human arm and recently also hand motions are tracked, segmented and re-executed in an adaptive way on the robotic system and the high-level planning approach which tries to generate a task sequence on a logical level and attributes geometric information as necessary to generate artificial trajectories to solve the task. Here we propose to combine the best of both worlds. For the very complex motion generation for a robotic hand, a rather direct approach to assign manipulation actions from human demonstration to a human hand is taken. For the combination of different basic manipulation actions the task constraints are segmented from the demonstration action and used to generate a task oriented plan. This plan is validated against the robot kinematic and geometric constraints and then a geometric motion planner can generate the necessary robot motions to fulfill the task execution on the system.