Visual-RRT: Integrating IBVS as a steering method in an RRT planner

In this paper, we present a new approach to robot motion planning that anticipates the use of visionbased feedback control during task execution. We accomplish this by incorporating an image-based visual servo (IBVS) controller directly into the steering function used by a Rapidly Exploring Random Tree (RRT) planner. Our approach requires a number of extensions to traditional RRT-style planning. First, we derive a new sampling strategy that augments the usual state information by including image features that will be used by the IBVS control law. These augmented samples are then used by our new IBVS steering function, which simulates an IBVS control law to generate local trajectories that extend the current tree. These trajectories must be validated to ensure that they are collision-free and that all image features remain unoccluded and within the camera field of view throughout the local trajectory. We also provide a formal proof showing that the proposed approach is probabilistically complete. We have applied our approach to the problem of planning trajectories for three different systems: a robotic arm, an unmanned aerial vehicle (UAV) and a car-like robot, which are equipped with an IBVS control law. We explore performance trade-offs in the control design via simulation studies and demonstrate real-world effectiveness via experiments in which a small-scale car-like robot uses IBVS to navigate a track that includes a number of obstacles and potential occlusions. By exploring performance trade-offs, we mean that several elements, such as the metric used to identify nearest neighbors in the RRT and the steering method used to generate nodes, are tested and compared. (c) 2023 Elsevier B.V. All rights reserved.