Using agents in virtual environments to assist controllers to manage multiple assets

Search and rescue operations often require complex coordination of a range of resources, including human and robotic resources. This paper discusses a proposed new framework that allows agent technology to be used in conjunction with a virtual environment to provide a human controller with an effective visualisation of the distribution of a collection of autonomous objects, in our case, Unmanned Aerial Vehicles (UAVs) so that they can be managed in a way that allows them to successfully complete the task in the minimum possible time. It is our contention that to do this effectively there needs to be two-way initiation of verbal conversations, but that it is not necessary for the system to completely understand the conversations required. An example scenario is presented that illustrates how such a system would be used in practice, illustrating how a single human can communicate with a swarm of semi-autonomous actors verbally and envisage their activities in a swarm based on the visual cues provided within the virtual environment. An agent-based solution is proposed that meets the requirements and provides a command station that can manage a search using a collection of UAVs effectively.