Visual Action Planning with Multiple Heterogeneous Agents
arxiv(2024)
摘要
Visual planning methods are promising to handle complex settings where
extracting the system state is challenging. However, none of the existing works
tackles the case of multiple heterogeneous agents which are characterized by
different capabilities and/or embodiment. In this work, we propose a method to
realize visual action planning in multi-agent settings by exploiting a roadmap
built in a low-dimensional structured latent space and used for planning. To
enable multi-agent settings, we infer possible parallel actions from a dataset
composed of tuples associated with individual actions. Next, we evaluate
feasibility and cost of them based on the capabilities of the multi-agent
system and endow the roadmap with this information, building a capability
latent space roadmap (C-LSR). Additionally, a capability suggestion strategy is
designed to inform the human operator about possible missing capabilities when
no paths are found. The approach is validated in a simulated burger cooking
task and a real-world box packing task.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要