Advancing Object Goal Navigation Through LLM-enhanced Object Affinities Transfer
arxiv(2024)
摘要
In object goal navigation, agents navigate towards objects identified by
category labels using visual and spatial information. Previously, solely
network-based methods typically rely on historical data for object affinities
estimation, lacking adaptability to new environments and unseen targets.
Simultaneously, employing Large Language Models (LLMs) for navigation as either
planners or agents, though offering a broad knowledge base, is cost-inefficient
and lacks targeted historical experience. Addressing these challenges, we
present the LLM-enhanced Object Affinities Transfer (LOAT) framework,
integrating LLM-derived object semantics with network-based approaches to
leverage experiential object affinities, thus improving adaptability in
unfamiliar settings. LOAT employs a dual-module strategy: a generalized
affinities module for accessing LLMs' vast knowledge and an experiential
affinities module for applying learned object semantic relationships,
complemented by a dynamic fusion module harmonizing these information sources
based on temporal context. The resulting scores activate semantic maps before
feeding into downstream policies, enhancing navigation systems with
context-aware inputs. Our evaluations in AI2-THOR and Habitat simulators
demonstrate improvements in both navigation success rates and efficiency,
validating the LOAT's efficacy in integrating LLM insights for improved object
goal navigation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要