Reconciling Spatial and Temporal Abstractions for Goal Representation
CoRR(2024)
摘要
Goal representation affects the performance of Hierarchical Reinforcement
Learning (HRL) algorithms by decomposing the complex learning problem into
easier subtasks. Recent studies show that representations that preserve
temporally abstract environment dynamics are successful in solving difficult
problems and provide theoretical guarantees for optimality. These methods
however cannot scale to tasks where environment dynamics increase in complexity
i.e. the temporally abstract transition relations depend on larger number of
variables. On the other hand, other efforts have tried to use spatial
abstraction to mitigate the previous issues. Their limitations include
scalability to high dimensional environments and dependency on prior knowledge.
In this paper, we propose a novel three-layer HRL algorithm that introduces,
at different levels of the hierarchy, both a spatial and a temporal goal
abstraction. We provide a theoretical study of the regret bounds of the learned
policies. We evaluate the approach on complex continuous control tasks,
demonstrating the effectiveness of spatial and temporal abstractions learned by
this approach.
更多查看译文
关键词
Hierarchical Reinforcement Learning,Goal Representation,Reachability Analysis
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要