Hierarchical Reinforcement Learning with Human-AI Collaborative Sub-Goals Optimization

Hierarchical reinforcement learning requires identifying relevant sub-goals to guide low-level decision-making, but this process can be time-consuming and challenging. Moreover, manually specifying sub-goals may introduce bias or mislead agents. To address these issues, we propose a collaborative human-AI algorithm that automatically optimizes candidate sub-goals and refines prior knowledge. Our algorithm can be integrated into various hierarchical frameworks and effectively prevent negative inferences that may arise from conflicting sub-goals. Our approach is robust in the face of different levels of human knowledge and able to accelerate convergence to optimal sub-goals and hierarchical policies.