Constructing Temporal Abstractions Autonomously In Reinforcement Learning

The idea of temporal abstraction, that is, learning, planning, and representing the world at multiple time scales, has been a constant thread in AI research, spanning subfields from classical planning and search, to control and reinforcement learning. While temporal abstraction is a very natural concept, learning these abstractions without human input has proved quite daunting. In this paper, we present a general architecture called option-critic for learning temporal abstractions end to end from the agent's experience. This approach allows for continual learning and provides interesting qualitative and quantitative results in several tasks.