Few-shot Quality-Diversity Optimization
IEEE ROBOTICS AND AUTOMATION LETTERS(2021)
摘要
In the past few years, a considerable amount of research has been dedicated
to the exploitation of previous learning experiences and the design of Few-shot
and Meta Learning approaches, in problem domains ranging from Computer Vision
to Reinforcement Learning based control. A notable exception, where to the best
of our knowledge, little to no effort has been made in this direction is
Quality-Diversity (QD) optimization. QD methods have been shown to be effective
tools in dealing with deceptive minima and sparse rewards in Reinforcement
Learning. However, they remain costly due to their reliance on inherently
sample inefficient evolutionary processes. We show that, given examples from a
task distribution, information about the paths taken by optimization in
parameter space can be leveraged to build a prior population, which when used
to initialize QD methods in unseen environments, allows for few-shot
adaptation. Our proposed method does not require backpropagation. It is simple
to implement and scale, and furthermore, it is agnostic to the underlying
models that are being trained. Experiments carried in both sparse and dense
reward settings using robotic manipulation and navigation benchmarks show that
it considerably reduces the number of generations that are required for QD
optimization in these environments.
更多查看译文
关键词
Optimization, Task analysis, Statistics, Sociology, Training, Reinforcement learning, Navigation, Evolutionary robotics, reinforcement learning, transfer learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要