A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertainty
ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)(2024)
摘要
Exploration is a significant challenge in practical reinforcement learning
(RL), and uncertainty-aware exploration that incorporates the quantification of
epistemic and aleatory uncertainty has been recognized as an effective
exploration strategy. However, capturing the combined effect of aleatory and
epistemic uncertainty for decision-making is difficult. Existing works estimate
aleatory and epistemic uncertainty separately and consider the composite
uncertainty as an additive combination of the two. Nevertheless, the additive
formulation leads to excessive risk-taking behavior, causing instability. In
this paper, we propose an algorithm that clarifies the theoretical connection
between aleatory and epistemic uncertainty, unifies aleatory and epistemic
uncertainty estimation, and quantifies the combined effect of both
uncertainties for a risk-sensitive exploration. Our method builds on a novel
extension of distributional RL that estimates a parameterized return
distribution whose parameters are random variables encoding epistemic
uncertainty. Experimental results on tasks with exploration and risk challenges
show that our method outperforms alternative approaches.
更多查看译文
关键词
Belief,Exploration,Uncertainty
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要