Nonasymptotic Regret Analysis of Adaptive Linear Quadratic Control with Model Misspecification
CoRR(2023)
摘要
The strategy of pre-training a large model on a diverse dataset, then
fine-tuning for a particular application has yielded impressive results in
computer vision, natural language processing, and robotic control. This
strategy has vast potential in adaptive control, where it is necessary to
rapidly adapt to changing conditions with limited data. Toward concretely
understanding the benefit of pre-training for adaptive control, we study the
adaptive linear quadratic control problem in the setting where the learner has
prior knowledge of a collection of basis matrices for the dynamics. This basis
is misspecified in the sense that it cannot perfectly represent the dynamics of
the underlying data generating process. We propose an algorithm that uses this
prior knowledge, and prove upper bounds on the expected regret after T
interactions with the system. In the regime where T is small, the upper
bounds are dominated by a term scales with either (log T) or
√(T), depending on the prior knowledge available to the learner. When T
is large, the regret is dominated by a term that grows with δ T, where
δ quantifies the level of misspecification. This linear term arises due
to the inability to perfectly estimate the underlying dynamics using the
misspecified basis, and is therefore unavoidable unless the basis matrices are
also adapted online. However, it only dominates for large T, after the
sublinear terms arising due to the error in estimating the weights for the
basis matrices become negligible. We provide simulations that validate our
analysis. Our simulations also show that offline data from a collection of
related systems can be used as part of a pre-training stage to estimate a
misspecified dynamics basis, which is in turn used by our adaptive controller.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要