Beyond Lengthscales: No-regret Bayesian Optimisation With Unknown Hyperparameters Of Any Type
CoRR(2024)
摘要
Bayesian optimisation requires fitting a Gaussian process model, which in
turn requires specifying hyperparameters - most of the theoretical literature
assumes those hyperparameters are known. The commonly used maximum likelihood
estimator for hyperparameters of the Gaussian process is consistent only if the
data fills the space uniformly, which does not have to be the case in Bayesian
optimisation. Since no guarantees exist regarding the correctness of
hyperparameter estimation, and those hyperparameters can significantly affect
the Gaussian process fit, theoretical analysis of Bayesian optimisation with
unknown hyperparameters is very challenging. Previously proposed algorithms
with the no-regret property were only able to handle the special case of
unknown lengthscales, reproducing kernel Hilbert space norm and applied only to
the frequentist case. We propose a novel algorithm, HE-GP-UCB, which is the
first algorithm enjoying the no-regret property in the case of unknown
hyperparameters of arbitrary form, and which supports both Bayesian and
frequentist settings. Our proof idea is novel and can easily be extended to
other variants of Bayesian optimisation. We show this by extending our
algorithm to the adversarially robust optimisation setting under unknown
hyperparameters. Finally, we empirically evaluate our algorithm on a set of toy
problems and show that it can outperform the maximum likelihood estimator.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要