A note on regularised NTK dynamics with an application to PAC-Bayesian training
CoRR(2023)
摘要
We establish explicit dynamics for neural networks whose training objective
has a regularising term that constrains the parameters to remain close to their
initial value. This keeps the network in a lazy training regime, where the
dynamics can be linearised around the initialisation. The standard neural
tangent kernel (NTK) governs the evolution during the training in the
infinite-width limit, although the regularisation yields an additional term
appears in the differential equation describing the dynamics. This setting
provides an appropriate framework to study the evolution of wide networks
trained to optimise generalisation objectives such as PAC-Bayes bounds, and
hence potentially contribute to a deeper theoretical understanding of such
networks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要