A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks
arXiv (Cornell University)(2022)
摘要
Predictive coding networks are neuroscience-inspired models with roots in
both Bayesian statistics and neuroscience. Training such models, however, is
quite inefficient and unstable. In this work, we show how by simply changing
the temporal scheduling of the update rule for the synaptic weights leads to an
algorithm that is much more efficient and stable than the original one, and has
theoretical guarantees in terms of convergence. The proposed algorithm, that we
call incremental predictive coding (iPC) is also more biologically plausible
than the original one, as it it fully automatic. In an extensive set of
experiments, we show that iPC constantly performs better than the original
formulation on a large number of benchmarks for image classification, as well
as for the training of both conditional and masked language models, in terms of
test accuracy, efficiency, and convergence with respect to a large set of
hyperparameters.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要