The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks

NIPS 2020, 2020.

Cited by: 2|Views18
EI
Weibo:
While we mainly focused on two-layer fully-connected neural networks, we further provided theoretical and empirical evidence suggesting that this phenomenon continues to exist in more complicated models

Abstract:

Modern neural networks are often regarded as complex black-box functions whose behavior is difficult to understand owing to their nonlinear dependence on the data and the nonconvexity in their loss landscapes. In this work, we show that these common perceptions can be completely false in the early phase of learning. In particular, we fo...More

Code:

Data:

0
Your rating :
0

 

Tags
Comments