Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced

ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018), pp. 384-395, 2018.

Cited by: 43|Views43
EI

Abstract:

We study the implicit regularization imposed by gradient descent for learning multi-layer homogeneous functions including feed-forward fully connected and convolutional deep neural networks with linear, ReLU or Leaky ReLU activation. We rigorously prove that gradient flow (i.e. gradient descent with infinitesimal step size) effectively en...More

Code:

Data:

Your rating :
0

 

Tags
Comments