Adaptive Methods for Nonconvex Optimization

    Sashank J. Reddi
    Sashank J. Reddi
    Devendra Singh Sachan
    Devendra Singh Sachan

    NeurIPS, pp. 9815-9825, 2018.

    Cited by: 5|Bibtex|Views3|Links
    EI
    Keywords:
    deep learningnonconvex optimizationconstant factorconvex optimizationsquare root

    Abstract:

    Adaptive gradient methods that rely on scaling gradients down by the square root of exponential moving averages of past squared gradients, such RMSProp, Adam, Adadelta have found wide application in optimizing the nonconvex problems that arise in deep learning. However, it has been recently demonstrated that such methods can fail to conve...More

    Code:

    Data:

    Your rating :
    0

     

    Tags
    Comments