Converge or Diverge? A Story of Adam

Invited talk at the Feng Kang Symposium on Scientific Computing (CAS). This talk spans our series of results on Adam convergence, the Adam-mini and Hessian-diagonal line of work, and the over-parameterized-network series.

Slides (PDF)