Neural Network Optimization: Global Landscape, Local Structure, and Algorithms Permalink
Plenary Talk, SIAM Conference on Optimization (OP26), Edinburgh, UK
Plenary talk at the SIAM Conference on Optimization (OP26), Edinburgh, UK, June 2-5, 2026.
Talks and Slides
Properties of the Hessian Matrix of Neural Networks and Its Applications in Algorithm Analysis Permalink
Invited Talk, FAI, Shanghai, China
Part I: why Transformers need Adam \u2014 from a Hessian perspective, leading to the Adam-mini algorithm. Part II: on the special (near block-diagonal) structure of the Hessian of neural networks.
Converge or Diverge? A Story of Adam Permalink
Invited Talk, CAS,
Invited talk at the Feng Kang Symposium on Scientific Computing (CAS). This talk spans our series of results on Adam convergence, the Adam-mini and Hessian-diagonal line of work, and the over-parameterized-network series.