Over-parameterized networks have no bad basins [slides]
When Do Neural Networks Have No Bad Local Minima? (for ICML and NeurIPS’19 papers) [slides]
- DEED: A General Quantization Scheme for Communication Efficiency in Bits, Tian Ye, Peijun Xiao, Ruoyu Sun.
- Optimization for deep learning: theory and algorithms. Ruoyu Sun. (Survey paper). Preprint. My recent courses IE598 “optimization theory for deep learning” and “mathematics of deep learning” (at PKU appliied math summer school) are partially based on this article.
- Understanding Limitation of Two Symmeterized Orders by Worst-case Complexity, Peijun Xiao, Zhisheng Xiao, Ruoyu Sun. Submitted.
- Spurious Local Minima Exist for Almost All Over-parameterized Neural Networks;oo-version Tian Ding, Dawei Li, Ruoyu Sun. Preprint.
- Revisiting Landscape Analysis for Neural-networks: Eliminating Decreasing Paths to Infinity, Shiyu Liang, Ruoyu Sun, Srikant. Submitted.
- Over-Parameterized Deep Neural Networks Have No Strict Local Minima For Any Continuous Activations, Dawei Li, Tian Ding, Ruoyu Sun. Preprint.
- Designing a better global landscape for GAN. Under modification.
PUBLICATIONS (by Time)
The Global landscape of neural networks. To appear in IEEE Signal Processing Magzine, 2020.
Max-sliced Wasserstein distance for fast GAN training,
Deshpande, I., Hu, Y.T., Sun, R., Pyrros, A., Siddiqui, N., Koyejo, S., Zhao, Z., Forsyth, D. and Schwing, A.G., 2019. Max-Sliced Wasserstein Distance and Its Use for GANs. CVPR 2019, Oral (5.58%).
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization,
Xiangyi Chen, Sijia Liu, Ruoyu Sun, Mingyi Hong. Part of the paper has been accepted to ICLR 2019.
Adding One Neuron Can Eliminate All Bad Local Minima [slides],
Shiyu Liang, Ruoyu Sun, Jason Lee, R. Srikant. NeurIPS 2018.
Understanding the Loss Surface of Neural Networks for Binary Classification[slides],
Shiyu Liang, Ruoyu Sun, Yixuan Li, R. Srikant. Part of the paper has appeared at ICML 2018.
Worst-case Complexity of Cyclic Coordinate Descent: O(n^2) Gap with Randomized Version, [arxiv], Ruoyu Sun, Yinyu Ye. Accepted to Mathematical Programming (Series A), 2019.
Globally Optimal Uplink Joint Base Station Association and Beamforming,[arxiv],
Wei Liu, Ruoyu Sun (corresponding author), Zhi-Quan Luo. Accepted to IEEE Transactions on Communications 2019. Part of the paper has appeared at ICASSP 2014.
On the Efficiency of Random Permutation for ADMM and Coordinate Descent, [arxiv],
Ruoyu Sun, Zhi-Quan Luo, Yinyu Ye. Accepted to Mathematics of Operations Research, 2018. [video]
Previous version: On the Expected Convergence of Randomly Permuted ADMM
2nd Place, 2015 INFORMS George Nicholson student paper competition.
Oral talk, NIPS 2015 workshop on optimization for machine learning (workshop link)
- Guaranteed Matrix Completion via Nonconvex Factorization,[arxiv], [slides] [short summary]
Ruoyu Sun, Zhi-Quan Luo.
IEEE Transaction on Information Theory 2016; a shorter version has appeared at FOCS 2015. Honorable mention, 2015 INFORMS Optimization Society student paper prize.(prize page)
- Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems,
Ruoyu Sun, Mingyi Hong (equal contribution). NIPS 2015.
- Interference alignment via Feasible Point Pursuit, Aritra Konar, Ruoyu Sun, Nikos Sidiropoulos, Zhi-Quan Luo. Proc. IEEE SPAWC 2015.
- Joint Downlink Base Station Association and Power Control for Max-Min Fairness:Computation and Complexity,
Ruoyu Sun, Mingyi Hong, Zhi-Quan Luo.
IEEE Journal of Selected Areas in Communications (JSAC),vol.33, no.6, pp.1040-1054, June 2015. [link][arxiv]
- Interference Alignment using Finite and Dependent Channel Extensions: the Single Beam Case,
Ruoyu Sun, Zhi-Quan Luo. IEEE Trans. on Information Theory (TIT), vol. 61, no.1, pp.239-255, Jan. 2015. [link] [arxiv] [slides]
- Cross-Layer Provision of Future Cellular Networks: A WMMSE-based approach,
(alphabet order) Hadi Baligh, Mingyi Hong,Wei-Cheng Liao, Zhi-Quan Luo, Meisam Razaviyayn, Maziar Sanjabi, Ruoyu Sun. IEEE Signal Processing Magazine, vol.31, no.6, pp.56-68, Nov. 2014
- Long-term Transmit Point Associationfor Coordinated Multipoint Transmission by Stochastic Optimization,
Ruoyu Sun, Hadi Baligh, Zhi-Quan Luo. Proc. IEEE SPAWC 2013.
- Joint Base Station Clustering and Beamformer Design for Partial CoordinatedTransmission in Heterogenous Networks,
Mingyi Hong, Ruoyu Sun, Zhi-Quan Luo.
IEEE Journal on Selected Areas in Communications (JSAC), special issues on Large-Scale multiple antenna systems , vol. 31, no. 2, pp. 226-240, Feb. 2013. [link][arxiv]
- Optimal Joint Base Station Assignment and PowerAllocation in a Multiuser SISO Network,
Ruoyu Sun, Mingyi Hong, Zhi-Quan Luo. Proc. IEEE SPAWC 2012.
- Joint Transceiver Design and Base Station Clustering for Heterogeneous Networks,
Mingyi Hong, Meisam Razaviyayn, Ruoyu Sun and Zhi-Quan Luo. Proc. Asilomar Conference on Signals, Systems and Computers, 2012
- Robust SINR-Constrained MISO Downlink Beamforming: When is Semidefinite Programming Relaxation Tight?,
Enbin Song, Qingjiang Shi, Maziar Sanjabi, Ruoyu Sun, Zhi-Quan Luo.
EURASIP Journal on Wireless Communications and Networking, 2012. [link] Conference versionIEEE ICASSP, 2011.
- System and Method for Transmission Point (TP)Association and Beamforming Assignment in Heterogeneous Networks, Ruoyu Sun, Mingyi Hong, Hadi Baligh, Zhi-Quan Luo, and Meisam Razaviyayn. U.S. Patent App. 13/757,303, filed Feb. 2013.
- Matrix Completion via Nonconvex Factorization: Algorithms and Theory,
Ruoyu Sun, University of Minnesota, May 2015.
Advised master projects (selected)
- Myung-Hwan Song, TRAINABILITY AND GENERALIZATION OF SMALL-SCALE NEURAL NETWORKS, 2019
- Deborshi Goswami, Application of capsule networks for image classification on complex datasets, 2019
- Ziyu Zhou, Multi-Domain Image-to-Image Translation using StarGAN with Max Sliced Wasserstein Distance, 2019