OPTIMIZATION LANDSCAPE – FROM MATRIX COMPLETION TO NEURAL NETWORKS
RONG GE – DUKE UNIVERSITY
Many popular machine learning tools, from more traditional tools such as matrix completion to modern neural networks, rely on optimizing non-convex objectives. In practice these objectives are optimized by simple algorithms such as gradient descent. In this talk, I will briefly discuss our recent work in understanding why simple algorithms can optimize non-convex objectives. In particular, we study the optimization landscape of this objectives, and show that surprisingly many objectives (including matrix completion and a special version of neural network) in practice are simple in the sense that they do not have any bad local minima.