OPTIMIZATION LANDSCAPE – FROM MATRIX COMPLETION TO NEURAL NETWORKS

RONG GE – DUKE UNIVERSITY

ABSTRACT

Many popular machine learning tools, from more traditional tools such as matrix completion to modern neural networks, rely on optimizing non-convex objectives. In practice these objectives are optimized by simple algorithms such as gradient descent. In this talk, I will briefly discuss our recent work in understanding why simple algorithms can optimize non-convex objectives. In particular, we study the optimization landscape of this objectives, and show that surprisingly many objectives (including matrix completion and a special version of neural network) in practice are simple in the sense that they do not have any bad local minima.