Nonconvex Optimization for Statistical Estimation and Learning: Beyond Smoothness and Convexity

DAMEK DAVIS – CORNELL UNIVERSITY 

ABSTRACT

How do we make sense of simple iterative algorithms — e.g., gradient descent — for large-scale optimization problems in the absence of classical assumptions, like smoothness and convexity? Such problems routinely arise, for example, in fitting neural networks, solving inverse problems in computational imaging, and in estimating statistical models under “adversarial perturbations.” In the talk, I will describe how one can answer several questions about these methods, such as whether they converge at all, how quickly they converge, whether they tend to saddle points and local minima, and whether we can provably accelerate them. In the process, we will encounter several mathematical tools in nonsmooth analysis, semi-algebraic geometry, and high dimensional probability and statistics.

Related Papers: 

• https://arxiv.org/abs/2205.00064
• https://link.springer.com/article/10.1007/s10208-018-09409-5
• https://epubs.siam.org/doi/10.1137/18M1178244
• https://arxiv.org/abs/2108.11832