Beyond PCA: Statistical and Computational Limits of Higher-Order Spectral Methods

A person wearing glasses and a dark turtleneck sweater stands with arms crossed, smiling, in front of large green plant leaves.
MING YUAN – COLUMBIA UNIVERSITY 

ABSTRACT

Spectral methods are fundamental tools in statistics, underlying PCA, factor models, and many approaches to latent structure recovery. But oftentimes second-order information is not enough, and the relevant structure appears only in higher-order moments. In this talk, I will use tensor spectral methods to explore this question, with independent component analysis as a motivating example. Higher-order structure leads to several phenomena that differ sharply from familiar matrix theory: individual components can remain identifiable even without spectral gaps, and they can often be estimated much more accurately than the full higher-order moment itself. These properties also open the door to statistical inference for the recovered components. On the computational side, simple iterative algorithms can achieve optimal statistical accuracy once properly initialized, but finding a good initialization can be substantially harder. This creates a fundamental gap between what is statistically possible and what can be achieved efficiently. The talk will highlight connections among PCA, ICA, high-dimensional estimation, and computation, and illustrate how moving beyond second-order methods can change both the statistical and computational landscape.