What Can Conformal Inference Offer to Statistics?

Lihua Lei – Stanford University


Valid uncertainty quantification is crucial for high-stakes decision-making. Conformal inference provides a powerful framework that can wrap around any black-box prediction algorithm, like random forests or deep neural networks, and generate prediction intervals with distribution-free coverage guarantees. In this talk, I will describe how conformal inference can be adapted to handle more complicated inferential tasks in statistics.

I will mainly focus on two important statistical problems: counterfactual inference and time-to-event analysis. In practice, the former can be used as a building block to infer individual treatment effects, and the latter can be applied for individual risk assessment. Unlike standard prediction problems, the predictive targets are only partially observable owing to selection and censoring. When the missing data mechanism is known, as in randomized experiments, our conformal inference-based approaches achieve desired coverage in finite samples without any assumption on the conditional distribution of the outcomes or the accuracy of the predictive algorithm; when the missing data mechanism is unknown, they satisfy a doubly robust guarantee of coverage. We demonstrate on both simulated and real datasets that conformal inference-based methods provide more reliable uncertainty quantification than other popular methods, which suffer from a substantial coverage deficit even in simple models. In addition, I will also briefly mention my work on adapting and generalizing conformal inference to other statistical problems, including election, outlier detection, and risk-calibrated predictions.

Links to the papers:

75% of the talk is on this paper (https://arxiv.org/abs/2006.06138), 15% of the talk is on this paper (https://arxiv.org/abs/2103.09763), and in the remaining time of the talk, I will very briefly touch upon three other papers (one slide for each, https://arxiv.org/abs/2104.08279https://arxiv.org/abs/2101.02703https://arxiv.org/abs/2110.01052).