Higher-Order von Mises Expansions, Bagging, and Assumption-Lean Inference



The starting point of this work is the fact that models obtained from model selection procedures cannot be assumed to be correct. They therefore require assumption-lean inference and, in particular, assumption-lean standard errors. These can be obtained in two ways: (1) asymptotic plug-in, resulting in sandwich standard errors, and (2) bootstrap and, more specifically in regression, “pairs” or “x-y” bootstrap (as opposed to the residual bootstrap). Two questions arise: Is there a connection between the two types of standard errors? Is one superior to the other? We give some partial answers by showing that (a) asymptotic plug-in standard errors are a limiting case of bootstrap standard errors when the resample size varies, (b) bootstrap standard errors can be thought of as resulting from a bagging operation, (c) bagging results in statistical functionals that have finite von Mises expansions, and finally (d) bootstrap standard errors are “smoother” than sandwich standard errors in the sense of von Mises expansions. We conclude that bootstrap standard errors have some advantages over sandwich standard errors. However, we do not yet fully understand the trade-offs in choosing specific bootstrap resample sizes. Joint with: W.Stuetzle, L.Brown, R.Berk, L.Zhao, E.George, A.Kuchibhotla, K.Zhang