TESTING WEAK NULLS IN PAIRED OBSERVATIONAL STUDIES

COLIN B. FOGARTY

ABSTRACT

A fundamental limitation of causal inference in observational studies is that perceived evidence for a treatment effect might instead be explained by factors not accounted for in the primary analysis. Methods for assessing the sensitivity of a study’s conclusions to unmeasured confounding have been devised for testing sharp null hypotheses, most commonly that the treatment effect is constant across all individuals. It has been argued that certain patterns of hidden bias may be directly attributable to the existence of heterogeneous treatment effects, stoking fear that sensitivity analyses assuming constant effects may be inadequate. We present a new method for sensitivity analysis for the sample average treatment effect in paired observational studies while leaving the individual-level treatment effects unspecified. Our recommended procedure relates recent work on robust permutation tests to observational studies, where randomizations are no longer drawn uniformly. The method naturally extends conventional modes of inference for the sample average treatment effect in paired experiments to the case of unknown, but bounded, probabilities of assignment to treatment. We further assess the performance of existing methods for sensitivity analysis when the assumption of effect homogeneity is violated. In so doing, we illustrate that concerns about certain sensitivity analyses operating under the presumption of constant effects are largely unwarranted.