Randomization Tests in Observational Studies with Staggered Adoption of Treatment

This paper considers the problem of inference in observational studies with time-varying adoption of treatment. In addition to an unconfoundedness assumption that the potential outcomes are independent of the times at which units adopt treatment conditional on the units’ observed characteristics, the main restriction underlying our analysis is that the time at which each unit adopts treatment follows a Cox proportional hazards model. This assumption permits the time at which each unit adopts treatment to depend on the observed characteristics of the unit, but imposes the restriction that the probability of multiple units adopting treatment at the same time is zero. In this context, we study Fisher-style randomization tests of a “sharp” null hypothesis that there is no treatment effect for all units and all time periods. We first show that an infeasible test that treats the parameters of the Cox model as known has rejection probability no greater than the nominal level in finite samples. We then establish that the feasible test that replaces these parameters with consistent estimators has limiting rejection probability no greater than the nominal level. These tests rely upon an important implication of the Cox model that provides a parametric expression for the probability that a particular unit is the first to adopt treatment conditional on both the observed characteristics and the time of first treatment. If these probabilities are equal across all units, then our testing procedure reduces to the randomization test proposed by Abadie et al. (2010). In a simulation study, we examine the practical relevance of our theoretical results, including robustness to misspecification of the model for the time at which each unit adopts treatment. Finally, we provide an empirical application of our methodology using the synthetic control-based test statistic and tobacco legislation data found in Abadie et al. (2010)