Research / BFI Working PaperOct 21, 2019

Design and Analysis of Cluster-Randomized Field Experiments in Panel Data Settings

Bharat K. Chandar, Ali Hortaçsu, John List, Ian Muir, Jeffrey M. Wooldridge

Field experiments conducted with the village, city, state, region, or even country as the unit of randomization are becoming commonplace in the social sciences. While convenient, subsequent data analysis may be complicated by the constraint on the number of clusters in treatment and control. Through a battery of Monte Carlo simulations, we examine best practices for estimating unit-level treatment effects in cluster-randomized field experiments, particularly in settings that generate short panel data. In most settings we consider, unit-level estimation with unit fixed effects and cluster-level estimation weighted by the number of units per cluster tend to be robust to potentially problematic features in the data while giving greater statistical power. Using insights from our analysis, we evaluate the effect of a unique field experiment: a nationwide tipping field experiment across markets on the Uber app. Beyond the import of showing how tipping affects aggregate market outcomes, we provide several insights on aspects of generating and analyzing cluster-randomized experimental data when there are constraints on the number of experimental units in treatment and control.

More Research From These Scholars

BFI Working Paper Aug 19, 2020

The $100 Million Nudge: Increasing Tax Compliance of Businesses and the Self-Employed using a Natural Field Experiment

Justin E. Holz, John List, Alejandro Zentner, Marvin Cardoza, Joaquin Zentner
Topics:  Fiscal Studies
BFI Working Paper May 20, 2019

The Science of Using Science: Towards an Understanding of the Threats to Scaling Experiments

John List, Dana Suskind, Omar Al-Ubaydli
Topics:  Uncategorized
BFI Working Paper Dec 28, 2020

The Social Side of Early Human Capital Formation: Using a Field Experiment to Estimate the Causal Impact of Neighborhoods

John List, Fatemeh Momeni, Yves Zenou
Topics:  Early Childhood Education