Research / BFI Working PaperDec 02, 2021

Selection in Surveys

Deniz Dutz, Ingrid Huitfeldt, Santiago Lacouture, Magne Mogstad, Alexander Torgovitsky, Winnie van Dijk

We evaluate how nonresponse affects conclusions drawn from survey data and consider how researchers can reliably test and correct for nonresponse bias. To do so, we examine a survey on labor market conditions during the COVID-19 pandemic that used randomly assigned financial incentives to encourage participation. We link the survey data to administrative data sources, allowing us to observe a ground truth for participants and nonparticipants. We find evidence of large nonresponse bias, even after correcting for observable differences between participants and nonparticipants. We apply a range of existing methods that account for nonresponse bias due to unobserved differences, including worst-case bounds, bounds that incorporate monotonicity assumptions, and approaches based on parametric and nonparametric selection models. These methods produce bounds (or point estimates) that are either too wide to be useful or far from the ground truth. We show how these shortcomings can be addressed by modeling how nonparticipation can be both active (declining to participate) and passive (not seeing the survey invitation). The model makes use of variation from the randomly assigned financial incentives, as well as the timing of reminder emails. Applying the model to our data produces bounds (or point estimates) that are narrower and closer to the ground truth than the other methods.

More Research From These Scholars

BFI Working Paper Dec 31, 2022

Domestic Violence and the Mental Health and Well-being of Victims and Their Children

Manudeep Bhuller, Gordon B. Dahl, Katrine V. Løken, Magne Mogstad
Topics:  Uncategorized
BFI Working Paper Jan 7, 2020

ivmte: An R Package for Implementing Marginal Treatment Effect Methods

Joshua Shea, Alexander Torgovitsky
Topics:  Uncategorized
BFI Working Paper Jun 30, 2021

How Americans Respond to Idiosyncratic and Exogenous Changes in Household Wealth and Unearned Income

Mikhail Golosov, Michael Graber, Magne Mogstad, David Novgorodsky
Topics:  Economic Mobility & Poverty