Research / BFI Working PaperAug 30, 2021

Errors in Reporting and Imputation of Government Benefits and Their Implications

Pablo A. Celhay, Bruce Meyer, Nikolas Mittag

We document the extent, nature, and consequences of survey errors in cash welfare and SNAP receipt in three major U.S. household surveys. We find high rates of misreporting, particularly failure to report receipt. The surveys inaccurately capture patterns of multiple program participation, even though there is little evidence of program confusion. Error rates are higher among imputed observations, which account for a large share of false positive errors. Many household characteristics have significant effects on both false positives and false negative errors. Error rates sharply differ by race, ethnicity, income and other household characteristics. The errors greatly affect models of program receipt and estimated effects of income and race are noticeably biased. We examine error due to item non-response and imputation, as well as whether imputation improves estimates. Item non-respondents have higher receipt rates than the population conditional on covariates. The assumptions for consistent estimates in multivariate models fail both when excluding item non-respondents and when using the imputed values. In binary choice models of program receipt, linked data estimates favor excluding item non- respondents rather than using imputed values. Biases are well predicted by the error patterns we document, helping researchers make informed decisions on whether to use imputed values. 

More Research From These Scholars

BFI Working Paper Mar 28, 2022

Changes in the Distribution of Economic Well-Being during the COVID-19 Pandemic: Evidence from Nationally Representative Consumption Data

Bruce Meyer, Connacher Murphy, James X. Sullivan
Topics:  Economic Mobility & Poverty
BFI Working Paper Jan 20, 2022

What Leads to Measurement Errors? Evidence from Reports of Program Participation in Three Surveys

Pablo A. Celhay, Bruce Meyer, Nikolas Mittag
Topics:  Uncategorized
BFI Working Paper Sep 13, 2017

Consumption and Income Inequality in the U.S. Since the 1960s

Bruce Meyer, James X. Sullivan
Topics:  Economic Mobility & Poverty, Tax & Budget, Monetary Policy, Employment & Wages, Health care