Scientific studies with human subjects often suffer from low and unequal participation rates across socioeconomic and demographic groups. Low participation rates mean there is a lot of “missing data”, leaving considerable room for unobserved differences between participants and non-participants to affect conventional estimates of population means. Inequality in participation rates can similarly cause bias and skew policy decisions away from achieving their intended goal. Survey estimates are used to allocate federal funds and other governmental resources in areas ranging from public health and education to housing, and to infrastructure. Hence, lower participation rates among low-income and minority groups may skew such decisions to their disadvantage.
Scientific studies that aim to survey a specific population exhibit non-participation for a number of reasons, including whether researchers are able to contact certain households (non-contact), or whether a contacted household believes that the costs of participating exceed the benefits (hesitancy). A challenge for researchers working to understand why it is difficult to recruit study participants is that participation data only reveal who does not participate, not why they don’t.
The distinction matters. In the case described in this new research, a lack of representation from Black, Hispanic, and low socioeconomic status households poses a risk to public health and a challenge for policymakers responding to COVID-19. If we don’t know why these households don’t participate, we cannot effectively encourage greater participation and, thus, improve health outcomes.
This paper addresses this knowledge gap by employing data from the Representative Community Survey Project’s (RECOVER) COVID-19 serological study, which experimentally varied financial incentives for participation. The study was conducted on Chicago households who were sent a package containing a self-administered blood sample collection kit, and were asked to return the sample by mail to a partner research lab to test for COVID-19 antibodies. Households in the sample were randomly assigned one of three levels of financial compensation: $0, $100, or $500.
The RECOVER study indeed saw that households with a high share of minorities and low-income households are underrepresented at lower incentives. For example, in the unincentivized arm, only 2% of households in high poverty neighborhoods participate, compared to 10% in low poverty areas. It is important to note that there are many other examples where underrepresentation matters. One prominent case beyond pandemic health policy concerns the 2020 US Census, where issues have been raised about under-counting Hispanic, Black, and Native American residents.¹
Please see the working paper for details, but broadly described, the authors develop a framework that uses experimentally induced variation in financial compensation for participation, along with a model of participation behavior, to separately identify and estimate the relative importance of non-contact and hesitancy for non-participation. They find the following:
- Financial compensation has a powerful effect on participation: the $100 incentive increases participation from 6% to 17%, and the $500 incentive increases it to 29%.
- The $100 incentive substantially increases participation among all groups, but widens differences in participation rates, while the $500 incentive increases participation further and, more importantly, it entirely closes the gap in participation.
- Both non-contact and hesitancy are key drivers of low participation.
- Underrepresentation occurs because poor and minority households are more hesitant and have higher perceived costs of participation, and not because they are harder to reach.
- For example, 61% of contacted households in majority minority neighborhoods would not participate for $100, compared to only 14% in majority White neighborhoods. Hesitancy explains 89% of the participation gap at $0, and 93% at $100.
Bottom line: This work offers valuable insights for policymakers about the quality of serological studies, where low participation rates can affect health outcomes, and about population surveys more generally. A better understanding of participation among racial and ethnic minorities, and households with lower incomes, offers the promise of better health and policy outcomes for all.