Research / BFI Working PaperMar 13, 2023

Machine Learning as a Tool for Hypothesis Generation

While hypothesis testing is a highly formalized activity, hypothesis generation remains largely informal. We propose a procedure that uses machine learning algorithms—and their capacity to notice patterns people might not—to generate novel hypotheses about human behavior. We illustrate the procedure with a concrete application: judge decisions. We begin with a striking fact: up to half of the predictable variation in who judges jail is explained solely by the pixels in the defendant’s mugshot—that is, the predictions from an algorithm built using just facial images. We develop a procedure that allows human subjects to interact with this black-box algorithm to produce hypotheses about what in the face influences judge decisions. The procedure generates hypotheses that are both interpretable and novel: They are not explained by factors implied by existing research (demographics, facial features emphasized by previous psychology studies), nor are they already known (even if just tacitly) to people or even experts. Though these results are specific, our procedure is general. It provides a way to produce novel, interpretable hypotheses from any high-dimensional dataset (e.g. cell phones, satellites, online behavior, news headlines, corporate filings, and high-frequency time series). A central tenet of our paper is that hypothesis generation is in and of itself a valuable activity, and hope this encourages future work in this largely “pre-scientific” stage of science.

More Research From These Scholars

BFI Working Paper Feb 20, 2023

Automating Automaticity: How the Context of Human Choice Affects the Extent of Algorithmic Bias

Amanda Agan, Diag Davenport, Jens Ludwig, Sendhil Mullainathan
Topics:  Technology & Innovation
BFI Working Paper Jul 27, 2021

A Note on the Level of Customer Support by State Governments: A Mystery-Shopping Approach

Oeindrila Dube, Sendhil Mullainathan, Devin Pope
Topics:  Economic Mobility & Poverty
White Paper Jun 22, 2020

Actionable Policy Insights to Cut the Red Tape in the Face of COVID-19

Alison Alexander, Gabriel Prado Correa, Jaymal Patel, Boseong Yun
Topics:  COVID-19, Health care