Research / BFI Working PaperJul 06, 2022

On the Use of Outcome Tests for Detecting Bias in Decision Making

Ivan A. Canay, Magne Mogstad, Jack Mountjoy

The decisions of judges, lenders, journal editors, and other gatekeepers often lead to significant disparities across affected groups. An important question is whether, and to what extent, these group-level disparities are driven by relevant differences in underlying individual characteristics, or by biased decision makers. Becker (1957, 1993) proposed an outcome test of bias based on differences in post-decision outcomes across groups, inspiring a large and growing empirical literature. We carefully examine what researchers can learn about bias in decision making from such outcome tests. We show that models of decision making underpinning outcome tests can be usefully recast as Roy models, since potential outcomes enter directly into the decision maker’s choice equation. Different members of the Roy model family, however, are distinguished by the tightness of the link between potential outcomes and decisions, and we show that these distinctions have important implications for defining bias, deriving logically valid outcome tests of such bias, and identifying the marginal outcomes that the test requires.

More Research From These Scholars

BFI Working Paper May 1, 2018

Educational Assortative Mating and Household Income Inequality

Lasse Eika, Magne Mogstad, Basit Zafar
Topics:  Economic Mobility & Poverty
BFI Working Paper Jun 30, 2021

How Americans Respond to Idiosyncratic and Exogenous Changes in Household Wealth and Unearned Income

Mikhail Golosov, Michael Graber, Magne Mogstad, David Novgorodsky
Topics:  Economic Mobility & Poverty
BFI Working Paper Feb 11, 2018

Incarceration, Recidivism, and Employment

Manudeep Bhuller, Gordon B. Dahl, Katrine V. Løken, Magne Mogstad
Topics:  Uncategorized