Research / BFI Working PaperSep 18, 2023

Generation Next: Experimentation with AI

Gary Charness, Brian Jabarian, John List

We investigate the potential for Large Language Models (LLMs) to enhance scientific practice within experimentation by identifying key areas, directions, and implications. First, we discuss how these models can improve experimental design, including improving the elicitation wording, coding experiments, and producing documentation. Second, we discuss the implementation of experiments using LLMs, focusing on enhancing causal inference by creating consistent experiences, improving comprehension of instructions, and monitoring participant engagement in real time. Third, we highlight how LLMs can help analyze experimental data, including pre-processing, data cleaning, and other analytical tasks while helping reviewers and replicators investigate studies. Each of these tasks improves the probability of reporting accurate findings.

More Research From These Scholars

BFI Working Paper Aug 29, 2023

Stress Testing Structural Models of Unobserved Heterogeneity: Robust Inference on Optimal Nonlinear Pricing

Aaron Bodoh-Creed, Brent Hickman, John List, Ian Muir, Gregory Sun
Topics:  Uncategorized
BFI Working Paper Aug 19, 2020

The $100 Million Nudge: Increasing Tax Compliance of Businesses and the Self-Employed using a Natural Field Experiment

Justin E. Holz, John List, Alejandro Zentner, Marvin Cardoza, Joaquin Zentner
Topics:  Fiscal Studies
BFI Working Paper Aug 23, 2022

The Impact of Team Incentives on Performance in Graduate School: Evidence from Two Pilot RCTs

John List, Rohen Shah
Topics:  Higher Education & Workforce Training