Simulating the Survey of Professional Forecasters
Anne Lundgaard Hansen, John J. Horton, Sophia Kazinnik, Daniela Puzzello, Ali Zarifhonarvar
Status: Working paper
Last updated: 2024-12
Summary
We develop a framework for simulating professional expectations formation using large language models (LLMs). Combining novel hand-collected data on Survey of Professional Forecasters (SPF) participant characteristics with real-time macroeconomic data and lagged SPF median forecasts, we prompt LLMs to generate quarterly forecasts for 23 variables over 1999–2023. The resulting synthetic panel replicates key properties of the human survey, including forecast accuracy, median forecast levels, revision dynamics, and cross-sectional dispersion. Ablation exercises show that performance is largely driven by information conditioning: removing forecaster characteristics modestly worsens performance, whereas removing real-time data and especially lagged SPF medians leads to substantial error increases. Our framework offers a scalable complement to traditional surveys, enabling counterfactual analysis under alternative information sets, retrospective “as-of” forecasting, and rapid prototyping of survey designs.
Paper links
Cite
@unpublished{horton2024simulatingthesurveyofprofessionalforecasters,
title = {{Simulating the Survey of Professional Forecasters}},
author = {Hansen, Anne Lundgaard and Horton, John J. and Kazinnik, Sophia and Puzzello, Daniela and Zarifhonarvar, Ali},
year = {2024},
url = {https://john-joseph-horton.com/papers/simulating-the-survey-of-professional-forecasters/}
}