Journal article · 2026

Using prediction markets and forecasting surveys to predict 28 replication outcomes of classic articles in social psychology and judgement and decision making

Subramanya Prasad Chandrashekar, Domenico Viganola, Anna Dreber, Magnus Johannesson, Thomas Pfeiffer, Adam Siegel, Gilad Feldman

Abstract

Abstract Can researchers predict if classic findings published in the field of social psychology and judgement and decision making replicate? We set up prediction markets and a forecasting survey for predicting replications of 28 experiments of classic well-cited articles. Forecasters predicted if the original results would replicate, where a successful replication was defined as an effect in the same direction as the original and a signal (p-value lower than 0.05). Of the 28 original studies, 16 (57%) met the replication success criteria, compared to a predicted replication rate of 70% in the prediction markets and 65% replication rate in the survey. We concluded only suggestive evidence for associations of replication outcomes with prediction market prices (r = 0.43, 95% CI [0.07, 0.69]) and average survey beliefs (r = 0.26, 95% CI [–0.12, 0.58]). The prediction market effects were similar to observed effects in previous prediction market studies and suggest that prediction markets can to some extent predict replication outcomes, yet predictions are far from perfect and conducting replications is much more informative about the credibility of published findings. Data and code are available at https://doi.org/10.17605/OSF.IO/2KMH7.

BibTeX Copy

@article{chandrashekar_using_2026,
	title = {Using prediction markets and forecasting surveys to predict 28 replication outcomes of classic articles in social psychology and judgement and decision making},
	volume = {13},
	copyright = {http://creativecommons.org/licenses/by/4.0/},
	issn = {2054-5703},
	url = {https://royalsocietypublishing.org/rsos/article/13/1/250377/479652/Using-prediction-markets-and-forecasting-surveys},
	doi = {10.1098/rsos.250377},
	abstract = {Abstract
            Can researchers predict if classic findings published in the field of social psychology and judgement and decision making replicate? We set up prediction markets and a forecasting survey for predicting replications of 28 experiments of classic well-cited articles. Forecasters predicted if the original results would replicate, where a successful replication was defined as an effect in the same direction as the original and a signal (p-value lower than 0.05). Of the 28 original studies, 16 (57\%) met the replication success criteria, compared to a predicted replication rate of 70\% in the prediction markets and 65\% replication rate in the survey. We concluded only suggestive evidence for associations of replication outcomes with prediction market prices (r = 0.43, 95\% CI [0.07, 0.69]) and average survey beliefs (r = 0.26, 95\% CI [–0.12, 0.58]). The prediction market effects were similar to observed effects in previous prediction market studies and suggest that prediction markets can to some extent predict replication outcomes, yet predictions are far from perfect and conducting replications is much more informative about the credibility of published findings. Data and code are available at https://doi.org/10.17605/OSF.IO/2KMH7.},
	language = {en},
	number = {1},
	urldate = {2026-11-05},
	journal = {Royal Society Open Science},
	author = {Chandrashekar, Subramanya Prasad and Viganola, Domenico and Dreber, Anna and Johannesson, Magnus and Pfeiffer, Thomas and Siegel, Adam and Feldman, Gilad},
	month = jan,
	year = {2026},
	pages = {250377},
}