ReplicHAI › LLM simulation & replication prediction

Shared by Changkun Ou with everyone

Year ↓	Title added	Authors	Venue
2026	Human-AI Collaboration for Estimating Scientific Replicability	Chakravorti et al.	arXiv
2026	Predicting results of social science experiments using large language models	Ashokkumar et al.	Nature
2026	SciCoQA: Quality Assurance for Scientific Paper--Code Alignment	Baumgärtner, Gurevych	arXiv
2026	The threat of analytic flexibility in using large language models to simulate human data	Cummins	Advances in Methods and Practices in Psychological Science
2026	This human study did not involve human subjects: Validating LLM simulations as behavioral evidence	Hullman et al.	arXiv
2026	Using prediction markets and forecasting surveys to predict 28 replication outcomes of classic articles in social psychology and judgement and decision making	Chandrashekar et al.	Royal Society Open Science
2025	A large-scale replication of scenario-based experiments in psychology and management using large language models	Cui et al.	Nature Computational Science
2025	Identifying Non-Replicable Social Science Studies with Language Models	Saynova et al.	arXiv
2025	Large language models surpass human experts in predicting neuroscience results	Luo et al.	Nature Human Behaviour
2025	Large language models that replace human participants can harmfully misportray and flatten identity groups	Wang et al.	Nature Machine Intelligence
2025	Predicting the replicability of social and behavioural science claims in COVID-19 preprints	Marcoci et al.	Nature Human Behaviour
2025	Six Fallacies in Substituting Large Language Models for Human Participants	Lin	Advances in Methods and Practices in Psychological Science
2024	Can Generative AI improve social science?	Bail	Proceedings of the National Academy of Sciences
2024	Generative Agent Simulations of 1,000 People	Park et al.	arXiv
2024	The Illusion of Artificial Inclusion	Agnew et al.	Proceedings of the CHI Conference on Human Factors in Computing Systems
2024	Using Large Language Models to Create AI Personas for Replication, Generalization and Prediction of Media Effects: An Empirical Test of 133 Published Experimental Research Findings	Yeykelis et al.	arXiv
2023	A discipline-wide investigation of the replicability of Psychology papers over the past two decades	Youyou et al.	Proceedings of the National Academy of Sciences
2023	Can AI language models replace human participants?	Dillion et al.	Trends in Cognitive Sciences
2023	Out of One, Many: Using Language Models to Simulate Human Samples	Argyle et al.	Political Analysis
2023	Replicating replicability modeling of psychology papers	Mottelson, Kontogiorgos	Proceedings of the National Academy of Sciences
2023	Reply to Crockett et al. and Mottelson and Kontogiorgos: Machine learning’s scientific significance and future impact on replicability research	Youyou et al.	Proceedings of the National Academy of Sciences
2023	The limitations of machine learning models for predicting scientific replicability	Crockett et al.	Proceedings of the National Academy of Sciences
2023	Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies	Aher et al.	Proceedings of the 40th International Conference on Machine Learning
2020	Estimating the deep replicability of scientific findings using human and artificial intelligence	Yang et al.	Proceedings of the National Academy of Sciences
2019	Predicting the replicability of social science lab experiments	Altmejd et al.	PLOS ONE
2015	Using prediction markets to estimate the reproducibility of scientific research	Dreber et al.	Proceedings of the National Academy of Sciences

/ filter · j k or ↑ ↓ select · Enter open page · c copy BibTeX