ReplicHAI

Shared by Changkun Ou with everyone

Year ↓	Title added	Authors	Venue
2022	Apéritif: Scaffolding Preregistrations to Automatically Generate Analysis Code and Methods Descriptions	Pang et al.	CHI Conference on Human Factors in Computing Systems
2022	Benchmarking the Spectrum of Agent Capabilities	Hafner	arXiv
2022	Competition-Level Code Generation with AlphaCode	Li et al.	Science
2022	Content Transfer Across Multiple Screens with Combined Eye-Gaze and Touch Interaction — A Replication Study	Biener, Grubert	2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)
2022	Digital Proxemics: Designing Social and Collaborative Interaction in Virtual Environments	Williamson et al.	CHI Conference on Human Factors in Computing Systems
2022	Do Open-Science Badges Increase Trust in Scientists Among Undergraduates, Scientists, and the Public?	Schneider et al.	Psychological Science
2022	Do People Use Games to Compensate for Psychological Needs During Crises? A Mixed-Methods Study of Gaming During COVID-19 Lockdowns	Ballou et al.	CHI Conference on Human Factors in Computing Systems
2022	DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation	Lai et al.	arXiv
2022	Gesture Elicitation as a Computational Optimization Problem	Tsandilas, Dragicevic	CHI Conference on Human Factors in Computing Systems
2022	Is Economics Research Replicable? Sixty Published Papers From Thirteen Journals Say “Often Not”	Chang, Li	Critical Finance Review
2022	Juvenile Graphical Perception: A Comparison between Children and Adults	Panavas et al.	CHI Conference on Human Factors in Computing Systems
2022	Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement	Klein et al.	Collabra: Psychology
2022	MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge	Fan et al.	arXiv
2022	Multifractal Mice: Operationalising Dimensions of Readiness-to-hand via a Feature of Hand Movement	Bennett et al.	CHI Conference on Human Factors in Computing Systems
2022	Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty	Breznau et al.	Proceedings of the National Academy of Sciences
2022	Preferences and Effectiveness of Sleep Data Visualizations for Smartwatches and Fitness Bands	Islam et al.	CHI Conference on Human Factors in Computing Systems
2022	Recommendations for Visualization Recommendations: Exploring Preferences and Priorities in Public Health	Bao et al.	CHI Conference on Human Factors in Computing Systems
2022	Remote Research on Locomotion Interfaces for Virtual Reality: Replication of a Lab-Based Study on Teleporting Interfaces	Kelly et al.	IEEE Transactions on Visualization and Computer Graphics
2022	Replicability, Robustness, and Reproducibility in Psychological Science	Nosek et al.	Annual Review of Psychology
2022	Round Numbers Can Sharpen Cognition	Nguyen et al.	CHI Conference on Human Factors in Computing Systems
2022	ScienceWorld: Is your Agent Smarter than a 5th Grader?	Wang et al.	arXiv
2022	The past, present and future of Registered Reports	Chambers, Tzavella	Nature Human Behaviour
2022	The Placebo Effect of Artificial Intelligence in Human–Computer Interaction	Kosch et al.	ACM Transactions on Computer-Human Interaction
2022	Trading Accuracy for Enjoyment? Data Quality and Player Experience in Data Collection Games	Gundry, Deterding	CHI Conference on Human Factors in Computing Systems
2022	Understanding Cultural Influence on Perspectives Around Contact Tracing Strategies	Lu et al.	Proceedings of the ACM on Human-Computer Interaction
2022	VocabulARy Replicated: Comparing Teenagers to Young Adults	Weerasinghe et al.	2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)
2021	A Chinese-Language Validation of the Video Game Demand Scale (VGDS-C): Measuring the Cognitive, Emotional, Physical, and Social Demands of Video Games	Bowman et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	ALFWorld: Aligning Text and Embodied Environments for Interactive Learning	Shridhar et al.	arXiv
2021	An Excess of Positive Results: Comparing the Standard Psychology Literature with Registered Reports	Scheel et al.	Advances in Methods and Practices in Psychological Science
2021	Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: an observational study	Hardwicke et al.	Royal Society Open Science
2021	Are You Open? A Content Analysis of Transparency and Openness Guidelines in HCI Journals	Ballou et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Are You Open? A Content Analysis of Transparency and Openness Guidelines in HCI Journals	Ballou et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Can Anthropographics Promote Prosociality?A Review and Large-Sample Study	Morais et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Challenges for assessing replicability in preclinical cancer biology	Errington et al.	eLife
2021	Consensus-based guidance for conducting and reporting multi-analyst studies	Aczel et al.	eLife
2021	Data-Driven Mark Orientation for Trend Estimation in Scatterplots	Liu et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Distress Disclosure across Social Media Platforms during the COVID-19 Pandemic: Untangling the Effects of Platforms, Affordances, and Audiences	Zhang et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Evaluating Large Language Models Trained on Code	Chen et al.	arXiv
2021	Flow Encourages Task Focus, but Frustration Drives Task Switching: How Reward and Effort Combine to Influence Player Engagement in a Simple Video Game	Bowman et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	From FOMO to JOMO: Examining the Fear and Joy of Missing Out and Presence in a 360° Video Viewing Experience	Aitamurto et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Investigating the replicability of preclinical cancer biology	Errington et al.	eLife
2021	Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparison	Wu et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	lm-evaluation-harness	EleutherAI
2021	Measuring Coding Challenge Competence With APPS	Hendrycks et al.	arXiv
2021	Program Synthesis with Large Language Models	Austin et al.	arXiv
2021	Proxemics and Social Interactions in an Instrumented Virtual Reality Workshop	Williamson et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Remote and Collaborative Virtual Reality Experiments via Social VR Platforms	Saffo et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Remote and Collaborative Virtual Reality Experiments via Social VR Platforms	Saffo et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Serious Snacking: A Survival Analysis of how Snacking Mechanics Affect Attrition in a Mobile Serious Game	Alexandrovsky et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Squish This: Force Input on Soft Surfacesfor Visual Targeting Tasks	Fruchard et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2021	Systematizing Confidence in Open Research and Evidence (SCORE)	Alipourfard et al.	SocArXiv
2021	The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers	Buschek et al.	Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
2020	Analysis of Open Data and Computational Reproducibility in Registered Reports in Psychology	Obels et al.	Advances in Methods and Practices in Psychological Science
2020	Artifact Review and Badging — Version 1.1	Machinery
2020	Emotional Footprints of Email Interruptions	Blank et al.	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Estimating the deep replicability of scientific findings using human and artificial intelligence	Yang et al.	Proceedings of the National Academy of Sciences
2020	Evaluating Multivariate Network Visualization Techniques Using a Validated Design and Crowdsourcing Approach	Nobre et al.	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Evaluating the Effect of Timeline Shape on Visualization Task Performance	Bartolomeo et al.	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Explain like I am a Scientist: The Linguistic Barriers of Entry to r/science	August et al.	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Keep Calm and Ride Along: Passenger Comfort and Anxiety as Physiological Responses to Autonomous Driving Styles	Dillen et al.	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability	Ebersole et al.	Advances in Methods and Practices in Psychological Science
2020	On Conducting Security Developer Studies with CS Students: Examining a Password-Storage Study with CS Students, Freelancers, and Company Developers	Naiakshina et al.	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Publish or Perish, but Do Not Forget Your Software Artifacts	Heumüller et al.	Empirical Software Engineering
2020	Reproducibility and Replicability in Economics	Vilhuber	Harvard Data Science Review
2020	Statistical Significance Testing at CHI PLAY: Challenges and Opportunities for More Transparency	Vornhagen et al.	Proceedings of the Annual Symposium on Computer-Human Interaction in Play
2020	The NetHack Learning Environment	Küttler et al.	arXiv
2020	Threats of a Replication Crisis in Empirical Computer Science	Cockburn et al.	Communications of the ACM
2020	Transparency in Qualitative Research: Increasing Fairness in the CHI Review Process	Sukumar et al.	Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Transparency of CHI Research Artifacts: Results of a Self-Reported Survey	Wacharamanotham et al.	Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
2020	Variability in the analysis of a single neuroimaging dataset by many teams	Botvinik-Nezer et al.	Nature
2020	What is replication?	Nosek, Errington	PLOS Biology
2019	AI-Mediated Communication: How the Perception that Profile Text was Written by AI Affects Trustworthiness	Jakesch et al.	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	Gamification in Science: A Study of Requirements in the Context of Reproducible Research	Feger et al.	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	Increasing the Transparency of Research Papers with Explorable Multiverse Analyses	Dragicevic et al.	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	Local Standards for Anonymization Practices in Health, Wellness, Accessibility, and Aging Research at CHI	Abbott et al.	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	Pay Attention, Please: Formal Language Improves Attention in Volunteer and Paid Online Experiments	August, Reinecke	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	Predicting the replicability of social science lab experiments	Altmejd et al.	PLOS ONE
2019	Put Your Warning Where Your Link Is: Improving and Evaluating Email Phishing Warnings	Petelka et al.	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	Reproducibility and Replicability in Science	Medicine	The National Academies Press
2019	TextWorld: A Learning Environment for Text-based Games	Côté et al.	arXiv
2019	The Impact of Web Browser Reader Views on Reading Speed and User Experience	Li et al.	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	The Role of HCI in Reproducible Science: Understanding, Supporting and Motivating Core Practices	Feger et al.	Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems
2019	Vulnerability & Blame: Making Sense of Unauthorized Access to Smartphones	Marques et al.	Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
2019	What's next for Registered Reports?	Chambers	Nature
2018	A Cognitive-Ecological Explanation of Intergroup Biases	Alves et al.	Psychological Science
2018	An empirical analysis of journal policy effectiveness for computational reproducibility	Stodden et al.	Proceedings of the National Academy of Sciences
2018	Badges for sharing data and code at Biostatistics: an observational study	Rowhani-Farid, Barnett	F1000Research
2018	Data Availability, Reusability, and Analytic Reproducibility: Evaluating the Impact of a Mandatory Open Data Policy at the Journal Cognition	Hardwicke et al.	Royal Society Open Science
2018	Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015	Camerer et al.	Nature Human Behaviour
2018	Evaluation Strategies for HCI Toolkit Research	Ledo et al.	Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
2018	HARK No More: On the Preregistration of CHI Experiments	Cockburn et al.	Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
2018	How Relevant are Incidental Power Poses for HCI?	Jansen, Hornbæk	Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
2018	Imagining Replications: Graphical Prediction and Discrete Visualizations Improve Recall and Estimation of Effect Uncertainty	Hullman et al.	IEEE Transactions on Visualization and Computer Graphics
2018	Improving Comprehension of Measurements Using Concrete Re-expression Strategies	Hullman et al.	Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
2018	Making replication mainstream	Zwaan et al.	Behavioral and Brain Sciences
2018	Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results	Silberzahn et al.	Advances in Methods and Practices in Psychological Science
2018	Many Labs 2: Investigating Variation in Replicability Across Samples and Settings	Klein et al.	Advances in Methods and Practices in Psychological Science
2018	Measuring the “Why” of Interaction: Development and Validation of the User Motivation Inventory (UMI)	Brühlmann et al.	Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
2018	Open Source, Open Science, and the Replication Crisis in HCI	Echtler, Häußler	Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
2018	Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network	Moshontz	Advances in Methods and Practices in Psychological Science
2018	Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration	Liu et al.	arXiv
2018	Reproducibility vs. Replicability: A Brief History of a Confused Terminology	Plesser	Frontiers in Neuroinformatics
2018	Special Interest Group on Transparent Statistics Guidelines	Wacharamanotham et al.	Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
2018	Terminologies for Reproducible Research	Barba	arXiv
2018	The preregistration revolution	Nosek et al.	Proceedings of the National Academy of Sciences
2017	Code and Contribution in Interactive Systems Research	Fogarty	CHI 2017 Workshop HCITools
2017	Moving Transparent Statistics Forward at CHI	Kay et al.	Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems
2017	The Coefficient of Determination R^2 and Intra-Class Correlation Coefficient from Generalized Linear Mixed-Effects Models Revisited and Expanded	Nakagawa et al.	Journal of the Royal Society Interface
2016	1,500 scientists lift the lid on reproducibility	Baker	Nature
2016	Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency	Kidwell et al.	PLOS Biology
2016	Evaluating replicability of laboratory experiments in economics	Camerer et al.	Science
2016	Fair Statistical Communication in HCI	Dragicevic	Modern Statistical Methods for HCI
2016	Many Labs 3: Evaluating participant pool quality across the academic semester via replication	Ebersole et al.	Journal of Experimental Social Psychology
2016	OpenAI Gym	Brockman et al.	arXiv
2016	Researcher-Centered Design of Statistics: Why Bayesian Statistics Better Fit the Culture and Incentives of HCI	Kay et al.	Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
2016	Special Interest Group on Transparent Statistics in HCI	Kay et al.	Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems
2016	The Geography and Importance of Localness in Geotagged Social Media	Johnson et al.	Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
2016	What does research reproducibility mean?	Goodman et al.	Science Translational Medicine
2015	Displacing Misinformation about Events: An Experimental Test of Causal Corrections	Nyhan, Reifler	Journal of Experimental Political Science
2015	Estimating the reproducibility of psychological science	Collaboration	Science
2015	Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say “Usually Not”	Chang, Li	Board of Governors of the Federal Reserve System
2015	Promoting an open research culture	Nosek et al.	Science
2015	Unequal Representation and Gender Stereotypes in Image Search Results for Occupations	Kay et al.	Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems
2015	Using prediction markets to estimate the reproducibility of scientific research	Dreber et al.	Proceedings of the National Academy of Sciences
2014	An open investigation of the reproducibility of cancer biology research	Errington et al.	eLife
2014	Investigating Variation in Replicability: A “Many Labs” Replication Project	Klein et al.	Social Psychology
2014	Is Once Enough? On the Extent and Content of Replications in Human-Computer Interaction	Hornbæk et al.	Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
2014	RepliCHI: The Workshop II	Wilson et al.	CHI '14 Extended Abstracts on Human Factors in Computing Systems
2013	Avoiding “It's JUST a Replication”	John	Proceedings of the CHI2013 Workshop on the Replication of HCI Research (RepliCHI 2013)
2013	Framing Influences Willingness to Pay but Not Willingness to Accept	Yang et al.	Journal of Marketing Research
2013	Is Replication Important for HCI?	Greiffenhagen, Reeves	Proceedings of the CHI2013 Workshop on the Replication of HCI Research (RepliCHI 2013)
2013	Proceedings of the CHI2013 Workshop on the Replication of HCI Research	Wilson et al.
2013	RepliCHI: The Workshop	Wilson et al.	CHI '13 Extended Abstracts on Human Factors in Computing Systems
2012	RepliCHI SIG: From a Panel to a New Submission Venue for Replication	Wilson et al.	CHI '12 Extended Abstracts on Human Factors in Computing Systems
2012	Rethinking Statistical Analysis Methods for CHI	Kaptein, Robertson	Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
2011	False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant	Simmons et al.	Psychological Science
2011	RepliCHI — CHI Should Be Replicating and Validating Results More: Discuss	Wilson et al.	CHI '11 Extended Abstracts on Human Factors in Computing Systems
2007	Evaluating user interface systems research	Olsen	Proceedings of the 20th annual ACM symposium on User interface software and technology
2007	HCI... Not As It Should Be: Inferential Statistics in HCI Research	Cairns	Proceedings of HCI 2007 (BCS-HCI)
2005	Why Most Published Research Findings Are False	Ioannidis	PLoS Medicine
2001	Generalizability Theory	Brennan	Springer
1996	Forming Inferences About Some Intraclass Correlation Coefficients	McGraw, Wong	Psychological Methods
1992	Electronic documents give reproducible research a new meaning	Claerbout, Karrenbach	SEG Technical Program Expanded Abstracts 1992
	Aaru
	AEA Data Editor
	Ai2 Asta
	Argos (Scitility)
	Autoscience
	cascad
	Claude for Science
	CloudResearch
	Code Ocean Aqua
	CODECHECK
	DARPA SCORE
	Evidenza
	FutureHouse / Edison Scientific Kosmos
	Gemini for Science
	Gorilla
	GRIM / SPRITE
	I4R AI Replication Engine
	ImageTwin
	Intology
	JATOS
	JATSdecoder
	jsPsych
	lab.js
	Labvanced
	Lila Sciences
	metacheck
	Odum Institute verification for AJPS
	Papermill Alarm (Clear Skies)
	Paperpal Preflight (Cactus)
	Pavlovia
	Periodic Labs
	Prolific
	Proofig
	Qualtrics synthetic-responses case study (Jan 2026)
	RepliBench (UK AISI)
	ReScience C
	Sakana AI
	Signals
	Simile
	statcheck
	Testable
	UL Solutions AI Algorithm Reproducibility Program
	Viewpoints AI

/ filter · j k or ↑ ↓ select · Enter open page · c copy BibTeX