The potential of preregistration in psychology: Assessing preregistration producibility and preregistration-study consistency

Olmo van den Akker*, Marjan Bakker, Marcel A. L. M. van Assen, Charlotte Rebecca Pennington, Leone Verweij, Mahmoud M. Elsherif, Aline Claesen, Stefan D. M. Gaillard, Siu Kit Yeung, Jan-Luca Frankenberger, Kai Krautter, Jamie P. Cockcroft, Katharina S. Kreuer, Thomas Rhys Evans, Frédérique M. Heppel, Sarah F. Schoch, Max Korbmacher, Yuki Yamada, Nihan Albayrak-Aydemir, Shilaan AlzahawiAlexandra Sarafoglou, Maksim M. Sitnikov, Filip Děchtěrenko, Sophia Wingen, Sandra Grinschgl, Helena Hartmann, Suzanne L. K. Stewart, Cátia M. F. de Oliveira, Sarah Ashcroft-Jones, Bradley J. Baker, Jelte M. Wicherts

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Study preregistration has become increasingly popular in psychology, but its potential to restrict researcher degrees of freedom has not yet been empirically verified. We used an extensive protocol to assess the producibility (i.e., the degree to which a study can be properly conducted based on the available information) of preregistrations and the consistency between preregistrations and their corresponding papers for 300 psychology studies. We found that preregistrations often lack methodological details and that undisclosed deviations from preregistered plans are frequent. These results highlight that biases due to researcher degrees of freedom remain possible in many preregistered studies. More comprehensive registration templates typically yielded more producible preregistrations. We did not find that the producibility and consistency of preregistrations differed over time or between original and replication studies. Furthermore, we found that operationalizations of variables were generally preregistered more producible and consistently than other study parts. Inconsistencies between preregistrations and published studies were mainly encountered for data collection procedures, statistical models, and exclusion criteria. Our results indicate that, to unlock the full potential of preregistration, researchers in psychology should aim to write more producible preregistrations, adhere to these preregistrations more faithfully, and more transparently report any deviations from their preregistrations. This could be facilitated by training and education to improve preregistration skills, as well as the development of more comprehensive templates.
Original languageEnglish
JournalPsychological Methods
Early online date10 Oct 2024
DOIs
Publication statusE-pub ahead of print - 10 Oct 2024

Bibliographical note

Copyright © 2024 The Author(s). This work is licensed under a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 International License (CC BY-NC-ND 4.0; https://creativecommons.org/licenses/by-nc-nd/4.0).

Keywords

  • preregistration
  • open science
  • meta-research
  • preregistration deviationreregistration template
  • preregistration template
  • preregistration producibility
  • preregistration-study consistency

Fingerprint

Dive into the research topics of 'The potential of preregistration in psychology: Assessing preregistration producibility and preregistration-study consistency'. Together they form a unique fingerprint.

Cite this