TY - JOUR
T1 - Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation
AU - Pownall, Madeleine
AU - Pennington, Charlotte R.
AU - Norris, Emma
AU - Juanchich, Marie
AU - Smailes, David
AU - Russell, Sophie
AU - Gooch, Debbie
AU - Evans, Thomas rhys
AU - Persson, Sofia
AU - Mak, Matthew h. c.
AU - Tzavella, Loukia
AU - Monk, Rebecca
AU - Gough, Thomas
AU - Benwell, Christopher s. y.
AU - Elsherif, Mahmoud
AU - Farran, Emily
AU - Gallagher-Mitchell, Thomas
AU - Kendrick, Luke t.
AU - Bahnmueller, Julia
AU - Nordmann, Emily
AU - Zaneva, Mirela
AU - Gilligan-Lee, Katie
AU - Bazhydai, Marina
AU - Jones, Andrew
AU - Sedgmond, Jemma
AU - Holzleitner, Iris
AU - Reynolds, James
AU - Moss, Jo
AU - Farrelly, Daniel
AU - Parker, Adam j.
AU - Clark, Kait
N1 - © The Author(s) 2023. This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
PY - 2023/12/21
Y1 - 2023/12/21
N2 - Research shows that questionable research practices (QRPs) are present in undergraduate final-year dissertation projects. One entry-level Open Science practice proposed to mitigate QRPs is “study preregistration,” through which researchers outline their research questions, design, method, and analysis plans before data collection and/or analysis. In this study, we aimed to empirically test the effectiveness of preregistration as a pedagogic tool in undergraduate dissertations using a quasi-experimental design. A total of 89 UK psychology students were recruited, including students who preregistered their empirical quantitative dissertation (n = 52; experimental group) and students who did not (n = 37; control group). Attitudes toward statistics, acceptance of QRPs, and perceived understanding of Open Science were measured both before and after dissertation completion. Exploratory measures included capability, opportunity, and motivation to engage with preregistration, measured at Time 1 only. This study was conducted as a Registered Report; Stage 1 protocol: https://osf.io/9hjbw (date of in-principle acceptance: September 21, 2021). Study preregistration did not significantly affect attitudes toward statistics or acceptance of QRPs. However, students who preregistered reported greater perceived understanding of Open Science concepts from Time 1 to Time 2 compared with students who did not preregister. Exploratory analyses indicated that students who preregistered reported significantly greater capability, opportunity, and motivation to preregister. Qualitative responses revealed that preregistration was perceived to improve clarity and organization of the dissertation, prevent QRPs, and promote rigor. Disadvantages and barriers included time, perceived rigidity, and need for training. These results contribute to discussions surrounding embedding Open Science principles into research training.
AB - Research shows that questionable research practices (QRPs) are present in undergraduate final-year dissertation projects. One entry-level Open Science practice proposed to mitigate QRPs is “study preregistration,” through which researchers outline their research questions, design, method, and analysis plans before data collection and/or analysis. In this study, we aimed to empirically test the effectiveness of preregistration as a pedagogic tool in undergraduate dissertations using a quasi-experimental design. A total of 89 UK psychology students were recruited, including students who preregistered their empirical quantitative dissertation (n = 52; experimental group) and students who did not (n = 37; control group). Attitudes toward statistics, acceptance of QRPs, and perceived understanding of Open Science were measured both before and after dissertation completion. Exploratory measures included capability, opportunity, and motivation to engage with preregistration, measured at Time 1 only. This study was conducted as a Registered Report; Stage 1 protocol: https://osf.io/9hjbw (date of in-principle acceptance: September 21, 2021). Study preregistration did not significantly affect attitudes toward statistics or acceptance of QRPs. However, students who preregistered reported greater perceived understanding of Open Science concepts from Time 1 to Time 2 compared with students who did not preregister. Exploratory analyses indicated that students who preregistered reported significantly greater capability, opportunity, and motivation to preregister. Qualitative responses revealed that preregistration was perceived to improve clarity and organization of the dissertation, prevent QRPs, and promote rigor. Disadvantages and barriers included time, perceived rigidity, and need for training. These results contribute to discussions surrounding embedding Open Science principles into research training.
KW - Open Science
KW - dissertations
KW - preregistration
KW - reproducibility
KW - research training
KW - undergraduate training
UR - http://www.scopus.com/inward/record.url?scp=85180694343&partnerID=8YFLogxK
UR - https://journals.sagepub.com/doi/10.1177/25152459231202724
U2 - 10.1177/25152459231202724
DO - 10.1177/25152459231202724
M3 - Registered Report
SN - 2515-2459
VL - 6
SP - 1
EP - 21
JO - Advances in Methods and Practices in Psychological Science
JF - Advances in Methods and Practices in Psychological Science
IS - 4
M1 - 4
ER -