Not all effects are indispensable: Psychological science requires verifiable lines of reasoning for whether an effect matters

Farid Anvari*, Roger A. Kievit, Daniël Lakens, Charlotte Rebecca Pennington, Andrew A. Przybylski, Leo Tiokhin, Brenton M. Wiernik, Amy Orben

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Psychological researchers currently lack guidance for how to make claims about and evaluate the practical relevance and significance of observed effect sizes, i.e. whether a finding will have impact when translated to a different context of application. Although psychologists have recently highlighted theoretical justifications for why small effect sizes might be practically relevant, such justifications fail to provide the information necessary for evaluation and falsification. Claims about whether an observed effect size is practically relevant need to consider both the mechanisms amplifying and counteracting practical relevance, as well as the assumptions underlying each mechanism at play. To provide guidance for making claims about whether an observed effect size is practically relevant in such a way that the claims can be systematically evaluated, we present examples of widely applicable mechanisms and the key assumptions needed for justifying whether an observed effect size can be expected to generalize to different contexts. Routine use of these mechanisms to justify claims about practical relevance has the potential to make researchers’ claims about generalizability substantially more transparent. This transparency can help move psychological science towards a more rigorous assessment of when psychological findings can be applied in the world.
Original languageEnglish
JournalPerspectives on Psychological Science
Publication statusAccepted/In press - 1 Mar 2022

Bibliographical note

(c) Sage, 2022. This manuscript has been accepted for publication in Perspectives on Psychological Science. The final publication is available via Sage at [DOI link here when available]

Keywords

  • effect size
  • practical significance
  • benchmarks
  • evaluation

Fingerprint

Dive into the research topics of 'Not all effects are indispensable: Psychological science requires verifiable lines of reasoning for whether an effect matters'. Together they form a unique fingerprint.

Cite this