Moving Towards the Post p < 0.05 Era via the Analysis of Credibility

Robert A.J. Matthews*

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

It is now widely accepted that the techniques of null hypothesis significance testing (NHST) are routinely misused and misinterpreted by researchers seeking insight from data. There is, however, no consensus on acceptable alternatives, leaving researchers with little choice but to continue using NHST, regardless of its failings. I examine the potential for the Analysis of Credibility (AnCred) to resolve this impasse. Using real-life examples, I assess the ability of AnCred to provide researchers with a simple but robust framework for assessing study findings that goes beyond the standard dichotomy of statistical significance/nonsignificance. By extracting more insight from standard summary statistics while offering more protection against inferential fallacies, AnCred may encourage researchers to move toward the post p < 0.05 era.

Original languageEnglish
Pages (from-to)202-212
Number of pages11
JournalAmerican Statistician
Volume73
Issue numbersup1
Early online date20 Mar 2019
DOIs
Publication statusE-pub ahead of print - 20 Mar 2019

Fingerprint

Credibility
Null hypothesis
Testing
Statistical Significance
Dichotomy
Resolve
Continue
Statistics
Alternatives
Standards

Bibliographical note

© 2019 The Author. Published with license by Taylor & Francis Group, LLC.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which
permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.

Keywords

  • Analysis of credibility
  • Bayesian inference
  • Null hypothesis Significance testing
  • p-Values

Cite this

Matthews, Robert A.J. / Moving Towards the Post p < 0.05 Era via the Analysis of Credibility. In: American Statistician . 2019 ; Vol. 73, No. sup1. pp. 202-212.
@article{a42e1ac959f54ebf98fa9031b0a95912,
title = "Moving Towards the Post p < 0.05 Era via the Analysis of Credibility",
abstract = "It is now widely accepted that the techniques of null hypothesis significance testing (NHST) are routinely misused and misinterpreted by researchers seeking insight from data. There is, however, no consensus on acceptable alternatives, leaving researchers with little choice but to continue using NHST, regardless of its failings. I examine the potential for the Analysis of Credibility (AnCred) to resolve this impasse. Using real-life examples, I assess the ability of AnCred to provide researchers with a simple but robust framework for assessing study findings that goes beyond the standard dichotomy of statistical significance/nonsignificance. By extracting more insight from standard summary statistics while offering more protection against inferential fallacies, AnCred may encourage researchers to move toward the post p < 0.05 era.",
keywords = "Analysis of credibility, Bayesian inference, Null hypothesis Significance testing, p-Values",
author = "Matthews, {Robert A.J.}",
note = "{\circledC} 2019 The Author. Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.",
year = "2019",
month = "3",
day = "20",
doi = "10.1080/00031305.2018.1543136",
language = "English",
volume = "73",
pages = "202--212",
journal = "American Statistician",
issn = "0003-1305",
publisher = "American Statistical Association",
number = "sup1",

}

Moving Towards the Post p < 0.05 Era via the Analysis of Credibility. / Matthews, Robert A.J.

In: American Statistician , Vol. 73, No. sup1, 20.03.2019, p. 202-212.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Moving Towards the Post p < 0.05 Era via the Analysis of Credibility

AU - Matthews, Robert A.J.

N1 - © 2019 The Author. Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.

PY - 2019/3/20

Y1 - 2019/3/20

N2 - It is now widely accepted that the techniques of null hypothesis significance testing (NHST) are routinely misused and misinterpreted by researchers seeking insight from data. There is, however, no consensus on acceptable alternatives, leaving researchers with little choice but to continue using NHST, regardless of its failings. I examine the potential for the Analysis of Credibility (AnCred) to resolve this impasse. Using real-life examples, I assess the ability of AnCred to provide researchers with a simple but robust framework for assessing study findings that goes beyond the standard dichotomy of statistical significance/nonsignificance. By extracting more insight from standard summary statistics while offering more protection against inferential fallacies, AnCred may encourage researchers to move toward the post p < 0.05 era.

AB - It is now widely accepted that the techniques of null hypothesis significance testing (NHST) are routinely misused and misinterpreted by researchers seeking insight from data. There is, however, no consensus on acceptable alternatives, leaving researchers with little choice but to continue using NHST, regardless of its failings. I examine the potential for the Analysis of Credibility (AnCred) to resolve this impasse. Using real-life examples, I assess the ability of AnCred to provide researchers with a simple but robust framework for assessing study findings that goes beyond the standard dichotomy of statistical significance/nonsignificance. By extracting more insight from standard summary statistics while offering more protection against inferential fallacies, AnCred may encourage researchers to move toward the post p < 0.05 era.

KW - Analysis of credibility

KW - Bayesian inference

KW - Null hypothesis Significance testing

KW - p-Values

UR - http://www.scopus.com/inward/record.url?scp=85063192626&partnerID=8YFLogxK

UR - https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1543136

U2 - 10.1080/00031305.2018.1543136

DO - 10.1080/00031305.2018.1543136

M3 - Article

AN - SCOPUS:85063192626

VL - 73

SP - 202

EP - 212

JO - American Statistician

JF - American Statistician

SN - 0003-1305

IS - sup1

ER -