Questionable research practices may have little effect on replicability
Rolf Ulrich; Jeff Miller · 2020 · eLife
WASTE classifies this as Replication Failure · AI classification, approximate
A previously reported effect did not replicate here — verify it holds before you build on it.
Abstract
This article examines why many studies fail to replicate statistically significant published results. We address this issue within a general statistical framework that also allows us to include various questionable research practices (QRPs) that are thought to reduce replicability. The analyses indicate that the base rate of true effects is the major factor that determines the replication rate of scientific results. Specifically, for purely statistical reasons, replicability is low in research domains where true effects are rare (e.g., search for effective drugs in pharmacology). This point is
Abstract by Rolf Ulrich; Jeff Miller, eLife (2020) — licensed CC BY 4.0.
About to run something similar?
Run an AI Precheck on your own design to catch failure modes like this one before you spend the time. Your first desk check is free.
Related failures
Estimating the reproducibility of psychological science
Negative / Null Result ReportWillingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results
Methods Dead-EndComparing multiple comparisons: practical guidance for choosing the best multiple comparisons test
Replication FailureInferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don’t Expect Replication
Negative / Null Result ReportMOBILE BANKING ADOPTION: APPLICATION OF DIFFUSION OF INNOVATION THEORY
Negative / Null Result ReportEnjoyment and social influence: predicting mobile payment adoption
WASTE indexes this work — it does not host or republish it. Failure-type classification is automated and approximate.
Metadata source: OpenAlex · DOI 10.7554/elife.58237