Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition
Tom E Hardwicke; Maya B. Mathur; Kyle MacDonald; Gustav Nilsonne; George C. Banks; Mallory C. Kidwell; Alicia Hofelich Mohr; Elizabeth Clayton · 2018 · Royal Society Open Science
WASTE classifies this as Replication Failure · AI classification, approximate
A previously reported effect did not replicate here — verify it holds before you build on it.
Abstract
Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (‘analytic reproducibility’). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition . Interrupted time-series analyses indicated a substantial post-policy increase in data availabl
Abstract by Tom E Hardwicke; Maya B. Mathur; Kyle MacDonald; Gustav Nilsonne; George C. Banks; Mallory C. Kidwell; Alicia Hofelich Mohr; Elizabeth Clayton, Royal Society Open Science (2018) — licensed CC BY 4.0.
About to run something similar?
Run an AI Precheck on your own design to catch failure modes like this one before you spend the time. Your first desk check is free.
Related failures
Estimating the reproducibility of psychological science
Negative / Null Result ReportWillingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results
Methods Dead-EndComparing multiple comparisons: practical guidance for choosing the best multiple comparisons test
Replication FailureInferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don’t Expect Replication
Negative / Null Result ReportMOBILE BANKING ADOPTION: APPLICATION OF DIFFUSION OF INNOVATION THEORY
Negative / Null Result ReportEnjoyment and social influence: predicting mobile payment adoption
WASTE indexes this work — it does not host or republish it. Failure-type classification is automated and approximate.
Metadata source: OpenAlex · DOI 10.1098/rsos.180448