GPT-4.1 and Llama 3.3 70 fail to detect clinically relevant errors in radiology reports in zero-shot evaluation.
Akinci D'Antonoli T; Adams LC; Lübberstedt J; Graf MM; Mertens CJ; Busch F; Ziegelmayer S; Makowski MR · 2026 · European radiology
WASTE classifies this as Negative / Null Result Report · AI classification, approximate
The study found no significant effect — useful as a negative control or null benchmark for your own design.
Abstract (excerpt)
Objectives To evaluate whether GPT-4.1 and Llama 3.3 70B, large language models (LLMs) assessed in zero-shot, baseline configurations, detect and categorize clinically consequential errors across types that range from pattern-based to…
Excerpt shown for reference under fair use — read the full paper at the publisher.
Hosted by the publisher — may require access.
About to run something similar?
Run an AI Precheck on your own design to catch failure modes like this one before you spend the time. Your first desk check is free.
Related failures
A Randomized Trial of Intraarterial Treatment for Acute Ischemic Stroke
Negative / Null Result ReportDuodenal Infusion of Donor Feces for Recurrent Clostridium difficile
Negative / Null Result ReportStenting versus Endarterectomy for Treatment of Carotid-Artery Stenosis
Negative / Null Result ReportEffects of Combination Lipid Therapy in Type 2 Diabetes Mellitus
Replication FailureA Randomized Trial of Bevacizumab for Newly Diagnosed Glioblastoma
Negative / Null Result ReportSpironolactone for Heart Failure with Preserved Ejection Fraction
WASTE indexes this work — it does not host or republish it. Failure-type classification is automated and approximate.
Metadata source: Europe PMC · DOI 10.1007/s00330-026-12697-z