Replication FailureOpen accessMedicine· cited by 9

Generative AI-enabled clinical decision support system in primary care: a pragmatic, cluster-randomized trial

Ambrose Agweyu; Paul Mwaniki; Vaishnavi Menon; Robert Korom; Lynda Isaaka; Conrad W. Wanyama; Jaspret Gill; Sarah Kiptinness · 2026 · Nature Medicine

WASTE classifies this as Replication Failure · AI classification, approximate

A previously reported effect did not replicate here — verify it holds before you build on it.

Abstract

Rigorous evidence on the performance of large language models (LLMs) in real-world, low-resource clinical settings remains limited. Here we conducted a pragmatic, cluster-randomized trial in 16 primary care facilities in Kenya. Clinical officers were randomized to use the electronic medical record with or without LLM assistance. The primary outcome was an expert-adjudicated composite of treatment failure events experienced within 14 days of enrollment. Between 22 April and 16 July 2025, 9,691 patients were enrolled, overseen by 103 clinical officers (52 in the LLM-assisted arm and 51 in the co

Abstract by Ambrose Agweyu; Paul Mwaniki; Vaishnavi Menon; Robert Korom; Lynda Isaaka; Conrad W. Wanyama; Jaspret Gill; Sarah Kiptinness, Nature Medicine (2026) — licensed CC BY 4.0.

Read the full paper (open access)