Take caution in using LLMs as human surrogates
Yuan Gao; Dokyun Lee; Gordon Burtch; Sina Fazelpour · 2025 · Proceedings of the National Academy of Sciences
WASTE classifies this as Negative / Null Result Report · AI classification, approximate
The study found no significant effect — useful as a negative control or null benchmark for your own design.
Abstract (excerpt)
Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, surveys, and political discourse. This has led many to propose that LLMs can be used as surrogates…
Excerpt shown for reference under fair use — read the full paper at the publisher.
About to run something similar?
Run an AI Precheck on your own design to catch failure modes like this one before you spend the time. Your first desk check is free.
Related failures
Power and Predictive Accuracy of Polygenic Risk Scores
Negative / Null Result ReportLocoregional Recurrence After Sentinel Lymph Node Dissection With or Without Axillary Dissection in Patients With Sentinel Lymph Node Metastases
Negative / Null Result ReportCritical aspects of using bacterial cell viability assays with the fluorophores SYTO9 and propidium iodide
Negative / Null Result ReportAlpelisib plus fulvestrant for PIK3CA-mutated, hormone receptor-positive, human epidermal growth factor receptor-2–negative advanced breast cancer: final overall survival results from SOLAR-1
Negative / Null Result ReportA randomized placebo-controlled trial of idebenone in Leber’s hereditary optic neuropathy
Negative / Null Result ReportThe CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens
WASTE indexes this work — it does not host or republish it. Failure-type classification is automated and approximate.
Metadata source: OpenAlex · DOI 10.1073/pnas.2501660122