A systematic review published yesterday in The Lancet reveals that of 97.1 million verified references in published biomedical papers, 4,046 were likely fake, with rates of fabrication rising 12-fold from 2023 through 2025—which commentators call a disturbing discovery that highlights the need to improve research integrity.
Columbia University researchers used artificial intelligence (AI) to develop an automated reference verification system that scanned PubMed Central’s Open Access subset from January 2023 to February 2026 to audit 2.5 million research papers and 126 million structured references.
For each verified reference, the team retrieved the bibliographical record for the claimed identifier from PubMed and Crossref and compared it with the citing paper’s claimed metadata, using text-similarity scoring.Flagged references underwent sequential filters to reduce false-positives results.
After filtering, references were verified against PubMed, Crossref, OpenAlex, and Google Scholar. A reference not found in any database was characterized as fabricated, and one found but linked to an incorrect identifier was considered a reference error. The precision of the automated reference verification system was 91%.
“Fabricated references (references whose claimed titles correspond to no existing publication) can arise from paper mill activity, intentional misconduct, or uncritical use of artificial intelligence (AI) writing tools,” the study authors wrote.
“Large language models (LLMs) generate plausible sounding but fictitious references, a well documented failure mode; previous studies estimate that 30-69% of LLM-generated references in biomedical contexts are fabricated,” they added.
Review articles had highest fabrication rate
Of 97.1 million verified references, 4,046 in 2,810 papers were fabricated. In 2023, about one in 2,828 papers contained at least one fictional reference, climbing to one in 458 in 2025 and one in 277 in the first seven weeks of this year. The fabrication rate rose over 12-fold, from four per 10,000 papers in 2023, to 51.3 per 10,000 at the end of 2025 and 56.9 per 10,000 in the early part of this year.
Fabricated references (references whose claimed titles correspond to no existing publication) can arise from paper mill activity, intentional misconduct, or uncritical use of artificial intelligence (AI) writing tools.
For example, in a 2025 paper on ureteroileal anastomotic (urinary diversion) techniques in an open-access cancer journal, 60% of 30 verified references were made up. “Each fabricated reference was tailored to the paper’s narrow surgical topic, attributed to real urologists, and bore claimed publication years of 2023 or 2024,” the authors wrote.
The team identified patterns consistent with paper mills, or commercial operations that sell fraudulent or low-quality manuscripts. For instance, the same two authors were the core coauthors in 11 papers in a single surgical journal in 2025, with 15 fabricated references on CRISPR (gene-editing) diagnostics, AI-guided nanovaccines, and gut microbiome biomarkers.
In total, 91% of 2,564 papers affected had one or two fabricated references, while 246 contained at least three. Review articles had a fabrication rate 57% higher than that of other paper types (16.7 vs 10.6 per 10,000 papers).
While the steep rise in contrived references in 2024 coincided with the expected publication lag after widespread use of LLMs, higher paper mill activity and changes in journal indexing might also have been contributing factors, the researchers said.
“LLMs became broadly available in late 2022 and 2023; with submission-to-publication times of 100-200 days,” they wrote. “LLM-assisted papers would appear in PubMed Central from mid-2024 onward.”
Fake references hard to spot
The authors noted that the fake references they identified were not obviously wrong and were topically specific, correctly formatted, attributed to real-life researchers, and bore plausible publication dates.
“Systematic reviews have found that approximately one in four references in medical journal articles contains errors, confirming that reference verification is not standard in peer review,” they wrote.
When references point to non-existent studies, the evidence they claim to support is fictional.
The team recommended that publishers incorporate automated reference verification in submission workflows before peer review and that indexing services add integrity metadata to paper records so that downstream users can assess reference reliability.
Publishers should also retroactively screen publications and issue corrections or retractions when fake references compromise a paper’s conclusions. Last, establishing fabricated references as a discrete category in major research integrity databases would enable systematic tracking and accountability, they said.
“When references point to non-existent studies, the evidence they claim to support is fictional,” they concluded. “Routine automated verification can close this gap before fabricated references reach the published record.”
In a Columbia University press release, lead author Maxim Topaz, PhD, said that clinicians and clinical guideline developers have no way of knowing that the evidence they rely on doesn’t exist. “This discovery directly impacts patients as medical professionals make treatment decisions based on clinical guidelines,” he said.
In a related commentary, Howard Bauchner, MD, of Boston University, and Frederick Rivara, MD, of the University of Washington, said the trend is disturbing and called for authors to take responsibility for an entire manuscript, including the references. “Given that public trust in science appears to be waning in countries around the world, renewed efforts are needed to enhance research integrity,” they wrote.
