Revealing the Paper Mill Iceberg: AI-Based Screening of Cancer Research Publications.
Baptiste ScancariD, Jennifer Anne ByrneiD, David CauseuriD, Adrian Gerard BarnettiD
Abstract
ObjectivesTo train and validate a machine learning model to distinguish paper mill publications from genuine cancer research articles, and to screen the cancer research literature to assess the prevalence of papers that have textual similarities to paper mill papers. DesignMethodological study applying a BERT-based text classification model to article titles and abstracts. SettingRetracted paper mill publications listed in the Retraction Watch database were used for model training. The cancer research corpus was screened by the model, using the PubMed database restricted to original cancer research articles published between 1999 and 2024. ParticipantsThe model was trained on 2,202 retracted paper mill papers and validated on independent data collected by image integrity experts. A total of 2.6 million cancer research papers were screened. Main outcome measuresClassification performance of the model. Prevalence of papers flagged as similar to retracted paper mill publications with 95% confidence intervals and their distribution over time, by country, publisher, cancer type, research area, and within high-impact journals (Decile 1). ResultsThe model achieved an accuracy of 0.91. When applied to the cancer research literature, it flagged 9.87% (95% CI 9.83 to 9.90) of papers and revealed a large increase in flagged papers from 1999 to 2024, both across the entire corpus and in the top 10% of journals by impact factor. Over 170,000 papers affiliated with Chinese institutions were flagged, accounting for 35% of Chinese cancer research articles. Most publishers had published substantial numbers of flagged papers. Flagged papers were overrepresented in fundamental research and in gastric, bone, and liver cancer. ConclusionsPaper mills are a large and growing problem in the cancer literature and are not restricted to low impact journals. Collective awareness and action will be crucial to address the problem of paper mill publications.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 26
- FWCI
- 129.64
- Field percentile
- top 1%
- Research Focus
- Approach
In-depth summary
Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 2 (10 Nov 2025) · CC BY 4.0
A BERT classifier on titles and abstracts flagged 9.87% of 2.6 million cancer papers (1999 to 2024) as resembling retracted paper mill papers, with yearly shares rising past 15%.
Paper mills fabricate manuscripts at scale, and cancer research is thought to be more exposed than the estimated 3% prevalence across biology and medicine. Earlier machine learning work predicted retractions and paper mill products from text across disciplines; here a cancer-specific title-and-abstract classifier is trained and then used to screen 2.6 million original cancer research articles in PubMed from 1999 to 2024.
Key results
ShownA fine-tuned BERT model that scores each sentence separates retracted paper mill papers from proxy-genuine controls with high specificity. Performance holds on an independent dataset from integrity experts.
Internal test set (n = 551): accuracy 0.91, sensitivity 0.87 (239/276), specificity 0.96 (263/275). External integrity-expert set (3,094 paper mill vs 3,100 controls): accuracy 0.93, sensitivity 0.87, specificity 0.99. Also flagged 67% (130/193), 75% (425/567) and 66% (75/113) of problematic papers from Oste et al., Park et al. and Pathmendra et al. (about 72% combined). · Model selection and training; Additional validation; Performance
ShownAlmost one in ten original cancer research articles in PubMed from 1999 to 2024 was flagged. The yearly number of flagged papers grew exponentially up to a peak in 2022.
261,245 of about 2.6 million papers flagged, 9.87% (95% CI 9.83 to 9.90; bootstrap, 1,000 resamples). Yearly flagged counts fit an exponential trend from 1999 to 2022 (R² = 0.92), peaked in 2022 and declined slightly in 2023 and 2024. The flagged share rose from around 1% in the early 2000s to above 15% in the early 2020s. · Screening the cancer literature; Flagged papers in the cancer literature; Trends in flagged papers
ShownFlags cluster by country and publisher. China has the highest flagged share and the most flagged papers. The largest publishers have lower flag rates but still carry the most flagged papers in absolute terms.
China 35% of its cancer papers (177,907 flagged), Iran 20% (6,801), Saudi Arabia 16%, Egypt 15%, Pakistan 14%, Malaysia 13%; US 2% (10,511). Verduci Editore about 65%, International Scientific Literature about 45%, E-Century 42%, Spandidos 37%; Elsevier, Springer Nature and John Wiley & Sons around 10% but 39,753, 40,293 and 28,330 flagged papers. · Countries of flagged papers; Publishers and journals of flagged papers
ShownFlag rates differ by cancer type and research area. They are highest in gastric, bone and liver cancer and in fundamental research, and lowest in population-level and care-oriented research.
Gastric cancer 22%, bone cancers (such as osteosarcoma) 21%, liver 19%; most types 10 to 15%; breast, skin, prostate and blood cancers lowest. Largest counts: lung (28,435) and liver (26,730). Cancer biology and fundamental research, treatment, and diagnosis and prognosis exceed 10%; survivorship, epidemiology and health-policy areas are under 2%. · Cancer types of flagged papers; Cancer research area of flagged papers
ShownFlagged papers are not confined to low-impact venues. Their share in the top-decile impact factor journals rose over time, while the impact factor needed to enter that decile also rose.
In Decile 1 cancer journals the flagged share was low in the early 2000s, 8% in 2016 and around 10% in 2022. The Decile 1 impact factor cut-off rose from 3 in 1999 to 7 in 2021. · Flagged cancer research papers in high impact…; Figure legends
Methods, data and reproducibility
Fine-tuned BERT-base-uncased as a binary classifier on sentence-split titles and abstracts. A paper's score is the mean of its sentence probabilities, with the threshold tuned for specificity. Training used Retraction Watch paper mill retractions versus proxy-genuine controls, and external validation used integrity-expert lists. The model was then applied to a keyword-filtered PubMed cancer corpus.
- PubMed 2025 baseline, filtered to English 'Journal Article' records from 1999 to 2024 that match cancer keywords · n = 2,647,471 papers in 11,632 journals
- Retraction Watch Database retractions tagged 'Paper Mill' that match the cancer corpus (training, optimisation, internal validation) · n = 2,202
- Proxy-genuine controls from four high-impact journals (Cell, Cancer Cell, Molecular Cell, The EMBO Journal) and Taiwanese, Swedish, Finnish and Norwegian institutions, checked on PubPeer · n = 2,202 (with the 2,202 paper mill papers, 4,404 split 3,082 / 771 / 551)
- Integrity experts' lists of suspected paper mill papers with image manipulation ('Spreadsheet of spreadsheet'), plus a similar number of randomly sampled controls, for external validation · n = 3,094 paper mill papers and 3,100 controls
- Problematic cancer papers with misidentified or non-verifiable nucleotide sequences or cell lines, from Oste et al., Pathmendra et al. and Park et al. · n = 873 (193, 113 and 567)
- Methods & tools
- BERT-base-uncased, RoBERTa, BioBERT, PubMedBERT, Longformer, Clinical Longformer, ggplot2, PubMed, Retraction Watch Database, PubPeer, OpenAlex, SCImago Journal Impact Factor
- Reproducibility gaps
- Code and fine-tuned models are deliberately withheld to stop evasion; the flagged PMID list is available only on reasonable request. Retraction Watch and the integrity experts' dataset are named as public but no URLs are given; optimisation and labelling details sit in supplementary files.
Limitations (6)
- Labels are imperfect: the Retraction Watch 'paper mill' tag reflects staff readings of retraction notices backed by varied evidence, expert lists rest on image manipulation that also occurs outside paper mills, and controls are proxies (selected journals and countries) with limited human verification. · stated by the authors
- Chinese institutions dominate the training paper mill set, so the model may partly learn Chinese scientific writing. The authors counter that false positives were rare (39 of 3,375) and that 90% of false negatives had China-affiliated first authors (vs 47% overall). · stated by the authors
- Positive predictive value depends on the unknown true prevalence. At an assumed 10% prevalence it would be about 0.7, so roughly 30% of flagged papers could be false positives. Flags are statistical signals that need human review, not findings of misconduct. · stated by the authors
- Training retractions date mostly from 2013 to 2023, so low counts before 2010 may reflect the training data rather than absent paper mill features. For the flattening after 2022 the authors offer three explanations: publishers fighting back, new AI-era templates, or PubMed listing delays. · stated by the authors
- External validation controls came from the same proxy pools as the training controls (Nordic institutions and four high-impact journals). The 0.99 specificity may therefore not hold for the broader corpus on which the 9.87% estimate rests. · noted in this summary
- Code and fine-tuned weights are withheld, the flagged PMID list is available only on request, and classification metrics have no confidence intervals, so neither the classifier nor the prevalence figures can be independently re-run. · noted in this summary
Who should read:Read this if you handle editorial or integrity screening, or rely on the cancer literature, and want a field-wide estimate of papers that read like paper mill retractions plus how a title-and-abstract BERT screen was validated.
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Suspected distortion of citations in high-impact cancer journals
Baptiste Scancar, Jennifer Anne Byrne, … Adrian Gerard Barnett·bioRxiv·Sep 16, 2026
- Analysis of >30,000 abstracts suggests higher false discovery rates for oncology journals, especially those with low impact factors
LM Hall, AE Hendricks·bioRxiv·Jan 6, 2019
- Making headlines: An analysis of US government-funded cancer research mentioned in online media
L. A. Maggio, C. L. Ratcliff, … J. P. Alperin·bioRxiv·Jul 19, 2018
- Cancer In The News: Bias And Quality In Media Reporting Of Cancer Research.
A. Amberg, D. N. Saunders·bioRxiv·Sep 19, 2018
- An exploratory analysis of 4844 withdrawn articles and their retraction notes.
C. S. Toma, L. Padureanu·bioRxiv·Sep 30, 2021
- 140 Years of mathematical modeling in oncology through AI-assisted curation
Franco Pradelli, Maximilian A.R. Strobl, … Jeffrey West·bioRxiv·Jun 17, 2026