Five-year study finds LLMs have not yet reshaped scientific publishing
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
A large-scale empirical study of 2.5 million papers across five disciplines from 2015 to 2025 finds that LLMs have not yet meaningfully altered scientific practice or output. Most pre-2023 trends in lexical, structural, and epistemological dimensions persisted through 2025, with few detectable inflection points attributable to model adoption. Notable exceptions include accelerated growth in computer science coauthorship teams in Asia and apparent shifts in biomedical topic diversity driven by format standardization rather than LLM influence. The finding challenges widespread assumptions about near-term AI disruption in research and establishes a methodological baseline for tracking whether transformative effects emerge over longer horizons.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
Skeptical readOur AI-generated reading of the wider context and the next developments to watch.
The study measures structural and lexical shifts in published papers but deliberately sidesteps the question of whether LLMs have changed *how scientists work* before publication. OpenAI's researcher survey from yesterday documents exactly that adoption in daily workflows, yet this paper treats it as orthogonal to its findings.
This directly contradicts the framing in OpenAI's survey released November 16th, which established that researchers report meaningful productivity gains and are adopting ChatGPT as research infrastructure. The arXiv study's 2.5 million paper analysis finds no corresponding inflection in output, suggesting either that adoption hasn't yet propagated into published work, that LLM assistance produces papers indistinguishable from pre-LLM research, or that the survey respondents are overstating impact. The methodological tension matters: workflow adoption and publication patterns may decouple for months or years.
If the same research groups surveyed by OpenAI show measurable shifts in paper structure, coauthor geography, or citation velocity by mid-2027, the null finding here becomes a timing artifact rather than evidence against LLM influence. If those groups' outputs remain statistically flat, the adoption survey becomes a measurement of perceived productivity rather than actual research output change.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·OpenAI
OpenAI surveys 1,800 researchers on ChatGPT's role in scientific workflows
OpenAI released findings from a survey of 1,800+ researchers on ChatGPT adoption in scientific workflows. The data maps how LLMs are reshaping daily research practice, quantifying productivity gains and researcher expectations around discovery acceleration. This signals institutional validation of generative AI as research infrastructure rather than novelty, establishing a baseline for how frontier labs should…
MentionsLarge Language Models
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Large Language Models Have Not Yet Transformed Science”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.