Skip to content
Modelwire
Subscribe

Five-year study finds LLMs have not yet reshaped scientific publishing

Source published ·Modelwire updated

Original coverage: arXiv cs.CL ↗·How Modelwire adds context

The development

A large-scale empirical study of 2.5 million papers across five disciplines from 2015 to 2025 finds that LLMs have not yet meaningfully altered scientific practice or output. Most pre-2023 trends in lexical, structural, and epistemological dimensions persisted through 2025, with few detectable inflection points attributable to model adoption. Notable exceptions include accelerated growth in computer science coauthorship teams in Asia and apparent shifts in biomedical topic diversity driven by format standardization rather than LLM influence. The finding challenges widespread assumptions about near-term AI disruption in research and establishes a methodological baseline for tracking whether transformative effects emerge over longer horizons.

Modelwire’s AI-generated summary of coverage from arXiv cs.CL.

Modelwire analysis

Skeptical read

Our AI-generated reading of the wider context and the next developments to watch.

The study measures structural and lexical shifts in published papers but deliberately sidesteps the question of whether LLMs have changed *how scientists work* before publication. OpenAI's researcher survey from yesterday documents exactly that adoption in daily workflows, yet this paper treats it as orthogonal to its findings.

This directly contradicts the framing in OpenAI's survey released November 16th, which established that researchers report meaningful productivity gains and are adopting ChatGPT as research infrastructure. The arXiv study's 2.5 million paper analysis finds no corresponding inflection in output, suggesting either that adoption hasn't yet propagated into published work, that LLM assistance produces papers indistinguishable from pre-LLM research, or that the survey respondents are overstating impact. The methodological tension matters: workflow adoption and publication patterns may decouple for months or years.

If the same research groups surveyed by OpenAI show measurable shifts in paper structure, coauthor geography, or citation velocity by mid-2027, the null finding here becomes a timing artifact rather than evidence against LLM influence. If those groups' outputs remain statistically flat, the adoption survey becomes a measurement of perceived productivity rather than actual research output change.

This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error

Coverage behind this analysis

These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.

  1. ·OpenAI

    OpenAI surveys 1,800 researchers on ChatGPT's role in scientific workflows

    OpenAI released findings from a survey of 1,800+ researchers on ChatGPT adoption in scientific workflows. The data maps how LLMs are reshaping daily research practice, quantifying productivity gains and researcher expectations around discovery acceleration. This signals institutional validation of generative AI as research infrastructure rather than novelty, establishing a baseline for how frontier labs should…

    Read Modelwire coverage →Original source ↗

MentionsLarge Language Models

MW

How this coverage is produced

Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.

Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Large Language Models Have Not Yet Transformed Science”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.

More to explore

Survey maps LLM role across full AI infrastructure lifecycle

arXiv cs.CL·

Stack Exchange reveals where AI-generated answers slip past moderation

arXiv cs.CL·

Engineering journals show 82% surge in AI-like language after ChatGPT launch