top of page

The Water We Swim In: Harm Reduction for AI in Scientific Writing

11 minutes ago
7 min read

The error was in the code. A student had asked ChatGPT to write it, had not tested it, and it ended up in a paper that Bailes’s team submitted for publication [2]. Bailes, whose group works with neutron-star data, is not an AI skeptic. His team is building a virtual universe with an AI model as a kind of co-teacher, and he talks about its educational potential as phenomenal [2]. His advice is not to avoid the tools. It is to embrace them enthusiastically and then scrutinize what comes out the way you would scrutinize any other result [2].


I keep coming back to that story because of what it does not contain. Nobody lied. Nobody committed fraud. A person used a tool, and nobody checked it. That is the shape of most of the AI problem in science, and it is about to matter a lot more, because the tool is no longer an exotic habit. It is the water we swim in every day.


The fingerprint

A new preprint from Holzwarth, González-Márquez, and Kobak puts a number on how much water [1]. Their approach is clever and a little funny. Large language models overuse certain words, and the words are not technical. They are ordinary connective tissue: these, potential, delves. Before ChatGPT arrived in November 2022, the frequency of such words in biomedical papers drifted along a predictable line. The authors fit that line to 2018 through 2022 and extended it forward to estimate how often human writers would have used each word on their own [1].


Then they compared the projection to reality. Take the word “these.” In December 2025 abstracts, the trend line predicted about 33% of papers would use it. The observed figure was 50%. That excess 17% has to come from somewhere [1].


Apply the same logic across a curated set of marker words and a full-text corpus of nearly 1.2 million open-access papers, and the estimate is that 89% of papers published in December 2025 showed signs of LLM-assisted writing or editing. Across all of 2025, it was 77% [1].


What the number does and does not say

I want to be careful here, because nitpicking is the job. “Show signs of” is doing real work in that sentence. The method detects LLM fingerprints anywhere in a paper, so a single polished paragraph counts. When the authors looked at random 255-word crops instead of whole sections, the picture was more modest: the Discussion section hit 68% in December 2025, while Methods came in at 32% [1]. People lean on the tool most where prose carries the argument and least where precision is the point. That seems about right.


Two other caveats. This is a preprint, so it has not been through peer review. And the whole approach rests on an assumption that human word use would have kept drifting in a straight line, which gets shakier the further we get from 2022 and language changes in response to AI. The authors say so themselves [1].


So is it a bombshell? No. Earlier word-frequency studies put 2024 usage at 12 to 16%, and this group argues that those were systematic underestimates, since their own 2024 figures are 31% for abstracts and 52% for full papers [1]. But it is confirmation of something many of us already assumed, now with a defensible estimate attached.


What people tell surveys

Surveys have been pointing the same direction. A Wiley survey of nearly 5,000 researchers, reported in Nature in early 2025, found that 45% of the first 1,043 respondents said they had actually used AI to help with research, most often for translation, proofreading, and editing manuscripts [3]. I will not pretend that figure subtracts cleanly from the preprint’s: different people, different year. But the preprint authors note that people probably use these tools more than they admit when asked, and that surveys tend to ask about narrow use cases [1].


The same survey found something I find more useful than any prevalence number. Nearly two-thirds of respondents said a lack of guidance and training was holding back their use, and about 70% wanted publishers to spell out which uses are acceptable [3]. Researchers are not asking permission to use AI. They are asking for rules.


Who is using it

The geography is where it gets interesting. In the preprint, papers from countries with predominantly native English speakers had an estimated LLM usage of 0.37, versus 0.72 for the rest of the world. South Korea (0.85) and China (0.82) sat near the top; the UK (0.28) sat at the bottom [1]. The authors read this as a rough proxy for English mastery, and they note that non-native-speaking countries’ use of the marker words converged on native-speaking countries’ by 2025 [1].


I am not an ethics expert, and I am not going to pretend to settle the ethics of this. But I will say that the equity argument is real. A tool that lets a talented researcher in Seoul or Beijing write with the fluency that a researcher in Sheffield gets for free is leveling something. A blanket prohibition would not fall evenly. It would fall hardest on the people the language barrier already costs the most.


The thinking objection

The strongest argument against all of this is that writing is thinking. If you outsource the sentences, you may be outsourcing the reasoning, and the preprint’s authors raise exactly this worry [1]. I take it seriously, so I went looking for data.


The picture is mixed, and it splits along a useful line. In an MIT Media Lab preprint, which has been contested in the literature and has not been peer reviewed, participants wrote essays with an LLM, a search engine, or no tools. Brain connectivity on EEG was weakest in the LLM group and strongest in the group working unaided [4]. The authors caution that this was essay writing in an educational setting and may not generalize [4]. A survey of 319 knowledge workers found that more confidence in the AI went with less critical thinking, while more confidence in oneself went with more, and that the work with AI shifted toward verification and oversight [5]. A third study found that cognitive offloading sat between AI use and critical thinking, with younger respondents offloading more and scoring lower [6]. Each of these is short-term, self-reported, or cross-sectional, which limits what it can say about long-term effects.


My read, and it is an inference rather than a finding: this evidence supports concern about how the tool is used, meaning passive offloading and over-trust, more than it supports the claim that using it harms thinking. It also suggests a practical rule. Do the thinking first, and keep your own judgment in charge of the output.


Where I am less torn is the classroom. Someone still learning to write and reason is doing something different from someone who has done it for twenty years. I do not think AI belongs in the first setting. That is a different question from what to do about the second.


So: despair, or harm reduction?

There are two easy responses to a number like 89%. One is despair. The other is to insist on a prohibition that most of the field is already ignoring. A third option comes from public health. Harm reduction starts by accepting that a behavior is happening, then asks how to make it safer.


Applied here, that looks like four things.

  1. Disclose. I use AI to draft and edit, then I check all the output myself, and I say so transparently. The point is not penance. It is that readers and reviewers deserve to know what they are evaluating.

  2. Verify. The preprint cites an audit of 2.5 million biomedical papers for fabricated citations [1]. A hallucinated reference is the written version of an untested script. The remedy for both is the same dull habit: check the thing before it leaves your hands.

  3. Teach. If two-thirds of researchers say they need guidance and training, the response is training, ideally as routine as good clinical practice training, which is a comparison one survey respondent made [3].

  4. Build guardrails into the tool. Peer review shows why. A Frontiers survey of about 1,600 academics found that more than half had used AI while reviewing manuscripts, often against guidance, and among those users 59% used it to help write their reports [2]. Uploading an unpublished manuscript to a public chatbot is a confidentiality problem, not a style choice. Frontiers’ answer was a closed in-house platform that bars pasting chatbot text into a report and lets editors see when it was used [2]. That is harm reduction in its cleanest form: a safer version of the behavior that was already happening.


The question I cannot close

Hao and colleagues, in a preprint reported by Nature, found that scientists who used AI published more, drew more citations, and became team leaders earlier, while the authors warned the effect might narrow the range of what gets studied [2]. Individual productivity up, collective diversity possibly down. I use these tools partly for the first reason, and I would be a hypocrite to pretend otherwise.


So here is where I land, without much resolution. If nearly everyone is using the tool and very few say so, what would it take to make disclosing use ordinary?


References

1. Holzwarth L, González-Márquez R, Kobak D. Most biomedical publications show signs of LLM-assisted writing. arXiv:2608.10715v1 [cs.CL]. 11 Aug 2026. Preprint, not peer reviewed.

2. Naddaf M. AI commonly used in peer review, often against guidance. Nature. 8 Jan 2026;649:273–274.

3. Naddaf M. How are researchers using AI? Survey reveals pros and cons for science. Nature. 2025. doi:10.1038/d41586-025-00343-5.

4. Kosmyna N, et al. Your brain on ChatGPT: accumulation of cognitive debt when using an AI assistant for essay writing task. arXiv:2506.08872. 2025. Preprint, not peer reviewed.

5. Lee H-P, et al. The impact of generative AI on critical thinking: self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. CHI ’25. 2025.

6. Gerlich M. AI tools in society: impacts on cognitive offloading and the future of critical thinking. Societies. 2025;15(1):6. doi:10.3390/soc15010006.

 
 
Raymond, Caitlin M._edited.jpg

Caitlin Raymond MD/PhD

I'm a hybrid of Family Medicine and Pathology training with a PhD in Neuroscience. I write about things that make me curious, medicine most of all. 

  • Twitter
  • LinkedIn
  • Instagram

​

​

Subscribe

Thanks for submitting!

©2023 by Caitlin Raymond. Powered and secured by Wix

bottom of page