Tapio Vepsäläinen
← Back to blog

Does using an LLM actually improve your writing?

· 12 min read

Preface

I use LLMs quite a lot myself, and as you might guess, I used them to help write this post. I also rely on them heavily in software development.

To make matters even “worse,” I’m currently building a static website generator powered by LLMs. It’s quite a cool project, but it probably makes me even more biased on this topic. You can check it out here.

On the editing side, I used GLM 5.3 Flash and Sonnet 5.0 for this post, mostly because they’re fast. If I use a slower model like Opus or Fable, I end up context-switching while waiting for a response, which breaks my flow and makes it harder to compile coherent text. There’s probably a whole separate post to be written on what makes a good writing model and how much of that depends on why someone reaches for one in the first place (speed, poor grammar, writing in a foreign language, or something else entirely).

What is good writing?

To be fair, the question in the title is inherently flawed. It is overly broad and completely subjective. However, I think it is still a question a lot of people are asking.

People don’t just disagree about whether AI-assisted writing is good. Many argue about it the way they would argue about a political identity. Some people who dislike AI writing don’t just think it is lower quality; they seem to find it distasteful in a way that is closer to an aesthetic or moral reaction than a quality judgment. On the other hand, some enthusiasts see nothing wrong with any of it.

A lot of what people are actually disagreeing about is taste, and taste is notoriously hard to research. You can measure whether a sentence is grammatically correct (usually) or whether a fact is right (sometimes). It is much harder to measure whether a piece of writing is good, because that depends on who is reading it and what they already value.

While working on my website generator, I wanted to find some answers, partly to have solid arguments ready whenever someone asks whether using AI to generate content actually makes sense. I looked for research that directly answers “does AI make content better,” and I don’t think it exists in a form that would settle the argument. Coming up with a reliable methodology to objectively measure “good” writing is nearly impossible, which is why there are virtually no papers tackling it head-on.

Doing it wrong

My initial stance has been that the answer is nuanced and depends heavily on who is using the tool and how. Someone with strong taste and a clear sense of what they want to say can use AI as a fast typist and end up with something that is still theirs.

Someone without a clear sense of what they want to say, or without the taste to recognize a bad sentence, can end up publishing something worse than what they would have written badly by themselves, just faster and with more confidence. I’ve read many blog posts halfway through, thinking I wish the writer had just given me the prompt they used to generate it, rather than making me read the slop.

Similarly, some people just love to send AI-generated content, unedited, as if it were a message from the person. That is rude. If someone wants to talk to an AI, they can do that themselves. Using a person as a relay for a chatbot’s output, without adding anything, treats the recipient’s time and attention as worth less than yours. Don’t be a meat proxy for a language model, like this post suggests.

Shifting the focus to cognitive mechanisms

I could find three relevant streams of research that might help give a bit more educated answer to the question:

  • Anchoring: an initial number or idea you are exposed to pulls your later judgment toward it, even when you know the starting point is arbitrary.

  • Design fixation: seeing an example narrows the range of ideas you generate afterward.

  • Automation bias: people trust a machine’s output even when it is visibly wrong, and stop weighing other evidence.

What ties these together, and what I think is the actual point of this piece, is that all three are largely subconscious. They don’t require you to be careless, uninformed, or unaware of the risk. They show up in domain experts, in people who were explicitly warned in advance, and in people who are actively trying to resist them. Knowing about a bias is not the same as being immune to it.

Anchoring

Anchoring is one of the more solid findings in the heuristics-and-biases literature. I was sort of a fan of Kahneman’s Thinking, Fast and Slow when it came out, although I understand it was a popularized piece. I’ve read a lot about heuristics and biases in my research on election forecasting, and I understand it’s really nuanced. Nevertheless, I think anchoring has something to do with how using LLMs shapes the outcome of our writing.

Anchoring means that an initial number or idea you’re exposed to pulls your later judgment toward it, even when that starting point is obviously irrelevant or arbitrary. The original demonstration showed this with an obviously random number: a rigged wheel-of-fortune spin shifted people’s later numeric estimates, and offering people money for accuracy did not reduce the effect.1

Other researchers have run their own experiments that support the finding. When a large-scale replication project ran thirteen classic psychology effects across thirty-six samples and more than six thousand people, the four anchoring tasks came out as the four largest effects tested, and they replicated in every single sample.2 A separate line of work went further and found that anchoring survives being warned about in advance, that most people who insisted they hadn’t been influenced had been, and that an accuracy incentive reduced how much people believed the anchor had affected them without reducing how much it actually did.3

I think this is relevant to AI-assisted writing because a generated draft is, functionally, an anchor: a specific set of words, framings, and choices you’re exposed to before you’ve formed your own. If anchoring generalizes to text the way it generalizes to numbers, and I think it’s reasonable to suspect it does even without a study that tests this exact case, starting from an AI-generated draft probably shapes what you end up with in ways you won’t notice, and won’t be able to talk yourself out of just by trying harder.

Design fixation

The design-fixation literature points in the same direction, from a different angle. One study found engineers who were shown a flawed example design reproduced its flaws even after being told about them explicitly,4 and a later meta-analysis across many studies found that seeing an example narrows the range of ideas people generate.5

On the flip side, the same research shows that having examples did still increase the quality of the outcome. It’s then quite possible that in certain situations, having an AI example improves the outcome. There will be a lot of confounding factors here that can’t be explicitly named, and everyone will, and can, theorize about when it helps them. The point is, however, that the presence of examples can improve the outcome according to the design fixation research.

Automation bias

The final stream of research I briefly looked at was the research on automation bias, mostly studied in aviation and medicine. Automation bias is a situation where users trust the machine even when it’s clearly giving them the wrong answer, and end up ignoring every other piece of feedback in front of them. That trust can be strong enough to cause fatal mistakes.

A rather large systematic review across seventy-four studies of clinical decision support found that when the system gave wrong advice, it measurably increased the rate at which clinicians made the wrong call, even though these are domain experts reviewing suggestions in their own specialty.6 A more recent randomized trial tested this on physicians who had already completed dedicated AI-literacy training. Physicians who saw AI-assisted diagnoses seeded with errors got the right diagnosis 73.3% of the time, versus 84.9% for physicians who worked without AI suggestions at all. The training on how to detect false AI suggestions was not enough to stop the accuracy from dropping.7

I’d say automation bias is here kind of a distant thing, but the mechanism still feels relevant: when a model’s output feels authoritative, we’re inclined to trust it. In terms of writing, a well-written sentence or paragraph from an LLM is something we’re scared to touch, because it already looks finished, and nothing about it signals that it might be wrong.

This is where the practical observation from earlier fits: current models are decent writers at the sentence level, often better than an average person’s first draft. That’s exactly what makes editing them hard. A clumsy first draft is obviously clumsy, and you know where to cut. A fluent, competent draft reads as finished, so the parts that are subtly wrong, generic, or not quite what you meant are easy to let slide, because nothing about the sentence signals “fix me”. I think this is a real skill gap, not a character flaw: recognizing that a well-written sentence is still the wrong sentence takes more effort than recognizing that a badly-written one is wrong. It’s a similar mechanism to automation bias, just applied to prose instead of a diagnosis.

Research gap

The evidence from these three streams of research suggests that the models definitely have influence on the final output. However, there does not seem to be any definitive answers whether that’s to a good or bad direction. The final answer is also about individuals: for some people it could be good, for another bad.

There is obviously a research gap here for some enthusiastic PhD student. Someone could try to figure out how to actually measure this: recruit people, have some write with an AI draft and some from scratch, and get independent judges to score the results without knowing which is which. It would probably be close to impossible to pull off properly, and would likely take years to even agree on what “better” means before a single participant is recruited, but if someone wants to spend their PhD on it, go for it!

Conclusions

Most of what I’ve read on this topic is either identity-driven, “AI slop” versus “it’s all fine”, or narrow craft advice about prompting. I tried to answer the question “does AI make writing better?” directly, but I couldn’t. The more I tried, the more clear it became that it cannot be answered directly. What I ended up with instead were dimensions you’d actually need to measure before that question could be answered objectively. I found at least these: how much a draft narrows the range of ideas someone considers, whether the quality gain from an example is real or just agreement with the anchor, and how much unearned trust a fluent draft gets compared to a clumsy one. There are almost certainly more, and I might’ve ignored the most obvious ones even. That’s what this piece actually gives you: not an answer to “is AI-assisted content good or bad”, but a first list of things to look at when you’re trying to judge a specific piece of it yourself.

Those dimensions are also why I don’t think there’s a single answer to “does AI make content better”. For someone with strong opinions about what they want to say, AI is probably a faster way to say it, and the result can genuinely remain their own voice. For someone without strong opinions yet, starting from an AI draft probably locks in the AI’s opinions instead, and the anchoring research suggests that happens even when you’re trying to guard against it.

I didn’t find any practical implications for my own product, for or against AI-generated websites. If a website is just a simple page of information that the entrepreneur can verify, I don’t think it really matters at all. If the company didn’t have a website before, and was just present on social media platforms, it’s also net beneficial for people to have their own information on their own domain.

However, the first draft still comes with a cost that’s hard to see and hard to measure, and I’m slightly worried that the tools I’m working on will be used to generate more slop. If that ends up being the case, I’ll make sure we add one of those consent tickboxes that require the user to agree to something like “I agree that I have read the content I’m now publishing online, and I also agree to donate my second kidney, because I always click on these without even reading what they say.”

Summa summarum: we don’t know whether starting from an AI draft makes content better or worse than writing it from scratch. We can’t know, because we don’t have the version of the text a person would have written without the draft.

Footnotes

  1. Amos Tversky & Daniel Kahneman, “Judgment under uncertainty: Heuristics and biases”, Science 185(4157), 1974, pp. 1124–1131; anchoring on p. 1128. doi:10.1126/science.185.4157.1124 ↩

  2. Richard A. Klein et al., “Investigating variation in replicability: A ‘Many Labs’ replication project”, Social Psychology 45(3), 2014, pp. 142–152. 36 samples, N = 6,344. doi:10.1027/1864-9335/a000178 ↩

  3. Timothy D. Wilson, Christopher E. Houston, Kathryn M. Etling & Nancy Brekke, “A new look at anchoring effects: Basic anchoring and its antecedents”, Journal of Experimental Psychology: General 125(4), 1996, pp. 387–402. doi:10.1037/0096-3445.125.4.387 ↩

  4. David G. Jansson & Steven M. Smith, “Design fixation”, Design Studies 12(1), 1991, pp. 3–11. doi:10.1016/0142-694X(91)90003-F. Worth noting this study was small and, as reported, contains no statistical comparison between groups; treat it as a demonstration rather than decisive evidence. ↩

  5. Sio, Kotovsky & Cagan, “Fixation or inspiration? A meta-analytic review of the role of examples on design processes”, Design Studies 39, 2015, pp. 70–99. doi:10.1016/j.destud.2015.04.004 ↩

  6. Kate Goddard, Abdul Roudsari & Jeremy C. Wyatt, “Automation bias: a systematic review of frequency, effect mediators, and mitigators”, Journal of the American Medical Informatics Association 19(1), 2012, pp. 121–127. Risk ratio 1.26, 95% CI 1.11–1.44. doi:10.1136/amiajnl-2011-000089 ↩

  7. Ihsan Ayyub Qazi et al., “Automation Bias in Large Language Model–Assisted Diagnostic Reasoning among Physicians Trained in AI Literacy — A Randomized Clinical Trial”, NEJM AI 3(5), 2026. doi:10.1056/AIoa2501001. 44 physicians, single site; treat the magnitude as indicative rather than settled. ↩