Anyone who’s spent time doing RF engineering knows the concept of signal-to-noise ratio (SNR) cold. You’ve got a signal, you’ve got a noise floor, and the gap between them determines whether the receiver can actually make sense of what’s coming in. Raise the noise floor high enough, and it doesn’t matter how strong the signal is. It gets swallowed.
Large language models have a noise floor problem, and it’s not the kind you fix with a firmware update. It’s baked into the architecture. Every time you’ve felt like you’re fighting an LLM to get it to just say the thing directly, without the hedging, the preamble, the “it’s important to note that,” you’re running into a structural property of how these models are built and trained. Understanding why that noise exists tells you exactly what prompt and context engineering are actually for, and why they’re not optional polish.
What does “signal” even mean for a language model?#
In RF terms, signal is the thing you want and noise is everything else competing for the same spectrum. For text, signal is the precise, correct, non-redundant information that answers your question or completes your task. Noise is filler, hedging, generic phrasing, and statistically safe language that doesn’t add information but doesn’t hurt the model’s training objective either.
The reason LLMs default to a low SNR isn’t that they’re bad at writing. It’s that the thing they were optimized to do isn’t the same thing as writing well. That distinction matters, and it’s where the whole problem starts.
Next-token prediction optimizes for plausibility, not precision#
An LLM is trained to predict the next token given everything that came before it. That’s the entire objective function. Not “be correct.” Not “be concise.” Not “say something worth saying.” Just: given this sequence, what token is statistically most likely to come next, based on a corpus that includes everything from peer-reviewed papers to Reddit threads to marketing copy to forum arguments.
That corpus is enormous and heterogeneous by design, because more data generally improves the model’s ability to generalize. But heterogeneity has a cost. The training signal is diluted by the sheer volume of mediocre, hedged, redundant, and low-information text that makes up most of human writing. Most writing, honestly, is noise. Most emails, most blog posts, most Slack messages. The model learns from all of it, weighted by frequency, not by quality.
So when you ask an LLM a direct question, it’s not retrieving “the correct answer” from some clean index. It’s generating the statistically plausible continuation of a prompt, shaped by a training distribution where verbose, hedged, generically-worded text is extremely common. Garbage in, garbage out isn’t just a data pipeline problem. It’s the literal training objective.
Attention doesn’t choose. It blends.#
Here’s the part that gets missed even by people who use these models daily. The transformer architecture’s attention mechanism doesn’t select a single “best” next token and commit to it the way a human writer commits to a word choice. It computes a weighted average over the entire vocabulary distribution, conditioned on every token in the context window attending to every other token.
That means the output at each step is a probabilistic blend of many possible continuations, not a deliberate authorial decision. When a human writer chooses a word, they’re drawing on intent, prior knowledge of the audience, and a mental model of what they’re trying to say. When a model “chooses” a token, it’s sampling from a distribution that represents an average over thousands of contexts the training data resembled.
This is why LLM output so often reads as competent but generic. It’s not wrong, exactly. It’s the statistical center of mass of everything similar it’s seen before. Averages are, by definition, low-variance and low-signal. You rarely get a sharp, distinctive point of view from an average. You get the safest, most broadly applicable version of an answer, because that’s what minimizes the model’s loss across the entire training distribution.
Human writers filter. LLMs skip that step by default.#
When you write something and publish it, you’ve gone through a filtering process that took place before anyone else saw a single word. You decided what mattered, cut what didn’t, revised the parts that were vague, and killed sentences that didn’t earn their place. That filtering is powered by expertise, intent, and editorial judgment, none of which are things an LLM has natively.
An LLM’s raw output skips all of that. There’s no editor in the loop unless you are the editor. There’s no internal voice saying “this sentence is filler, cut it,” because the model doesn’t have a concept of “filler” as distinct from “content.” It has a probability distribution. Hedge phrases like “it’s worth noting” or “in many cases” show up constantly in LLM output not because the model is being cautious, they show up because those phrases are extremely common in the training corpus and statistically safe continuations in almost any context.
This is the honest answer to “why are LLMs so verbose.” It’s not a personality quirk. It’s the absence of the authorial filtering pass that human writers perform automatically and invisibly, every single time, before anyone else reads their work.
Why prompt and context engineering exist at all#
Once you see the noise as architectural, prompt and context engineering stop looking like a soft skill and start looking like exactly what they are: an attempt to manually impose the filtering step that human authorial judgment provides for free.
Notice what most effective prompt engineering actually consists of. It’s rarely “generate more.” It’s almost always constraint. “Don’t include a summary at the end.” “Respond in one paragraph.” “Do not hedge. State the answer directly.” “Do not use the word ’leverage.’” These are negative instructions, and negative instructions exist because the model’s default behavior, absent constraint, is to reproduce the noisy, hedged, average-of-everything output that its training distribution encourages.
Context engineering does the same job from a different angle. Instead of constraining what the model outputs, it constrains what the model attends to. By narrowing the context window to only relevant, high-quality information, you’re artificially raising the SNR of the input, which raises the SNR of the output, because the model’s attention mechanism can only average over what’s actually there. Feed it a clean, tightly scoped context and the averaging happens over a much narrower, higher-quality distribution. Feed it everything and let it figure it out, and you get the statistical mush that comes from averaging over noise.
This also explains why so much of the discipline feels like pruning rather than generation. You’re not teaching the model new information most of the time. You’re cutting away the paths that lead to low-signal output and narrowing the model toward the paths that produce what you actually want. It’s editorial work, just performed upstream instead of downstream.
The practical implication#
If you’re treating prompt and context engineering as a bag of clever tricks, you’re missing what they’re actually compensating for. They exist because the model has no built-in mechanism for authorial judgment, and the training objective it optimizes for actively works against precision and concision.
That means the work isn’t finished once you find a prompt that works. It means you should expect to keep doing this work, indefinitely, for every task, because the noise floor doesn’t go away. It’s structural. You’re not fixing a bug. You’re building a filter, every time, around a system that was never designed to filter for you.
The engineers who get the most out of these models aren’t the ones with the cleverest prompts. They’re the ones who’ve internalized that the model’s raw output is closer to a first draft generated by averaging over the entire internet than it is to a finished thought. Treat it that way. Constrain aggressively, narrow the context ruthlessly, and don’t expect the model to know what to cut. That’s still your job.
If you found this post useful, you might also be interested in:
- The LLM Engineer's Handbook by Paul Iusztin and Maxime Labonne
- Generative AI Design Patterns by Valliappa Lakshmanan and Hannes Hapke
Featured image by Egor Komarov on Unsplash
Recommended Reading#
- The LLM Engineer's Handbook by Paul Iusztin and Maxime Labonne
- Generative AI Design Patterns by Valliappa Lakshmanan and Hannes Hapke