Your AI Reads the Edges and Skips the Middle — and Nobody Has Fixed It
Here is an uncomfortable fact about the AI tools you use every day: where you put information often matters as much as what it says. Ask about a clause near the top of a contract and you usually get it right; bury that clause in the middle and the model may confidently invent a different one, though the text sits right there. It's not a glitch but a structural quirk of how large language models read — and three years of effort and far larger context windows haven't fixed it.
The U-shaped memory
The effect is "lost in the middle," after a 2023 Stanford study by Nelson Liu and colleagues. Give a model a question and documents with exactly one holding the answer, then slide it from first to middle to last: accuracy traces a U, high at the edges and dropping more than 20 points in the middle. The models weren't failing to read the middle, but to weigh it. ([Liu et al., 2023](https://arxiv.org/abs/2307.03172)) Long-context models haven't outgrown it: on the harder RULER benchmark, all seventeen tested showed recall fading as input grew. ([Hsieh et al., 2024](https://arxiv.org/abs/2404.06654))
The debate: a bug to patch, or a flaw at birth?
Here it gets interesting: two camps look at the same U-shaped curve and draw opposite conclusions.
Camp one says it's an engineering problem we're already fixing. The "Found in the Middle" team traced the U to an attention bias — models favor the start and end regardless of relevance — and recovered up to 15 points by recalibrating attention. ([Hsieh et al., 2024](https://arxiv.org/abs/2406.16008)) Another gained up to 15.2% by scaling a single dimension of the hidden state. ([Yu et al., 2024](https://arxiv.org/abs/2406.02536)) Even crude prompts help: Anthropic found that priming with "Here is the most relevant sentence in the context:" lifted accuracy on a 200,000-token test from 27% to 98%.
Camp two says the curve is wired into the architecture. A 2025 analysis from MIT and the Technical University of Munich found the causal mask — what lets models read left-to-right — inherently tilts attention toward earlier tokens, while position encoding pulls the other way. The U falls out of the math of how transformers are built, not messy training data. ([Wu et al., 2025](https://arxiv.org/abs/2502.01951)) A 2026 follow-up argues it's present at initialization, before training — a newer theoretical result worth treating with caution until replicated. ([Source](https://arxiv.org/abs/2603.10123))
Both camps are partly right, and that's the honest takeaway. The mitigations help — but they're patches applied around a tendency that keeps reasserting itself, just what you'd expect if it's structural. That the fixes recover 15 points, not the full gap, tells you something.
Why this lands in your actual life
Strip away the math and the stakes are concrete: the things we ask AI to read are long, and the important bit is rarely at the top. A hiring tool skimming CVs where the disqualifying line sits mid-history. A lawyer asking about an indemnity clause buried on page 47. A tool summarizing a record where a critical allergy was noted three visits back. The model has the information; the question is whether it attends to it. It also reframes the "one million tokens!" race: the more you stuff in, the more you bury — a bigger haystack, not a stronger magnet.
The defenses, fortunately, are simple. Put the information you care about at the start or end of your prompt, not buried in a wall of text. Restate your question right before asking for an answer. And resist pasting a whole document when an excerpt would do — trimming the haystack beats enlarging it.
The takeaway
Lost-in-the-middle is a useful reminder that these systems don't read the way we picture. They don't scan a document evenly; they shine a spotlight brightest at the edges and dimmest in the center, whether or not the important part stands in the light.
The optimistic reading: we understand the effect well enough to design around it, and new architectures may flatten the curve. The sober reading: we've understood it for three years, thrown enormous resources at it, and it's still here. Either way, the AI is paying attention — just not everywhere. Put what matters where it's looking.