What the term "context window" means, and the two misconceptions it carries.
Definition
The context window is the span of tokens a model attends to in a single
request: the prompt, any retrieved material, and the generated answer.
Misconception one: it is memory
It is not. Nothing outside the window is available to the model at all,
and nothing inside it persists once the request ends.
Misconception two: full is fast
Attention cost grows with window fill. A half-full window of junk is
slower and worse than a small window of exactly the right material.