Context Window vs Memory
A context window is what the model can actively use right now; memory is information that may be stored and retrieved when needed.
Key terms
Context window (now)
Active prompt
Recent conversation
Retrieved text
What fits here right now is what the model can use right now.
Memory (stored)
Session state
Saved preferences
Stored summaries
Only useful when the right pieces are retrieved back into the active context.
Business impact
How this shapes cost, speed, risk, and control.
Continuity
Persistent-memory driven
Context alone cannot create cross-session continuity.
Governance impact
High
Persistent memory needs clear policy on what is stored and who can see it.
Cost
Longer context = more tokens
Retrieval lets you keep context tight while reaching broader knowledge.
What can go wrong
Common failure modes to watch for when this concept shows up in production.
Assuming the model remembers everything
Without retrieval, prior conversations or documents are invisible to the model.
Context overflow
Stuffing the window too full crowds out the signal and increases cost.
Wrong memory retrieved
Bad retrieval pulls irrelevant or stale memory into the active prompt.