NOTES / PROMPTING AND CONTEXT ENGINEERING

Memory

Giving agents recall beyond one window: working notes, files the agent can read and write, and the OS-style idea of paging information in and out of context.

UPDATED 18 NOV 2026#memory#agents#memgpt#memory-tool

Why a model needs memory

A model has no memory between calls beyond what is in its prompt. Everything it “remembers” is text you supplied. For a single task the context window is enough. For work that spans sessions or exceeds the window, you need storage outside it.

The operating-system analogy

The MemGPT paper proposed treating the context window like main memory in an operating system, with external storage like disk. Borrowing the idea of virtual memory paging, it lets the LLM manage what sits in its own context, paging information in and out via function calls, to give the illusion of unbounded context with a fixed-window model1. The system alerts the model when context space is running low, so it can write important data to persistent memory, and it can search out-of-context data to bring it back in1.

Diagram: a fixed-size main context and large external storage, with information paged out and in

Diagram: my own, illustrating the idea in 1.

Memory as files the agent manages

A simple, robust pattern is a directory of files the agent reads and writes. Anthropic’s memory tool follows this shape: Claude can create, read, update and delete files in a memory directory that persist across conversations, building knowledge over time without keeping everything in the window2. It is a client-side tool: Claude requests the operation and your application executes it against storage you control, returning the result as a tool result. Your handler must reject paths outside the memory directory to prevent path traversal2.

Diagram: the model requests file operations, your handler executes them against the memories directory and returns results

Diagram: my own, following the flow in 2.

Anthropic’s engineering post describes the same idea as structured note-taking, such as a to-do list or a notes file the agent keeps outside the window and pulls back in later3.

Kinds of memory

This split is my own working taxonomy rather than one from the sources:

Kind What it holds Typical home
Working Current plan and progress A notes or to-do file
Episodic What happened in past sessions Logs, summaries
Semantic Stable facts and preferences A curated file or database
Procedural How to do tasks Instructions and skills

Design questions

  • What to store. Save decisions, constraints and unresolved items, not transcripts.
  • When to read. Load a short index at start and fetch detail on demand, to avoid refilling the window.
  • Staleness and conflict. Memories go out of date. Record dates and sources, and prefer updating over appending.
  • Privacy. Stored memory can hold personal data. Apply retention, deletion and access rules; see Privacy and regulation for a high-stakes example.
  • Security. Memory is an input like any other, so poisoned content can persist across sessions. See Prompt injection.
  • Retrieval at scale. Large memories need search, not just reading files. See Embeddings and vector search.

Memory pairs with compaction: write what must survive before summarising what need not.

Sources

  1. [1]MemGPT: Towards LLMs as Operating Systems — Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez · arXiv · 2023PARAPHRASED
  2. [2]Memory tool — Anthropic, Claude Platform docsPARAPHRASED
  3. [3]Effective context engineering for AI agents — Anthropic Engineering · 2025PARAPHRASED

© Ashish Kumar. All rights reserved. Quoted and cited material belongs to its owners. How I attribute sources.

Product, company and framework names are trademarks of their respective owners. Mentioning them does not imply affiliation or endorsement.