sysnote
Research

Anthropic details "dreaming" for Claude's memory: offline passes that fix notes and build wikis, with results from its own evals

Anthropic has published how it builds memory for Claude agents, and the most concrete part is "dreaming": offline sessions in which Claude rereads its own transcripts, checks its memory notes against them, fixes mistakes and reorganizes what it has stored. The engineering post, written by Lance Martin and published November 6, borrows the idea from sleep-time compute, proposed by researchers at Letta and UC Berkeley, and reports results from several internal evaluations.

The numbers are Anthropic's own and come from small test sets, but a few are worth knowing:

  • Wikis built by dreaming. On about 350,000 internal files (roughly 420M tokens) from Stainless, the company Anthropic acquired in May, Opus 5 agents built a linked wiki over 128 sessions. Answering 25 questions from the wiki alone went from 13% of graded criteria after two sessions to 48% after 128. With a search tool plus the wiki the score rose from 54% to 61%, against 53% for search alone, and used about 20,000 fewer tokens per question.
  • Where wikis help and where they don't. On 24 questions about a merger, most needing facts from several documents, Claude scored 87% with the wiki and 73% with search alone. On a Harvey LAB legal-brief task whose criteria each sat in one document, the wiki showed no clear gain, and on another task the wiki grew larger than the source documents.
  • Instructions matter more than volume. In one LAB acquisition task, several agents writing into one shared table of contract terms overwrote each other, and only 41 of 57 graded facts survived. After Claude rewrote its own wiki-building instructions to give each contract its own page and quote terms verbatim, all 57 survived.
  • Fixing a wrong note. In Claude Plays Pokémon, a note that put a landmark on the wrong side of Mt. Moon sent Claude off a one-way ledge three times. A dreaming pass caught the error against the transcript, and with the corrected note Claude got through to Cerulean City.

The post also argues that stronger models need less steering. On Continual Learning Bench 1.0, Opus 5 was the first model in Anthropic's comparison to close the loop of recording a failure, testing a theory and storing a validated rule; telling Opus 4.7 and Opus 5 to "record only failures" cost them much of the gain memory gave them. Older Sonnet 4.6 nearly doubled its score when prompted to read its memory and test its theories. In Pokémon, Sonnet 3.7 wrote 413 memory files in 225 folders and earned one badge by about step 10,000; Fable 5 wrote nine files plus a global.md dashboard and earned four.

One practical detail for Claude Code users: auto memory loads only the first 200 lines or 25 KB of MEMORY.md into each session, so the file works as an index, and the post says keeping that top layer small is much of what dreaming is for. Anthropic lists the open problems itself: dreaming costs compute, memories drift as environments change, and checking every note continuously may not be practical. It has published cookbooks for dreaming and knowledge wikis alongside the post.

Comments

No comments yet. Be the first.

Write a comment

Comments are published after review.