Mutable.ai leapfrogs vector RAG with a generated 'language map' wiki of each codebase

Mutable.ai wanted to give developers a chat that genuinely understood their code. The naive route — plugging a repository into a large language model — failed on two hard limits: context windows too small for most codebases, and models that answer immediately instead of reasoning step by step. Their first attempt, a codebase chat built on keyword retrieval plus vector embeddings, failed in a more instructive way: even after the team trained its own embedding model, questions like 'how do I quantize llama.cpp' returned answers built from the wrong context, often pulled from test files.

The creative move was to stop retrieving and write the codebase down first. Auto Wiki generated a Wikipedia-style article for each repository, with diagrams and citations pointing at the underlying code. The chat then retrieved through those citations — reading the relevant wiki sections plus the actual code they reference — which the founders called a 'language map' of the codebase. Answers flipped from wrong to accurate, comprehensive and cited, because the model received language and structure instead of snippets that merely sat nearby in vector space.

The Show HN thread on 18 July 2024 read like a map of an idea whose time had come. Commenters compared the approach to Aider's AST-driven repository map, which held the second-highest SWE-bench score without code RAG; one described nearly identical 'knowledge mapping' work at a consultancy; others framed it as graph RAG. The founders answered in-thread, offered free processing for open-source repositories named in the comments, and pointed to a growing list of generated wikis — llama.cpp, Terraform, go-ethereum, NVIDIA TensorRT, langchain and VS Code among them.