Outlines guarantees JSON from LLMs — masking tokens makes invalid output ungeneratable

In mid-2023 the obstacle between a language model and a production app was not raw capability but shape: an LLM answers in prose, while software needs typed records, enums and booleans. The standard remedy was generate-then-repair — prompt for JSON, parse it, validate it, and call the model again when the output broke — and Microsoft's TypeChat, discussed in the launch thread, typified that approach. Outlines, shown on Hacker News on August 14, 2023, proposed the opposite: make the invalid output impossible to generate.

The library's API mirrors Python typing. A caller passes the required shape to the model — Literal["Positive", "Negative"], int, a Pydantic model or a JSON schema — and Outlines compiles that type into a finite-state machine. At every decoding step it masks tokens that would push the response outside the allowed structure, so the model samples only from completions the type accepts. The authors built the index once at initialization, then built each step's mask with a dictionary lookup, avoiding the full-vocabulary scans of grammar sampling and needing no second call to fix a bad answer.

The thread scored 854 points and 303 comments, with engineers comparing the mechanism to Microsoft's TypeChat and to guidance libraries, and the project kept building on the idea: by the September 2026 capture the repository carried 15.7k stars, 868 forks and 1,325 commits, and its feature table had grown to multiple choices, function calls, JSON/Pydantic schemas, regular expressions and grammars, running across transformers, llama.cpp, vLLM, Ollama and hosted APIs.