The archive · Work & Ways of Doing · Operational decision · 2003–2016
Google SRE turns ops into software engineering — reliability becomes a budget
Google in 2003 had software engineers design operations; SLOs set an error budget that teams spend on shipping instead of chasing 100% uptime.
What it had to solve
By 2003 Google's growth made the traditional operations model unworkable — hiring enough operators was cost-prohibitive and near-impossible — so VP Ben Treynor Sloss had to invent a different way to run production.
How it works
Site Reliability Engineering started at Google in 2003 out of necessity. Google's services were growing so fast that the traditional operations model — hiring more operators to manage machines — was cost-prohibitive and near-impossible to staff. VP Ben Treynor Sloss decided to engineer his way out of the problem, in the formulation he later made famous: 'SRE is what happens when you ask a software engineer to design an operations function.'
Mechanically, SRE teams are software engineers who run production, writing code that does the work sysadmins once did manually. Google capped SREs' manual ops work at 50% of their time so the rest went into automation, and it defined reliability as a budget: teams agree a service-level objective, and the gap to 100% becomes an error budget they can spend on shipping features. When the budget is exhausted, launches freeze until stability returns.
The idea also changed how failure is handled: postmortems are blameless, because human error is treated as a system problem. As Google SRE education director Jennifer Petoff put it, you can't fix people, but you can fix systems and processes so people make better choices — and if blame is punished, incidents get hidden instead of fixed.
Google published its practices in 2016 as the free book 'Site Reliability Engineering: How Google Runs Production Systems,' turning an internal discipline into a portable one. SLOs, error budgets and toil reduction became standard across the industry, and by 2021 Google itself was advertising more than 600 SRE roles across Alphabet.
Why it lands
- It reframed a hiring problem as an engineering problem: instead of feeding machines with human toil, SREs write software that does the work.
- The error budget makes reliability a currency both sides can trade: developers spend it on features, ops reclaims it through stability work.
- The 50% toil cap institutionalizes automation — an ops team that is fully occupied reacting can never make the system more operable.
- Blameless postmortems remove the incentive to cover up failures, so incidents become data for fixing systems, not careers.
- Publishing the book made the practice portable: any company could adopt the discipline without hiring Google's engineers.
What it did
Error budgets aligned development, operations and business: when the budget is spent, launches stop until stability returns. Google published the free SRE book in 2016, and SRE became a standard industry discipline — Google itself advertised 600+ SRE jobs by 2021.
What you can take
When a problem stops scaling with headcount, redesign the job itself: give teams a measurable budget for imperfection, cap the toil, and fix systems instead of blaming people.
Since then
The SRE book, released free in 2016, turned Google's internal practice into an industry standard. SRE job titles spread across tech; cloud providers and startups adopted SLOs and error budgets as the normal way to run services. Inside Google, SRE continued to run its largest products, and by 2021 the company was advertising over 600 SRE positions. The discipline also fed the DevOps movement, which absorbed many of its practices. Because the original idea was published rather than kept secret, its influence is measured in job titles and tooling — not revenue.
Sources
- The Origins of SRE from the Director of SRE Education at Google
- What is Site Reliability Engineering?
spotted an error? The archive wants to know.
Your turn
You just read one. Describe the brief you are staring at, and see who has been given the same problem.
Free account · 3 free questions · no card