Chaos Monkey: Netflix breaks its own service on purpose so streaming survives

Netflix ran its streaming service on AWS, where instance failures are common and outages had bitten the industry hard; instead of waiting to be surprised, the engineering team decided to surprise itself.

Break production on purpose: Chaos Monkey randomly turns off live AWS instances so engineers must build services that survive without customers noticing.

A small tool picks an instance at random from each cluster and, during business hours on a workday, turns it off without warning — with engineers standing by to watch how the service reacts. Netflix open-sourced Chaos Monkey in July 2012, then added Janitor Monkey and Security Monkey to the 'Simian Army'; its stated rule: 'the best defense against major unexpected failures is to fail often.'

Chaos Monkey forced Netflix's architecture to tolerate instance loss without customer impact and turned failure drills into routine; the open-sourced tool got coverage in TechCrunch and Wired, Stack Exchange's Jeff Atwood credited the same idea for fixing his outages, and the practice grew into the discipline of chaos engineering.

Netflix ran its streaming service on AWS, and in the cloud, instances fail — hosts die, network blips, load balancers misbehave. Rather than wait for an outage to expose the weak spots, the engineering team built Chaos Monkey, software that randomly disables production instances during business hours, while engineers stood by to watch.

The name came from 'unleashing a wild monkey with a weapon in your data center to randomly shoot down instances and chew through cables — all the while we continue serving our customers without interruption.' The point was not the tool itself but the change in mindset: if your system must survive random daily attacks, you build automatic recovery instead of hoping nothing breaks.

In July 2012 Netflix open-sourced Chaos Monkey, and the idea spread. Coverage in TechCrunch and Wired made 'fail often' a known engineering motto; Stack Exchange's Jeff Atwood said his team only solved its outage problem after the 'Chaos Monkey chose them'. The practice was later formalized as chaos engineering — experimenting on production systems to build confidence they can withstand turbulent conditions.

Random, unattended failures during work hours turn rare terrifying outages into routine drills with engineers ready.

Failing often forces automatic recovery mechanisms, so a 3am failure stops being a crisis.

Open-sourcing gave the idea legs — other companies adopted the practice, and it became a discipline.

The name made the concept unforgettable, which is why the practice spread as fast as it did.

Don't wait for the outage you can't predict: induce small failures during the day, with engineers watching, so the system — and the team — learn resilience before a 3am disaster teaches them.

Netflix kept expanding the idea: Latency Monkey, Doctor Monkey, Chaos Kong for whole availability zones, and later the Chaos Automation Platform, with a dedicated chaos engineering team formed in 2015. The original Simian Army repo was retired in 2021, replaced by standalone tools, but the practice outgrew Netflix — chaos engineering became an industry discipline with books, conferences, commercial tools and adopters like Capital One.