The archive · Business Models · Strategic decision · 2006–2009
Netflix offers $1M to the world to beat its recommendation engine — 50,000 enter
Netflix turned its recommendation problem into a public contest: $1 million to whoever improved Cinematch by 10%, with 100M ratings released.
Netflix
What it had to solve
In 2006 Netflix's Cinematch recommendation engine had stalled: the machine-learning literature held many promising ideas, but Netflix could not test them all and no public dataset let outside researchers compete. The company wanted the experiments it could not run internally.
How it works
In October 2006 Netflix announced the Netflix Prize: $1 million to the first person or team who could improve the accuracy of its Cinematch recommendation system by 10 percent. To make the contest possible, the company released 100 million anonymous movie ratings and invited the data-mining, statistics and machine-learning communities to compete.
The design borrowed from history — Netflix said the prize was modeled on the 1714 Longitude Prize — and added $50,000 annual progress prizes for the best improvement. Entries were judged against a private test set, so algorithms could not be over-tuned to the public leaderboard.
Within a week of the announcement, 9,940 contestants on 8,152 teams from 99 countries had registered. Nearly three years later, about 50,000 contestants had taken part; BellKor's Pragmatic Chaos — AT&T researchers teamed with Austrian and Quebecois collaborators who had never met in person — won with a 10.06 percent improvement, submitting just ten minutes before a rival with an identical score.
Why it lands
- Opening the problem to outsiders let Netflix test thousands of approaches it could never explore internally, for one fixed $1M price.
- Releasing 100M anonymized ratings removed the data barrier that had kept academic researchers away from the problem.
- A measurable 10% bar and a public leaderboard turned a private engineering goal into a transparent global game.
- Paying only for success turned innovation into a buyer's market: Netflix bought a result, not a team or a project.
What it did
Within a week of the announcement, 9,940 contestants on 8,152 teams from 99 countries had registered. Nearly three years later, about 50,000 contestants had taken part; BellKor's Pragmatic Chaos won with a 10.06 percent improvement, submitting just ten minutes before a rival with an identical score.
What you can take
When your team cannot test every idea, open the problem: publish the data, set a measurable bar, and pay for results — you get thousands of experiments for one prize.
Since then
The contest ended after nearly three years and about 50,000 contestants: BellKor's Pragmatic Chaos won the $1 million with a 10.06 percent improvement, beating a tied rival by ten minutes. The released dataset became a standard benchmark in recommendation research, and the prize became the reference point for open-innovation contests in the decade that followed.
Sources
- Netflix $1 million prize for improving accuracy of movie recommendations
- The $1 Million Netflix Challenge
- BellKor's Pragmatic Chaos Wins $1 Million Netflix Prize by Mere Minutes
spotted an error? The archive wants to know.
Your turn
You just read one. Describe the brief you are staring at, and see who has been given the same problem.
Free account · 3 free questions · no card