The archive · Product Ideas · Product decision · 2024
Cognition's Devin runs whole dev jobs as an autonomous AI teammate
Devin is an AI software engineer that plans, codes, debugs and tests tasks end-to-end — setting a then-record 13.86% on SWE-bench.
Cognition AI
What it had to solve
Coding assistants like GitHub Copilot generated snippets, but planning, debugging and testing still fell on the human. Cognition set out to build an agent that completes whole engineering tasks and reports back.
How it works
By 2024, AI coding tools had settled into one shape: an autocomplete or chat copilot that suggests code while the human plans, runs and fixes it. Cognition, a startup founded by competitive programmers, bet that the next step was not a better copilot but an engineer.
Devin is given a task in natural language and then works inside a sandboxed compute environment with its own shell, code editor and browser. It builds a step-by-step plan, writes and edits code, runs tests, fixes failures and posts progress in real time; the human can step in through the chat interface and redirect it at any point.
The company demonstrated Devin completing jobs end-to-end — deploying an app, fixing bugs, even taking an Upwork project — and reported 13.86% on SWE-bench's real GitHub issues without human assistance, against 4.80% for Claude 2 and 1.74% for GPT-4, which also needed file hints.
Why it lands
- The idea reframes the product category: instead of assisting a human coder, the AI owns the task and the human supervises.
- Giving Devin its own shell, editor and browser means it works through real developer tools, not canned completions.
- Posting progress in real time keeps the human in the loop without waiting for a final result.
- The SWE-bench result made autonomy measurable — 13.86% of real GitHub issues resolved end-to-end versus 4.80% for the best prior model.
- Positioning Devin as a 'teammate' changed the pitch from cost-cutting to delegating grunt work.
What it did
On the SWE-bench benchmark Devin resolved 13.86% of real GitHub issues end-to-end with no human help, versus 4.80% for Claude 2 and 1.74% for GPT-4 — and Cognition introduced it in March 2024 as the first fully autonomous AI software engineer.
What you can take
The next step of a product isn't a faster version of the current interaction — it's changing who does the work: the AI becomes the executor, the human the reviewer.
Since then
Devin launched in March 2024 with early access limited to select users, backed by $21 million in funding from Founders Fund and others. Cognition said coding was 'just the beginning' of the AI workers it planned to build, and framed the product as a glimpse of computer work overseen by humans rather than performed by them.
Sources
- Introducing Devin, the first AI software engineer
- Cognition emerges from stealth to launch AI software engineer Devin
spotted an error? The archive wants to know.
Your turn
You just read one. Describe the brief you are staring at, and see who has been given the same problem.
Free account · 3 free questions · no card