The archive · Product Ideas · Product decision · 2023–2026
ScratchDB makes the warehouse an API: post JSON, get a table, then query or share it
Scratch Data wraps ClickHouse, DuckDB and friends in an HTTP service where JSON payloads auto-create tables and expiring links share query results.
Scratch Data
What it had to solve
Powerful analytical databases existed, but getting data into them still meant pipelines, schemas and connectors — the annoying part of analytics. Scratch Data, which launched on Hacker News as an open-source answer to the Snowflake experience, built a wrapper that streamed data in and out of an analytics database over plain HTTP.
How it works
By 2023, analytical databases were fast and capable, but the experience around them still belonged to data engineers: every new dataset meant a pipeline, a schema, connectors and permissions before anyone could run a query. Scratch Data's answer was to make the database itself speak HTTP — a wrapper that accepts arbitrary JSON as input and performs analytical queries on it, with the database choice handled underneath.
The quickstart showed how far the ergonomics had moved. Clone the repository and run the Go server, and a local DuckDB is ready with no configuration. A single curl POST to the insert endpoint creates the events table and its columns automatically from the JSON; a query endpoint takes SQL; a share endpoint returns a link that serves the result as CSV or JSON for a set duration; and a copy endpoint runs a SQL query against one database and auto-creates the destination table in another. The same interface sat on top of ClickHouse, BigQuery, Redshift and other backends through a config file.
Posted to Hacker News on 27 October 2023, the project drew 261 points and 56 comments. It continued to mature as an open-source tool rather than a headline product: by the September 2026 repository snapshot it showed roughly 1.1k stars, 52 forks and 245 commits, and still introduced itself the same way — a swiss army knife for big data that streams data into and out of your analytics database.
Why it lands
- Speaking HTTP meant any script, service or developer tool could treat the warehouse as an endpoint instead of a project with its own SDK and pipeline.
- Auto-creating tables and columns from the JSON payload removed the schema step that stalled most ad-hoc analytics.
- Expiring share links turned query results into distributable artifacts, covering the reporting case without a dashboard product.
- The copy endpoint made moving data between warehouses a SQL statement rather than an ETL job.
What it did
The Show HN on 27 October 2023 drew 261 points and 56 comments, and the repository kept growing: the September 2026 snapshot showed roughly 1.1k stars, 52 forks and 245 commits, with the project still describing itself as a swiss army knife for big data and documentation live at docs.scratchdata.com.
What you can take
When the painful part of analytics is getting data in, design the ingestion experience: letting JSON auto-create schema turns a warehouse into something a developer can curl, not a project.
Since then
Scratch Data stayed on its original thesis after the launch: the wrapper approach let it default to DuckDB for zero-setup starts while supporting heavier analytical databases for production, and the repository continued to accumulate commits and forks through the 2026 snapshot. The project's quiet success was showing how much of the 'warehouse experience' is actually ingestion and sharing ergonomics — the parts Snowflake sold as products — and how far an open-source wrapper could get by turning them into a few HTTP calls.
Sources
spotted an error? The archive wants to know.
Your turn
You just read one. Describe the brief you are staring at, and see who has been given the same problem.
Free account · 3 free questions · no card