
Measuring Quack: DuckDB's New Client-Server Protocol
In my home, ducks are revered – a story arc that started in my childhood, with unexpected turns. This is not the story I’ll tell today. 💚
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Ramona C. Truta
Jun 22, 2026
5
3
Share
In my home, ducks are revered – a story arc that started in my childhood, with unexpected turns. This is not the story I’ll tell today. 💚
On my birthday this year, I received an unexpected special gift. My friend Joe Reis hosted a Lunch and Learn session on his Practical Data Modeling Discord community, during which Hannes Mühleisen introduced us to Quack: DuckDB’s new client-server protocol.
Ever since that session, I’ve been thinking of (1) how to incorporate Quack in my SQL benchmark lab, and (2) the experiments to run to provide independent measurements. I set out to answer this particular question: what does the network layer being introduced actually cost? While Quack is in beta, I happen to own the right instrument to make these independent measurements.
And this is the duck story I’ll tell today. 💚
A year ago, I started building an SQL benchmark framework, originally designed to support a series of Substack articles I started writing. What I have learned during the initial iteration was how far the query optimizers have come. Some of my hypotheses were not supported by the results of the experiments I ran. What I came to suspect was that the optimizers were quietly handling the antipatterns I meant to demonstrate.
Months later, during another Lunch and Learn delivered by Nick Schrock , I got inspired to learn Dagster. During the days I spent on Dagster University, I realized that this orchestrator was providing me the right tool to transform the original framework idea into what eventually became an SQL benchmarking laboratory.
This lab has PostgreSQL and DuckDB as its core engines, with Actian Vector™ and TypeDB as expansion proofs. Quack was added as the fifth engine. The lab differs from the usual “dataset + queries + stopwatch” benchmark in one structural way: it produces reproducible experiments, not numbers.
What that means concretely — every result is a published capsule: a folder you can open. The IDs referenced in the article — b8e2bfaf, 25b0e134, b198363e, 902d1277 — are real directories in the repo (the lab is also installable: pip install sqlbenchdag). Two things hold for all of them: (1) how each number was produced, and (2) what each one lets you verify without trusting me.
Cold: before each individual query — not each batch — the OS page cache is forcibly evicted and the engine is reset (containers restarted, servers cold-started, connections reopened). Five replications per data point, on an otherwise idle machine.
Seeded: identical data regenerates from the config — including the TPC-H validation, whose data DuckDB’s official tpch extension generates deterministically from a fixed scale factor (CALL dbgen).
Orchestrated by Dagster: an experiment declares a matrix, and its cartesian product becomes the Dagster partitions; each engine runs as its own asset across them — so one run is (matrix cells) × (engines) independent, cold-cached measurements. That partitioning is what makes parallel dispatch, granular retries, and per-partition lineage possible at all.
Whole, not averaged — the capsule stores the raw per-replication timings (not just means) and the bench conditions (engine versions, OS, hardware), so the spread is yours to inspect, not mine to summarize.
(2) What every result lets you verify — without trusting me
Each capsule carries four independent guarantees, every one checkable with a single command:
Reproducible [Automatic] — a content-addressed ID: the first 8 hex of a SHA-256 of the config, the SQL, and the lab code that ran it. Because the ID includes the code, each capsule records the exact build that produced it (sqlbenchdag@<sha>), in its metadata. Re-running the config on that build produces the same ID; a different revision produces a different ID, and the comparison is refused by construction — the code is part of the question.
Intact [Automatic] — an integrity seal (SHA-256 over every file in the capsule except the seal and its sidecars1). A silent edit to any result breaks it.
Timestamped [Optional] — an OpenTimestamps proof anchoring that seal to the Bitcoin blockchain. No one — not me, not GitHub — can backdate or quietly alter a published result.
Attributed [Optional] — the published set is a signed git tag, so you can verify who produced it.
How these guarantees are enforced — the semantic hashing, the isolation harness, the capsule contract — is beyond the scope of this article. For this one, you only need to know what the instrument promises: every claim below can be checked, re-run, or proven wrong.2
Moreover, the lab analyzes itself: when an experiment is scaled across row counts, each engine’s power-law exponent is fit from the sealed raw timings and stored alongside them (scaling.json). Not every experiment needs one — a thread sweep or a single-scale run has no exponent to fit — and sometimes, even when scaling is the question, there is no single exponent at all, as a separate article on the selectivity experiments will show. Where there is one, those exponents are the spine of what follows.
Quack (beta in DuckDB 1.5.3, stable planned for v2.0.0 in Fall 2026) turns a DuckDB instance into a server other DuckDB instances reach over HTTP. It offers two ways to execute work remotely:
Attach mode: ATTACH ‘quack:host’ AS remote — remote tables behave like local ones; your client plans the queries.
Pushdown mode: remote.query(’SELECT …’) — the SQL text ships to the server and executes entirely there.
As you’ll read next, that distinction is the entire story, told in four acts, starting with Act I.
This was the first experiment, designed to measure Quack's two client-server modes — attach and pushdown — against in-process DuckDB as the floor.
Hypothesis: protocol overhead dominates at small result sets and amortizes as scan volume grows.
Identical SQL (the Quack engine reuses the DuckDB scenario directory).
Identical cold-cache methodology – the only variable is the transport: in-process function calls vs HTTP with DuckDB’s native serialization.
This is what an experiment config in this lab is:
meta: name: "Quack Execution Modes: Attach vs Pushdown vs In-Process" description: > Mechanism experiment for the Quack protocol (DuckDB 1.5.3 beta). An earlier exploratory run showed attach-mode protocol overhead GROWS with scan size (2.7x at 100K rows, 4.9x at 1M) even though the analytical_wall result set is a constant 12 rows. Hypothesis: in attach mode (ATTACH + USE) the client plans the query and streams table data over HTTP; remote.query() pushdown executes fully server-side, so its overhead curve should stay flat at the constant cost of shipping 12 result rows. Three variants, identical data, identical SQL, identical cold-cache discipline: duckdb (in-process floor), quack (attach), quack_pushdown.
dataset: source: sql_benchmarks.plugins.data_sources.declarative_gen tables: analytical_data: rows: rows columns:
- name: id provider: sequence primary_key: true
- name: region provider: choice options: ["North", "South", "East", "West"]
- name: category provider: choice options: ["A", "B", "C"]
- name: price provider: random_float min_value: 10.0 max_value: 500.0
- name: quantity provider: random_int min_value: 1 max_value: 100
- name: discount provider: random_float min_value: 0.0 max_value: 0.30
definitions: rows: tiny: 1_000 small: 100_000 medium: 1_000_000 large: 10_000_000
execution: test_suite: analytical_wall engines:
- duckdb
- quack
- quack_pushdown replication: 5 matrix: rows:
- tiny
- small
- medium
- large
The experiment was run on the following query, marked in the experiment file as analytical_wall.
-- analytical_wall.sql (DuckDB) SELECT region, category, COUNT(*) as transaction_count, ROUND(SUM(price * quantity * (1 - discount)), 2) as net_revenue, ROUND(AVG(price * quantity * (1 - discount)), 2) as avg_transaction_value, ROUND(SUM(CASE WHEN discount > 0.15 THEN price * quantity ELSE 0 END) / SUM(price * quantity + 0.00001) * 100, 2) as high_discount_impact_pct FROM {{ analytical_data_table }} GROUP BY region, category ORDER BY net_revenue DESC;
This query is deliberately unfiltered.
Two things move data in a remote query — (1) the rows the engine must read, and (2) the rows it returns. To attribute the protocol’s cost to reading, I had to hold the returned side constant while the read side grew. A full-table aggregation does precisely that: it collapses any number of input rows into the same twelve-row result, so output transport is fixed across every scale, while a missing WHERE clause forces the engine to read the entire table. Full scan in, constant result out: any growth in cost can come from only one place — moving the input across the wire. A filter would have reintroduced exactly the confounder I was trying to remove.
five cold replications per point (raw per-rep timings retained),
three modes — in-process DuckDB, Quack attach, Quack pushdown.
Table 1 summarizes the durations the experiment revealed3. What was observed is that the protocol overhead GROWS with scan size. In particular:
Quack attach: 2.6x at 100K; 4.7x at 1M; 9.5× @10M, and
Pushdown’s overhead is bounded — it starts high (fixed protocol toll on a tiny query), amortizes to ~1.8x by 1M rows, and settles near 2x, and
Attach’s overhead is unbounded — it grows with every row scanned.
Table 1: Experiment b8e2bfaf results.
From the experiment results (they are sealed timings), I fit a power law:
where α is the scaling class. On a log–log axis, a power law is a straight line whose slope is α.
Figure 1 below shows how the protocol overhead scales. In particular:
Quack attach climbs steeply: α≈0.55, ~√N, while
Quack pushdown tracks in-process at a near-constant: α≈0.32 vs 0.35. This indicates that both in-process and pushdown belong in the same scaling class.
This large difference between Quack attach and Quack pushdown comes from the fact that pushdown ships only the 12-row result, so its cost is fixed protocol overhead (~2x). Attach, on the other hand, streams the whole table over HTTP, so its cost grows with rows scanned.
Figure 1: Act I scaling — mean duration vs row count (log-log), in-process DuckDB vs Quack attach vs Quack pushdown.
The results of this experiment falsified the amortization hypothesis — attach overhead grows with scan4, not shrinks, while pushdown stays flat at ~2x. The split is “streams rows” vs “ships results.”
At this point, you may wonder, as I did, why is it that Quack pushdown has a ~2x factor, and not 1x? The next experiment addresses this question in Act II.
As I’m writing this, I realized an unfortunate collision: there are two twelves mentioned in this section. They are unrelated twelves:
(1) the query returns 12 rows — 4 regions × 3 categories, the GROUP BY result it holds constant, while
(2) the experiment produces 12 measurements — 4 row-scales × 3 engines, the rows in <ID>.csv.
The same number, same 4x3 shape, but nothing to do with each other: one is data the query groups, the other is cells the lab measures.
The execution-modes experiment (described in Act I) left an open question I wanted to investigate further: pushdown-mode Quack runs the identical query on the identical file ~2x slower than in-process DuckDB, and the residual scales with execution work. This means that this residual cannot be explained by either round-trip cost or result serialization (12-row result set). What does explain it, though?
Hypothesis: the Quack server executes queries with fewer effective threads than an in-process connection.
To test this hypothesis I swept duckdb.threads in-process on the same partitions where pushdown runs with server defaults. If pushdown timings match in-process at a reduced thread count, the residual is explained by parallelism, not protocol.
five cold replications per point (raw per-rep timings retained),
two modes — in-process Du
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。