Skip to content

Read DataFusion VortexTable scans through split and open - #11666

Draft
gatesn wants to merge 32 commits into
ngates/scan-native-poolfrom
ngates/datafusion-scan-pool
Draft

gatesn wants to merge 32 commits into
ngates/scan-native-poolfrom
ngates/datafusion-scan-pool

Conversation

@gatesn

@gatesn gatesn commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #11663.

DataFusion is a partitioned engine: it plans a fixed number of output partitions and drives each as its own stream. VortexTable maps a Vortex scan onto those partitions through the Scan API's DataSourceScan::open and DataSourceScan::split. Each execution of the DataSourceExec scans once, through DataFusion's sibling state (DataSource::create_sibling_state), and the first output partition polled runs scan(request). reset_state starts a fresh execution.

How the partitions divide the work follows DataFusion's own file scan. Its FileGroupPartitioner cuts files into fixed groups at planning, and FileScanConfig::create_sibling_state pools those groups into shared work unless order or partitioning must be preserved, in which case each sorted group declares the ordering and a SortPreservingMergeExec above merges them.

  • Unordered: the execution opens the scan once, and every output partition polls a clone of its ScanStream. The clones draw from one queue of work, so a partition that finishes early keeps reading, and per-partition row counts are unknown until the scan runs.
  • Ordered: for a declared ordering, or when DataFusion asks each partition to keep its rows in order (with_preserve_order), the execution calls split(N) once for the N output partitions, and output partition i reads fixed piece i in order. The pieces are contiguous, so every partition keeps the ordering and DataFusion merges them where it needs one global order. If the scan yields fewer pieces than partitions, the remaining partitions return no rows.
  • Fetch: a fetch bounds each output partition, and DataFusion's limit (or the merge's fetch) above the scan trims the total. DataFusion pushes a fetch after it has fixed the partitioning, so an ordered fetch reads up to the fetch from each piece.

Each partition's arrays become Arrow record batches through a new ArrowArrayStreamExt::to_arrow in vortex-arrow. It converts arrays as CPU tasks on the session runtime, with bounded concurrency, and keeps their order.

Per-partition statistics are not reported: shared work has no per-partition row count, and partition_statistics runs at plan time, before the execution scans.

VORTEX_SCAN_POOL=0 (or VortexDataSourceBuilder::with_split_scan(false)) instead has the output partitions take partitions from one shared partition stream, which keeps an ordered or limited scan in one output partition, for A/B comparison; the default is 1. The switch and that path go away when the Scan API drops partitions().

In every mode an output partition holds work no other partition reads: what its clone has taken, its piece, or the partitions it has taken. Dropping an output partition before it finishes therefore cancels the execution, and the other partitions report an error rather than finish without those rows.

Measured at c55c047 against develop 7591044: DataFusion VortexTable with the V1 scan engine, geomean of per-query time ratios (lower is better), from four rotations of five iterations in which every configuration runs twice, with a load gate before each query. "Split on vs off" compares VORTEX_SCAN_POOL=1 with =0; "develop vs develop" is the noise between two runs of develop:

suite this PR vs develop split on vs off develop vs develop
TPC-H SF1 0.998 1.002 1.006
ClickBench 0.987 1.000 0.983

No query is more than 5% slower in every rotation. The largest gaps against develop are ClickBench q19 (8.3 → 8.8 ms, 1.08) and q36 (1.06), and TPC-H q14 (1.02).

In the Scan API stack this PR maps a scan onto DataFusion's fixed output partitions, the partitioned counterpart to DuckDB's use of open in #11664. The roadmap is tracked in #9062.

🤖 Generated with Claude Code

@gatesn gatesn added the changelog/performance A performance improvement label Nov 19, 2026
@gatesn gatesn mentioned this pull request Nov 19, 2026
16 of 46 tasks
gatesn and others added 2 commits November 19, 2026 18:30
Layout scans now split on their layout splits and open into a pool
whose workers share those splits through a channel. Multi-file scans
queue an open for each deferred file; the worker that opens one sends
the file's splits back into the channel for any worker to take.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Decoding a split inline kept each worker's splits on its own thread.
Spawning them onto the session's runtime, as V1 scans do, lets idle
runtime threads decode them and starts their I/O at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn

gatesn commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Benchmarks for the v2 VortexTable path: datafusion-bench --formats vortex with VORTEX_USE_SCAN_API=1, release_debug with -C target-cpu=native, TPC-H SF1 and ClickBench (100 partitioned files), on an 18-core Apple Silicon Mac. Develop is d368834 and this PR is 4bc6703. Each suite ran 3 interleaved rounds of develop then this PR, 5 iterations per round. The tables show the median over all rounds, dropping each round's first iteration. Other builds were running on the machine the whole time (load average ~25–30), so treat differences under ~10% as noise.

vx-bench run does not exercise this code. It passes --scan-v1 false, which routes Vortex through the persistent FileFormat path, and the v2 VortexTable only registers with --scan-v1 true plus VORTEX_USE_SCAN_API=1. My first vx-bench run of both builds was therefore an A/A run of identical code. It showed 1.02× (TPC-H) and 1.15× (ClickBench) geomean, with single tiny queries off by up to 3.8×, which gives a measure of the noise.

Headline: TPC-H geomean 0.43× (develop's shared partition stream was badly serialized: q01 went from 160 ms to 19 ms). ClickBench geomean 0.94×, total 1.04×.

Regressions over 5%, rechecked. I reran the flagged ClickBench queries over 4 rounds with alternating order, 8 iterations each. q04, q05, q07 and q08 came out at 0.84–0.99×, so those were noise. q21, q22 and q23 are real: 1.11×, 1.20× and 1.32× by median, 1.18–1.76× by min. On the base branch (#11663, same setup, 3 rounds), base matches develop, so this PR is the cause:

query develop (ms) base #11663 (ms) this PR (ms) PR / base
q01 6.0 5.9 6.5 1.10
q08 185.9 180.7 186.3 1.03
q21 159.0 150.6 168.0 1.12 ⚠️
q22 185.7 173.3 211.4 1.22 ⚠️
q23 176.6 157.3 218.1 1.39 ⚠️

Cause (q23, Samply over 20 iterations). This PR uses about 70% more user CPU than base for the same query (/usr/bin/time: 44 s vs 26 s over 15 iterations), so the cost is extra work, not just lost parallelism. A third of the CPU samples spin in dashmap::lock::RawRwLock::lock_exclusive_slow, called from SharedSegmentSource::request (vortex-io/src/segments/shared.rs) via FlatReader::array_future under split_exec and pool::Shared::admit.

request takes an exclusive entry() lock on every segment request. With the pool, all 18 workers share one file's splits, and so one SharedSegmentSource, and SELECT * requests ~100 segments per split. On develop and base, output partitions spread across the 100 files, so that lock was never contended. The CPU-heavy LIKE '%google%' queries suffer most, and the machine load likely amplifies it (a preempted lock holder leaves the others spinning).

The likely fix sits below this PR: either a read-locked fast path in SharedSegmentSource::request, or handing out splits so that workers tend to stay on different files.

The scan's elapsed_compute now includes decoding, since workers decode inside the output partition's poll rather than on spawned tasks.

Full interleaved results (median ms)

tpch

query develop (ms) this PR (ms) ratio
q01 160.5 18.9 0.12 ✅
q02 19.0 10.1 0.53 ✅
q03 28.6 9.2 0.32 ✅
q04 21.9 7.4 0.34 ✅
q05 58.9 18.1 0.31 ✅
q06 5.0 3.5 0.70 ✅
q07 108.0 52.8 0.49 ✅
q08 51.2 13.6 0.27 ✅
q09 94.0 26.6 0.28 ✅
q10 36.7 15.7 0.43 ✅
q11 11.2 6.2 0.56 ✅
q12 17.1 8.6 0.50 ✅
q13 16.0 12.1 0.76 ✅
q14 8.2 5.2 0.63 ✅
q15 16.8 8.2 0.49 ✅
q16 12.9 8.8 0.68 ✅
q17 106.5 28.7 0.27 ✅
q18 151.8 52.2 0.34 ✅
q19 9.6 6.8 0.71 ✅
q20 20.7 10.5 0.51 ✅
q21 118.5 46.5 0.39 ✅
q22 11.8 6.6 0.56 ✅
geomean 0.427
sum 1085 376 0.347

clickbench

query develop (ms) this PR (ms) ratio
q00 3.3 3.3 1.01
q01 6.4 6.1 0.95 ✅
q02 14.8 13.7 0.93 ✅
q03 14.5 14.0 0.97
q04 136.1 150.2 1.10 ⚠️
q05 157.2 187.6 1.19 ⚠️
q06 3.8 3.8 1.01
q07 8.5 11.0 1.29 ⚠️
q08 201.4 250.3 1.24 ⚠️
q09 277.7 286.5 1.03
q10 32.4 35.3 1.09 ⚠️
q11 41.8 42.2 1.01
q12 145.1 151.8 1.05
q13 251.7 240.4 0.96
q14 174.6 169.7 0.97
q15 207.6 211.4 1.02
q16 419.7 439.3 1.05
q17 432.3 417.5 0.97
q18 771.4 802.9 1.04
q19 12.3 11.8 0.96
q20 115.0 121.0 1.05 ⚠️
q21 187.9 214.7 1.14 ⚠️
q22 224.7 272.4 1.21 ⚠️
q23 186.2 264.3 1.42 ⚠️
q24 12.7 13.8 1.09 ⚠️
q25 36.1 33.5 0.93 ✅
q26 23.0 22.0 0.96
q27 202.9 197.1 0.97
q28 1521.9 1494.6 0.98
q29 21.5 22.9 1.06 ⚠️
q30 119.9 128.7 1.07 ⚠️
q31 119.9 124.6 1.04
q32 581.0 598.2 1.03
q33 864.9 926.9 1.07 ⚠️
q34 875.2 945.9 1.08 ⚠️
q35 181.3 192.0 1.06 ⚠️
q36 43.9 11.8 0.27 ✅
q37 23.9 9.7 0.41 ✅
q38 15.7 6.8 0.43 ✅
q39 69.9 21.1 0.30 ✅
q40 5.7 5.2 0.90 ✅
q41 5.6 5.3 0.94 ✅
q42 6.0 5.2 0.86 ✅
geomean 0.938
sum 8757 9086 1.038

gatesn and others added 2 commits November 19, 2026 18:33
Each DataSourceExec execution now opens its Vortex scan into a pool whose
workers back the output partitions, or splits it into one piece per
partition when order matters or work stealing is disabled, replacing the
partition stream shared under a mutex. A fetch now bounds each of several
output partitions rather than forcing one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
The layout file in the split-mode test had a single layout split, so
splitting it returned one piece. The test scan now splits natively.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
gatesn force-pushed the ngates/datafusion-scan-pool branch from 4bc6703 to d135991 Compare November 19, 2026 23:34
gatesn and others added 3 commits November 19, 2026 18:37
Adds ArrowArrayStreamExt::to_arrow, which converts each array of a
stream to a RecordBatch as a CPU task on the session's runtime with
bounded, order-preserving concurrency. The v2 DataFusion source now
calls it on each worker instead of spawning conversions itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Benchmarking VortexTable needed both --scan-v1 true and
VORTEX_USE_SCAN_API=1, so it could not be compared with the listing table's
V2 scan in one run. --scan-api true registers Vortex tables through the Scan
API whatever --scan-v1 says; the environment variable keeps working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
The v2 DataFusion source keeps develop's execution, output partitions
sharing the scan's partition stream with its single-partition fetch,
selectable at runtime so one binary can compare it with split and open.
VORTEX_SCAN_POOL defaults to split and open; the partition stream goes
away with the partition API.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
gatesn and others added 7 commits November 19, 2026 19:19
Takes the base's version of every file outside vortex-datafusion,
vortex-arrow and the datafusion-bench flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
gatesn added a commit that referenced this pull request Nov 20, 2026
Replace the `VORTEX_USE_SCAN_API` environment variable in `datafusion-bench`
with a `--scan-api` flag, defined as #11666 defines it, and have the
orchestrator pass it for Scan API targets. PR benchmarks compare against the
benchmarks database rather than a base build, so the flag only needs to exist
on develop once this lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
gatesn added a commit that referenced this pull request Nov 20, 2026
Replace the `VORTEX_USE_SCAN_API` environment variable in `datafusion-bench`
with a `--scan-api` flag, defined as #11666 defines it, and have the
orchestrator pass it for Scan API targets. PR benchmarks compare against the
benchmarks database rather than a base build, so the flag only needs to exist
on develop once this lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
gatesn and others added 4 commits November 20, 2026 12:54
The bench's VORTEX_USE_SCAN_API fallback is dropped: develop's #11685 removed it, so only
--scan-api reaches VortexTable now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Split and open are now required on DataSourceScan, so the test scan keeps its own split and
gains an open over PartitionPool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn changed the title Read DataFusion v2 scans through split and open Read DataFusion VortexTable scans through split and open Nov 20, 2026
@gatesn
gatesn added this pull request to stack #11706 November 20, 2026 18:52
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
gatesn and others added 4 commits November 20, 2026 14:02
DataFusion runs output partitions independently, so output partition i of n now splits the
scan into at most n pieces and reads piece i with one worker. Partitions share no state, which
keeps them correct when they run in separate processes. VORTEX_SCAN_POOL=0 keeps the shared
partition stream for comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
An unordered source may have the pieces of one split share a queue of work, so the pieces must
come from a single split call. The first output partition polled scans and splits through the
execution's sibling state, and output partition i reads piece i.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
gatesn and others added 9 commits November 20, 2026 14:39
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn added the action/bench-scan Run scan benchmarks label Nov 20, 2026
@github-actions github-actions Bot removed the action/bench-scan Run scan benchmarks label Nov 20, 2026
@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done c55c047 clickbench-nvme-datafusion-datasource-v1 3 Explore Profiling Data
🟢 Done c55c047 clickbench-nvme-duckdb-file-v1 1 Explore Profiling Data
🟢 Done c55c047 clickbench-nvme-datafusion-file-v1 1 Explore Profiling Data
🟢 Done c55c047 clickbench-nvme 1 Explore Profiling Data
🟢 Done c55c047 tpch-nvme-datafusion-file-v1 1 Explore Profiling Data
🟢 Done c55c047 tpch-nvme-datafusion-datasource-v1 1 Explore Profiling Data
🟢 Done c55c047 tpch-nvme-duckdb-file-v1 1 Explore Profiling Data
🟢 Done c55c047 tpch-nvme 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟡 In Progress c55c047 tpch-nvme-datafusion-datasource-v1 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on NVME 📖

Commits: PR c55c0471 vs base 75910442
Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.4%
Engines: DataFusion No clear signal (+0.2%, low confidence) · DuckDB No clear signal (+0.6%, low confidence)
Vortex (geomean): hot 1.004x ➖ · cold 1.002x ➖
Parquet (geomean): hot 1.001x ➖ · cold 0.998x ➖
Shifts: Parquet (control) +0.1% · Median polish -0.0%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 1.003x ➖ · 22 queries × 7 targets (0↑ 1↓)

Each cell is develop 75910442 → this PR c55c0471 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex [file, v1] datafusion:vortex [datasource, v1] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex [file, v1] duckdb:parquet
geomean 20.1 → 20.2 ms (1.01) 21.2 → 21.3 ms (1.00) 63.9 → 63.9 ms (1.00) 77.7 → 77.8 ms (1.00) 15.5 → 15.5 ms (1.00) 15.5 → 15.6 ms (1.01) 55.8 → 55.8 ms (1.00)
tpch_q01 15.6 → 16.0 ms (1.03) 16.2 → 16.2 ms (1.00) 253 → 254 ms (1.00) 71.2 → 71.2 ms (1.00) 9.53 → 8.96 ms (0.94) 8.97 → 10.1 ms (1.13) 🔴 47.5 → 47.3 ms (1.00)
tpch_q02 17.6 → 17.7 ms (1.01) 17.9 → 17.9 ms (1.00) 37.0 → 37.5 ms (1.01) 76.2 → 76.7 ms (1.01) 15.8 → 15.2 ms (0.96) 14.8 → 14.8 ms (1.00) 48.2 → 48.1 ms (1.00)
tpch_q03 19.6 → 19.1 ms (0.97) 19.5 → 19.3 ms (0.99) 52.3 → 52.0 ms (1.00) 81.9 → 82.1 ms (1.00) 18.6 → 18.7 ms (1.01) 18.9 → 18.7 ms (0.99) 60.3 → 60.1 ms (1.00)
tpch_q04 10.2 → 10.5 ms (1.03) 9.80 → 9.88 ms (1.01) 56.7 → 56.3 ms (0.99) 39.5 → 39.5 ms (1.00) 12.0 → 11.9 ms (0.99) 11.9 → 11.8 ms (0.98) 34.6 → 34.7 ms (1.00)
tpch_q05 37.8 → 38.0 ms (1.00) 37.8 → 37.3 ms (0.99) 161 → 162 ms (1.01) 110 → 111 ms (1.00) 20.2 → 20.2 ms (1.00) 20.0 → 20.1 ms (1.00) 67.0 → 67.0 ms (1.00)
tpch_q06 4.57 → 4.55 ms (1.00) 6.11 → 6.24 ms (1.02) 7.33 → 7.27 ms (0.99) 31.4 → 31.4 ms (1.00) 4.70 → 4.98 ms (1.06) 4.78 → 5.02 ms (1.05) 21.9 → 21.5 ms (0.98)
tpch_q07 52.7 → 52.6 ms (1.00) 51.8 → 52.2 ms (1.01) 194 → 191 ms (0.98) 113 → 113 ms (1.00) 24.3 → 23.9 ms (0.98) 23.9 → 24.1 ms (1.01) 63.9 → 63.7 ms (1.00)
tpch_q08 51.7 → 51.4 ms (0.99) 51.0 → 51.8 ms (1.02) 89.4 → 88.3 ms (0.99) 118 → 117 ms (1.00) 21.4 → 20.8 ms (0.97) 20.7 → 20.8 ms (1.01) 75.6 → 75.7 ms (1.00)
tpch_q09 47.2 → 49.2 ms (1.04) 48.9 → 49.9 ms (1.02) 191 → 192 ms (1.00) 137 → 138 ms (1.01) 22.9 → 23.2 ms (1.01) 21.4 → 21.9 ms (1.02) 107 → 108 ms (1.00)
tpch_q10 18.4 → 18.2 ms (0.99) 18.8 → 18.5 ms (0.99) 85.6 → 85.9 ms (1.00) 104 → 103 ms (1.00) 27.8 → 27.5 ms (0.99) 27.7 → 27.9 ms (1.01) 105 → 106 ms (1.01)
tpch_q11 14.7 → 14.6 ms (0.99) 15.1 → 15.0 ms (1.00) 28.6 → 28.2 ms (0.99) 61.9 → 61.9 ms (1.00) 8.00 → 7.92 ms (0.99) 7.71 → 7.82 ms (1.02) 29.6 → 29.3 ms (0.99)
tpch_q12 18.2 → 18.6 ms (1.02) 22.3 → 22.3 ms (1.00) 47.3 → 47.1 ms (1.00) 64.0 → 63.5 ms (0.99) 14.3 → 14.4 ms (1.01) 15.7 → 15.4 ms (0.98) 37.1 → 37.1 ms (1.00)
tpch_q13 19.6 → 19.7 ms (1.00) 19.3 → 19.3 ms (1.00) 34.9 → 35.5 ms (1.02) 143 → 143 ms (1.00) 19.3 → 20.6 ms (1.07) 19.7 → 20.1 ms (1.02) 175 → 175 ms (1.00)
tpch_q14 14.2 → 14.4 ms (1.01) 14.5 → 14.9 ms (1.03) 23.0 → 23.1 ms (1.01) 47.8 → 47.6 ms (1.00) 11.2 → 11.1 ms (1.00) 11.1 → 11.2 ms (1.00) 42.3 → 42.0 ms (0.99)
tpch_q15 18.9 → 18.4 ms (0.97) 18.8 → 18.4 ms (0.98) 37.1 → 36.7 ms (0.99) 73.8 → 75.1 ms (1.02) 8.18 → 8.34 ms (1.02) 8.50 → 8.40 ms (0.99) 32.7 → 32.6 ms (1.00)
tpch_q16 18.4 → 18.4 ms (1.00) 18.4 → 18.3 ms (1.00) 31.4 → 31.1 ms (0.99) 43.4 → 42.9 ms (0.99) 16.7 → 16.7 ms (1.00) 16.7 → 16.7 ms (1.00) 47.0 → 47.3 ms (1.01)
tpch_q17 29.4 → 29.7 ms (1.01) 28.7 → 28.7 ms (1.00) 214 → 218 ms (1.02) 123 → 124 ms (1.01) 13.0 → 12.8 ms (0.99) 13.6 → 13.6 ms (1.00) 47.9 → 47.9 ms (1.00)
tpch_q18 36.5 → 36.9 ms (1.01) 36.2 → 36.2 ms (1.00) 280 → 278 ms (0.99) 121 → 121 ms (1.00) 23.9 → 24.7 ms (1.03) 25.6 → 25.6 ms (1.00) 73.8 → 73.7 ms (1.00)
tpch_q19 9.76 → 9.97 ms (1.02) 15.0 → 15.1 ms (1.01) 18.3 → 18.7 ms (1.02) 57.6 → 57.6 ms (1.00) 11.1 → 11.7 ms (1.05) 12.0 → 11.8 ms (0.99) 47.5 → 48.5 ms (1.02)
tpch_q20 20.5 → 20.9 ms (1.02) 20.9 → 20.9 ms (1.00) 51.9 → 51.4 ms (0.99) 78.4 → 79.3 ms (1.01) 14.5 → 14.7 ms (1.01) 14.9 → 15.0 ms (1.01) 63.4 → 63.4 ms (1.00)
tpch_q21 34.6 → 35.7 ms (1.03) 37.8 → 38.3 ms (1.02) 227 → 226 ms (0.99) 114 → 113 ms (1.00) 48.9 → 48.2 ms (0.99) 48.1 → 47.1 ms (0.98) 107 → 107 ms (1.00)
tpch_q22 10.7 → 10.8 ms (1.01) 12.2 → 12.4 ms (1.01) 35.0 → 35.1 ms (1.00) 51.4 → 51.9 ms (1.01) 16.6 → 17.0 ms (1.02) 16.7 → 16.8 ms (1.01) 50.4 → 50.5 ms (1.00)
Cold runs (first iteration): geomean 1.001x ➖ · 22 queries × 7 targets (0↑ 1↓)
query datafusion:vortex [file, v2] datafusion:vortex [file, v1] datafusion:vortex [datasource, v1] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex [file, v1] duckdb:parquet
geomean 46.6 → 46.6 ms (1.00) 46.1 → 47.1 ms (1.02) 91.2 → 91.3 ms (1.00) 100 → 99.8 ms (1.00) 29.0 → 28.6 ms (0.99) 29.0 → 29.0 ms (1.00) 64.4 → 64.4 ms (1.00)
tpch_q01 43.5 → 44.5 ms (1.02) 42.2 → 44.9 ms (1.06) 273 → 272 ms (1.00) 89.0 → 87.2 ms (0.98) 21.1 → 19.5 ms (0.92) 20.3 → 19.2 ms (0.95) 54.0 → 53.7 ms (0.99)
tpch_q02 37.8 → 37.9 ms (1.00) 34.3 → 35.6 ms (1.04) 54.2 → 52.0 ms (0.96) 92.6 → 93.9 ms (1.01) 24.7 → 24.6 ms (1.00) 24.2 → 25.5 ms (1.06) 55.4 → 54.8 ms (0.99)
tpch_q03 46.9 → 46.6 ms (0.99) 46.4 → 46.5 ms (1.00) 78.2 → 80.3 ms (1.03) 104 → 103 ms (0.99) 35.5 → 35.7 ms (1.01) 34.1 → 35.7 ms (1.05) 68.6 → 66.8 ms (0.97)
tpch_q04 34.1 → 33.8 ms (0.99) 32.0 → 31.1 ms (0.97) 74.1 → 73.5 ms (0.99) 58.9 → 57.8 ms (0.98) 25.8 → 25.4 ms (0.99) 23.7 → 24.2 ms (1.02) 42.8 → 42.6 ms (0.99)
tpch_q05 70.3 → 65.4 ms (0.93) 63.5 → 65.8 ms (1.04) 185 → 186 ms (1.00) 136 → 137 ms (1.01) 37.7 → 37.2 ms (0.99) 36.7 → 36.3 ms (0.99) 77.1 → 76.3 ms (0.99)
tpch_q06 24.5 → 27.1 ms (1.11) 🔴 27.4 → 27.4 ms (1.00) 23.0 → 24.6 ms (1.07) 48.2 → 47.9 ms (0.99) 14.3 → 14.0 ms (0.98) 15.0 → 15.2 ms (1.01) 26.9 → 27.3 ms (1.02)
tpch_q07 78.5 → 78.6 ms (1.00) 76.5 → 79.8 ms (1.04) 220 → 224 ms (1.02) 141 → 141 ms (1.00) 42.0 → 40.0 ms (0.95) 40.3 → 39.2 ms (0.97) 72.9 → 73.4 ms (1.01)
tpch_q08 83.5 → 84.1 ms (1.01) 78.6 → 84.9 ms (1.08) 113 → 112 ms (1.00) 143 → 143 ms (1.00) 36.8 → 38.3 ms (1.04) 38.6 → 38.8 ms (1.00) 86.7 → 86.5 ms (1.00)
tpch_q09 76.6 → 79.5 ms (1.04) 79.9 → 79.1 ms (0.99) 220 → 216 ms (0.98) 168 → 165 ms (0.98) 37.1 → 36.7 ms (0.99) 39.5 → 40.0 ms (1.01) 118 → 119 ms (1.01)
tpch_q10 49.3 → 47.4 ms (0.96) 48.3 → 47.6 ms (0.98) 117 → 116 ms (0.99) 128 → 125 ms (0.97) 45.2 → 44.0 ms (0.97) 43.0 → 43.1 ms (1.00) 117 → 116 ms (0.99)
tpch_q11 34.7 → 34.3 ms (0.99) 32.0 → 32.7 ms (1.02) 47.6 → 46.6 ms (0.98) 81.3 → 83.0 ms (1.02) 17.2 → 17.4 ms (1.01) 17.2 → 16.9 ms (0.98) 34.9 → 34.6 ms (0.99)
tpch_q12 43.4 → 40.6 ms (0.93) 43.7 → 45.9 ms (1.05) 67.0 → 68.0 ms (1.01) 83.3 → 84.8 ms (1.02) 25.9 → 25.6 ms (0.99) 29.2 → 28.8 ms (0.99) 44.4 → 44.1 ms (0.99)
tpch_q13 37.9 → 37.1 ms (0.98) 41.6 → 42.2 ms (1.02) 51.1 → 51.1 ms (1.00) 163 → 162 ms (1.00) 31.7 → 31.5 ms (0.99) 31.7 → 32.2 ms (1.02) 182 → 182 ms (1.00)
tpch_q14 38.7 → 37.5 ms (0.97) 37.7 → 39.8 ms (1.06) 43.5 → 45.2 ms (1.04) 68.5 → 69.0 ms (1.01) 24.3 → 23.3 ms (0.96) 24.2 → 24.1 ms (0.99) 51.0 → 50.6 ms (0.99)
tpch_q15 47.4 → 45.3 ms (0.95) 44.8 → 44.9 ms (1.00) 54.3 → 54.1 ms (1.00) 93.6 → 92.5 ms (0.99) 21.5 → 20.4 ms (0.95) 21.2 → 20.0 ms (0.95) 41.2 → 41.2 ms (1.00)
tpch_q16 39.3 → 39.3 ms (1.00) 38.5 → 37.9 ms (0.99) 50.4 → 50.3 ms (1.00) 61.6 → 62.4 ms (1.01) 23.4 → 23.5 ms (1.00) 23.3 → 23.5 ms (1.01) 52.2 → 52.3 ms (1.00)
tpch_q17 59.2 → 59.3 ms (1.00) 59.4 → 59.6 ms (1.00) 236 → 235 ms (1.00) 147 → 148 ms (1.00) 27.8 → 27.9 ms (1.00) 27.3 → 27.4 ms (1.01) 57.1 → 57.7 ms (1.01)
tpch_q18 58.3 → 59.9 ms (1.03) 58.3 → 58.1 ms (1.00) 309 → 311 ms (1.01) 143 → 142 ms (0.99) 34.6 → 34.3 ms (0.99) 34.8 → 36.0 ms (1.03) 80.0 → 79.6 ms (0.99)
tpch_q19 35.6 → 36.9 ms (1.04) 40.1 → 40.8 ms (1.02) 38.8 → 38.4 ms (0.99) 81.9 → 83.0 ms (1.01) 27.2 → 25.8 ms (0.95) 26.7 → 26.9 ms (1.01) 54.3 → 57.5 ms (1.06)
tpch_q20 44.8 → 48.3 ms (1.08) 45.3 → 47.1 ms (1.04) 74.3 → 72.1 ms (0.97) 102 → 101 ms (0.99) 29.9 → 29.8 ms (1.00) 30.4 → 30.2 ms (0.99) 74.2 → 73.3 ms (0.99)
tpch_q21 66.6 → 65.7 ms (0.99) 66.9 → 67.4 ms (1.01) 265 → 265 ms (1.00) 139 → 138 ms (0.99) 63.1 → 63.5 ms (1.01) 64.7 → 64.9 ms (1.00) 115 → 117 ms (1.02)
tpch_q22 30.5 → 30.9 ms (1.01) 29.1 → 30.9 ms (1.06) 52.6 → 52.0 ms (0.99) 68.1 → 65.8 ms (0.97) 26.3 → 26.5 ms (1.01) 27.2 → 26.9 ms (0.99) 58.6 → 58.3 ms (1.00)

No file size changes detected.

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Benchmarks: Clickbench on NVME 📖

Commits: PR c55c0471 vs base 75910442
Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.2%
Engines: DataFusion No clear signal (+0.7%, low confidence) · DuckDB No clear signal (-0.4%, low confidence)
Vortex (geomean): hot 1.000x ➖ · cold 1.022x ➖
Parquet (geomean): hot 0.998x ➖ · cold 1.022x ➖
Shifts: Parquet (control) -0.2% · Median polish -0.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.999x ➖ · 43 queries × 7 targets (3↑ 2↓)

Each cell is develop 75910442 → this PR c55c0471 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex [file, v1] datafusion:vortex [datasource, v1] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex [file, v1] duckdb:parquet
geomean 41.1 → 41.1 ms (1.00) 42.4 → 42.1 ms (0.99) 52.4 → 52.8 ms (1.01) 65.3 → 64.9 ms (0.99) 35.7 → 35.9 ms (1.01) 35.1 → 34.8 ms (0.99) 67.1 → 67.3 ms (1.00)
clickbench_q00 1.25 → 1.29 ms (1.04) 1.25 → 1.27 ms (1.02) 2.47 → 2.51 ms (1.02) 1.27 → 1.26 ms (0.99) 2.22 → 2.30 ms (1.04) 2.26 → 2.18 ms (0.96) 9.78 → 9.15 ms (0.94)
clickbench_q01 8.99 → 8.84 ms (0.98) 9.22 → 8.84 ms (0.96) 7.71 → 7.81 ms (1.01) 15.9 → 16.3 ms (1.02) 7.34 → 7.50 ms (1.02) 7.06 → 6.89 ms (0.98) 15.7 → 15.2 ms (0.97)
clickbench_q02 13.8 → 13.8 ms (1.00) 12.3 → 12.4 ms (1.00) 9.88 → 9.56 ms (0.97) 18.7 → 18.8 ms (1.01) 9.54 → 9.68 ms (1.01) 8.06 → 7.84 ms (0.97) 22.7 → 22.6 ms (0.99)
clickbench_q03 14.0 → 13.9 ms (0.99) 12.9 → 13.1 ms (1.02) 9.76 → 10.4 ms (1.07) 16.6 → 16.7 ms (1.01) 13.9 → 13.7 ms (0.98) 13.7 → 13.7 ms (1.00) 23.0 → 22.8 ms (0.99)
clickbench_q04 58.8 → 58.4 ms (0.99) 56.9 → 56.9 ms (1.00) 73.0 → 73.7 ms (1.01) 65.5 → 65.3 ms (1.00) 56.1 → 56.9 ms (1.01) 54.3 → 53.8 ms (0.99) 62.3 → 61.4 ms (0.99)
clickbench_q05 82.2 → 80.1 ms (0.97) 76.1 → 78.9 ms (1.04) 120 → 119 ms (0.99) 88.9 → 88.3 ms (0.99) 63.7 → 64.8 ms (1.02) 62.4 → 61.3 ms (0.98) 82.2 → 85.2 ms (1.04)
clickbench_q06 1.26 → 1.24 ms (0.98) 1.25 → 1.25 ms (1.00) 8.47 → 8.57 ms (1.01) 1.30 → 1.28 ms (0.98) 2.67 → 2.63 ms (0.98) 2.70 → 2.68 ms (0.99) 20.8 → 20.8 ms (1.00)
clickbench_q07 12.4 → 12.4 ms (1.00) 12.2 → 12.2 ms (1.00) 13.6 → 13.8 ms (1.02) 17.1 → 16.8 ms (0.98) 9.00 → 9.10 ms (1.01) 8.61 → 8.25 ms (0.96) 17.1 → 17.1 ms (1.00)
clickbench_q08 82.4 → 81.7 ms (0.99) 79.4 → 77.5 ms (0.98) 98.3 → 100 ms (1.02) 84.2 → 85.3 ms (1.01) 69.1 → 69.8 ms (1.01) 67.0 → 66.7 ms (0.99) 76.4 → 76.4 ms (1.00)
clickbench_q09 180 → 175 ms (0.97) 169 → 172 ms (1.02) 184 → 187 ms (1.02) 190 → 188 ms (0.99) 92.7 → 92.6 ms (1.00) 90.1 → 89.4 ms (0.99) 102 → 103 ms (1.01)
clickbench_q10 24.6 → 24.7 ms (1.00) 24.5 → 24.7 ms (1.01) 26.2 → 26.2 ms (1.00) 36.8 → 36.2 ms (0.98) 27.3 → 27.7 ms (1.02) 26.8 → 26.5 ms (0.99) 40.8 → 42.0 ms (1.03)
clickbench_q11 25.8 → 26.6 ms (1.03) 27.2 → 26.6 ms (0.98) 29.8 → 29.0 ms (0.97) 42.4 → 42.3 ms (1.00) 30.6 → 30.3 ms (0.99) 28.9 → 28.6 ms (0.99) 46.3 → 46.5 ms (1.01)
clickbench_q12 75.1 → 74.7 ms (0.99) 72.6 → 73.8 ms (1.02) 103 → 109 ms (1.06) 95.3 → 95.9 ms (1.01) 69.6 → 70.8 ms (1.02) 68.0 → 67.2 ms (0.99) 92.5 → 94.3 ms (1.02)
clickbench_q13 122 → 124 ms (1.02) 120 → 120 ms (1.00) 123 → 123 ms (1.00) 146 → 146 ms (1.00) 117 → 121 ms (1.04) 116 → 118 ms (1.02) 149 → 153 ms (1.03)
clickbench_q14 78.4 → 79.0 ms (1.01) 78.7 → 77.3 ms (0.98) 101 → 109 ms (1.08) 102 → 101 ms (0.99) 72.2 → 74.1 ms (1.03) 73.9 → 71.1 ms (0.96) 99.9 → 102 ms (1.02)
clickbench_q15 65.9 → 66.8 ms (1.01) 65.3 → 63.9 ms (0.98) 81.8 → 82.1 ms (1.00) 73.6 → 70.3 ms (0.96) 75.5 → 77.4 ms (1.02) 75.7 → 75.0 ms (0.99) 85.8 → 87.0 ms (1.01)
clickbench_q16 156 → 158 ms (1.01) 154 → 153 ms (0.99) 204 → 209 ms (1.02) 157 → 156 ms (0.99) 134 → 137 ms (1.02) 134 → 134 ms (1.00) 169 → 163 ms (0.96)
clickbench_q17 158 → 158 ms (1.00) 151 → 154 ms (1.02) 207 → 206 ms (1.00) 157 → 155 ms (0.99) 137 → 139 ms (1.01) 136 → 135 ms (0.99) 163 → 163 ms (1.00)
clickbench_q18 300 → 299 ms (0.99) 296 → 293 ms (0.99) 347 → 347 ms (1.00) 310 → 310 ms (1.00) 256 → 258 ms (1.01) 253 → 256 ms (1.01) 290 → 291 ms (1.00)
clickbench_q19 10.0 → 9.47 ms (0.95) 9.15 → 9.34 ms (1.02) 8.92 → 8.96 ms (1.00) 19.1 → 19.2 ms (1.01) 9.03 → 8.97 ms (0.99) 9.21 → 9.33 ms (1.01) 20.1 → 19.8 ms (0.98)
clickbench_q20 46.2 → 46.1 ms (1.00) 45.3 → 45.8 ms (1.01) 41.6 → 41.6 ms (1.00) 178 → 178 ms (1.00) 51.6 → 52.3 ms (1.01) 48.2 → 48.2 ms (1.00) 180 → 190 ms (1.06)
clickbench_q21 59.4 → 58.4 ms (0.98) 89.0 → 89.9 ms (1.01) 84.0 → 84.4 ms (1.00) 174 → 173 ms (0.99) 56.7 → 57.2 ms (1.01) 88.6 → 87.1 ms (0.98) 221 → 222 ms (1.00)
clickbench_q22 75.4 → 74.5 ms (0.99) 99.3 → 98.2 ms (0.99) 92.1 → 90.5 ms (0.98) 262 → 260 ms (0.99) 89.6 → 89.8 ms (1.00) 137 → 134 ms (0.98) 305 → 309 ms (1.01)
clickbench_q23 96.7 → 94.1 ms (0.97) 147 → 127 ms (0.87) 🟢 150 → 141 ms (0.94) 1.01 → 1.00 s (1.00) 70.8 → 62.6 ms (0.88) 🟢 34.4 → 38.7 ms (1.13) 🔴 173 → 172 ms (0.99)
clickbench_q24 8.78 → 9.45 ms (1.08) 15.4 → 15.0 ms (0.97) 13.1 → 14.1 ms (1.07) 29.9 → 29.8 ms (1.00) 16.8 → 17.0 ms (1.01) 13.6 → 13.2 ms (0.97) 30.2 → 30.8 ms (1.02)
clickbench_q25 19.8 → 20.9 ms (1.06) 19.7 → 19.5 ms (0.99) 16.6 → 17.2 ms (1.03) 38.2 → 36.1 ms (0.95) 19.1 → 19.4 ms (1.02) 19.0 → 18.8 ms (0.99) 42.8 → 42.1 ms (0.98)
clickbench_q26 14.1 → 14.8 ms (1.05) 16.0 → 16.2 ms (1.01) 17.6 → 18.1 ms (1.03) 30.8 → 30.5 ms (0.99) 13.8 → 14.1 ms (1.02) 11.7 → 11.6 ms (1.00) 30.5 → 30.7 ms (1.01)
clickbench_q27 101 → 104 ms (1.02) 99.3 → 97.2 ms (0.98) 100 → 103 ms (1.03) 193 → 191 ms (0.99) 51.2 → 50.7 ms (0.99) 50.4 → 50.0 ms (0.99) 185 → 198 ms (1.07)
clickbench_q28 559 → 559 ms (1.00) 564 → 568 ms (1.01) 840 → 828 ms (0.99) 622 → 616 ms (0.99) 515 → 513 ms (1.00) 505 → 502 ms (0.99) 956 → 968 ms (1.01)
clickbench_q29 26.5 → 26.4 ms (1.00) 25.2 → 25.0 ms (0.99) 21.8 → 21.6 ms (0.99) 29.4 → 29.3 ms (1.00) 17.0 → 17.3 ms (1.01) 16.4 → 16.6 ms (1.01) 24.3 → 24.0 ms (0.99)
clickbench_q30 60.3 → 58.9 ms (0.98) 60.1 → 56.6 ms (0.94) 75.2 → 81.9 ms (1.09) 91.1 → 89.4 ms (0.98) 61.7 → 61.0 ms (0.99) 58.6 → 58.4 ms (1.00) 106 → 107 ms (1.00)
clickbench_q31 69.4 → 69.5 ms (1.00) 69.5 → 70.2 ms (1.01) 77.1 → 74.9 ms (0.97) 104 → 102 ms (0.98) 110 → 111 ms (1.00) 109 → 107 ms (0.99) 157 → 148 ms (0.94)
clickbench_q32 293 → 292 ms (1.00) 288 → 287 ms (1.00) 290 → 292 ms (1.01) 307 → 308 ms (1.00) 359 → 368 ms (1.02) 371 → 369 ms (0.99) 386 → 388 ms (1.01)
clickbench_q33 325 → 326 ms (1.00) 326 → 324 ms (0.99) 454 → 467 ms (1.03) 381 → 377 ms (0.99) 343 → 342 ms (1.00) 350 → 340 ms (0.97) 478 → 470 ms (0.98)
clickbench_q34 328 → 318 ms (0.97) 326 → 323 ms (0.99) 514 → 450 ms (0.88) 🟢 378 → 376 ms (0.99) 354 → 353 ms (1.00) 357 → 363 ms (1.02) 457 → 483 ms (1.06)
clickbench_q35 61.6 → 62.1 ms (1.01) 60.2 → 58.0 ms (0.96) 85.4 → 84.9 ms (0.99) 64.4 → 64.2 ms (1.00) 57.4 → 59.3 ms (1.03) 56.5 → 56.5 ms (1.00) 60.9 → 62.3 ms (1.02)
clickbench_q36 37.5 → 37.9 ms (1.01) 37.3 → 37.2 ms (1.00) 46.9 → 51.6 ms (1.10) 🔴 73.9 → 73.6 ms (1.00) 11.2 → 11.8 ms (1.05) 11.8 → 11.3 ms (0.96) 30.1 → 29.9 ms (0.99)
clickbench_q37 21.2 → 20.9 ms (0.98) 21.9 → 21.6 ms (0.99) 29.6 → 28.3 ms (0.96) 35.7 → 36.0 ms (1.01) 7.79 → 7.81 ms (1.00) 7.86 → 7.62 ms (0.97) 23.6 → 23.5 ms (1.00)
clickbench_q38 12.9 → 13.1 ms (1.01) 12.0 → 11.9 ms (0.99) 24.9 → 23.4 ms (0.94) 48.1 → 47.7 ms (0.99) 8.36 → 8.14 ms (0.97) 8.53 → 8.38 ms (0.98) 23.1 → 23.2 ms (1.00)
clickbench_q39 78.4 → 78.5 ms (1.00) 76.9 → 76.5 ms (0.99) 124 → 128 ms (1.03) 144 → 144 ms (1.00) 17.9 → 18.0 ms (1.01) 18.5 → 18.3 ms (0.99) 52.5 → 52.7 ms (1.00)
clickbench_q40 10.5 → 9.96 ms (0.95) 11.1 → 11.3 ms (1.02) 18.3 → 18.4 ms (1.00) 20.0 → 20.1 ms (1.00) 6.81 → 7.06 ms (1.04) 6.99 → 6.85 ms (0.98) 14.7 → 14.9 ms (1.01)
clickbench_q41 9.59 → 9.51 ms (0.99) 10.5 → 10.6 ms (1.01) 17.1 → 17.3 ms (1.01) 19.0 → 18.9 ms (0.99) 7.81 → 7.83 ms (1.00) 7.80 → 7.72 ms (0.99) 15.4 → 15.5 ms (1.01)
clickbench_q42 9.76 → 9.92 ms (1.02) 11.1 → 10.9 ms (0.98) 15.5 → 15.9 ms (1.02) 19.3 → 19.2 ms (0.99) 7.00 → 7.05 ms (1.01) 7.09 → 6.97 ms (0.98) 14.4 → 14.0 ms (0.97)
Cold runs (first iteration): geomean 1.022x ➖ · 43 queries × 7 targets (4↑ 18↓)
query datafusion:vortex [file, v2] datafusion:vortex [file, v1] datafusion:vortex [datasource, v1] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex [file, v1] duckdb:parquet
geomean 105 → 110 ms (1.04) 103 → 106 ms (1.03) 112 → 113 ms (1.00) 156 → 160 ms (1.03) 82.8 → 85.3 ms (1.03) 80.5 → 80.7 ms (1.00) 82.2 → 83.6 ms (1.02)
clickbench_q00 22.4 → 25.3 ms (1.13) 🔴 22.2 → 26.0 ms (1.17) 🔴 24.9 → 26.5 ms (1.06) 47.6 → 51.3 ms (1.08) 24.7 → 24.9 ms (1.01) 24.0 → 23.6 ms (0.98) 14.1 → 14.0 ms (1.00)
clickbench_q01 36.3 → 33.5 ms (0.92) 37.8 → 38.8 ms (1.03) 31.7 → 29.9 ms (0.94) 67.4 → 72.6 ms (1.08) 28.2 → 27.6 ms (0.98) 25.9 → 26.9 ms (1.04) 21.8 → 20.8 ms (0.95)
clickbench_q02 46.4 → 46.7 ms (1.01) 43.8 → 39.6 ms (0.90) 33.5 → 33.7 ms (1.01) 82.2 → 82.4 ms (1.00) 35.8 → 37.5 ms (1.05) 31.1 → 31.9 ms (1.02) 30.8 → 31.8 ms (1.03)
clickbench_q03 51.6 → 54.7 ms (1.06) 47.9 → 49.6 ms (1.04) 43.2 → 45.5 ms (1.05) 76.6 → 82.5 ms (1.08) 44.1 → 48.8 ms (1.11) 🔴 47.6 → 48.0 ms (1.01) 33.3 → 31.9 ms (0.96)
clickbench_q04 106 → 104 ms (0.98) 88.4 → 94.7 ms (1.07) 106 → 126 ms (1.19) 🔴 124 → 132 ms (1.07) 86.9 → 87.9 ms (1.01) 86.2 → 86.5 ms (1.00) 69.6 → 76.3 ms (1.10)
clickbench_q05 128 → 138 ms (1.08) 126 → 123 ms (0.97) 158 → 151 ms (0.95) 155 → 158 ms (1.02) 94.0 → 94.8 ms (1.01) 89.5 → 89.2 ms (1.00) 93.2 → 96.2 ms (1.03)
clickbench_q06 22.2 → 25.3 ms (1.14) 🔴 22.4 → 25.0 ms (1.12) 🔴 28.9 → 30.2 ms (1.05) 48.5 → 51.0 ms (1.05) 32.2 → 31.9 ms (0.99) 28.9 → 30.5 ms (1.06) 29.9 → 28.4 ms (0.95)
clickbench_q07 36.6 → 39.8 ms (1.09) 35.4 → 38.0 ms (1.07) 36.7 → 36.5 ms (1.00) 67.7 → 68.6 ms (1.01) 30.3 → 31.6 ms (1.04) 29.3 → 29.1 ms (0.99) 23.3 → 22.9 ms (0.98)
clickbench_q08 121 → 133 ms (1.10) 124 → 120 ms (0.97) 146 → 142 ms (0.97) 163 → 163 ms (1.01) 96.5 → 102 ms (1.05) 99.4 → 101 ms (1.01) 91.4 → 91.1 ms (1.00)
clickbench_q09 212 → 218 ms (1.03) 211 → 204 ms (0.97) 229 → 234 ms (1.02) 268 → 271 ms (1.01) 123 → 127 ms (1.03) 123 → 124 ms (1.01) 122 → 118 ms (0.97)
clickbench_q10 67.4 → 68.2 ms (1.01) 62.6 → 64.5 ms (1.03) 61.1 → 59.3 ms (0.97) 99.3 → 102 ms (1.03) 63.3 → 67.3 ms (1.06) 61.4 → 60.4 ms (0.98) 53.8 → 54.1 ms (1.01)
clickbench_q11 67.5 → 71.6 ms (1.06) 63.5 → 64.6 ms (1.02) 65.9 → 64.7 ms (0.98) 107 → 111 ms (1.04) 65.3 → 71.7 ms (1.10) 61.0 → 61.9 ms (1.02) 61.5 → 60.8 ms (0.99)
clickbench_q12 124 → 128 ms (1.03) 120 → 121 ms (1.01) 151 → 146 ms (0.97) 159 → 164 ms (1.03) 101 → 98.4 ms (0.97) 94.1 → 93.9 ms (1.00) 108 → 108 ms (1.01)
clickbench_q13 184 → 197 ms (1.07) 179 → 177 ms (0.99) 188 → 187 ms (1.00) 224 → 230 ms (1.03) 158 → 166 ms (1.05) 151 → 151 ms (1.00) 168 → 167 ms (0.99)
clickbench_q14 129 → 131 ms (1.01) 123 → 129 ms (1.05) 171 → 147 ms (0.86) 🟢 169 → 173 ms (1.03) 100 → 104 ms (1.04) 99.3 → 97.9 ms (0.99) 117 → 122 ms (1.05)
clickbench_q15 108 → 104 ms (0.97) 99.4 → 101 ms (1.02) 124 → 125 ms (1.01) 137 → 132 ms (0.96) 106 → 111 ms (1.04) 108 → 108 ms (1.00) 103 → 99.0 ms (0.96)
clickbench_q16 213 → 220 ms (1.03) 215 → 215 ms (1.00) 277 → 273 ms (0.98) 234 → 235 ms (1.00) 171 → 169 ms (0.99) 172 → 170 ms (0.99) 180 → 180 ms (1.00)
clickbench_q17 224 → 223 ms (1.00) 208 → 219 ms (1.06) 276 → 270 ms (0.98) 237 → 237 ms (1.00) 168 → 171 ms (1.02) 168 → 168 ms (1.00) 181 → 183 ms (1.01)
clickbench_q18 384 → 388 ms (1.01) 379 → 370 ms (0.98) 438 → 455 ms (1.04) 415 → 421 ms (1.01) 307 → 312 ms (1.02) 303 → 302 ms (1.00) 319 → 322 ms (1.01)
clickbench_q19 43.1 → 45.3 ms (1.05) 44.4 → 44.3 ms (1.00) 41.0 → 40.8 ms (0.99) 74.6 → 78.4 ms (1.05) 35.5 → 33.4 ms (0.94) 35.6 → 36.1 ms (1.01) 25.5 → 26.9 ms (1.06)
clickbench_q20 236 → 235 ms (1.00) 226 → 226 ms (1.00) 203 → 207 ms (1.02) 276 → 284 ms (1.03) 156 → 158 ms (1.02) 156 → 155 ms (1.00) 224 → 231 ms (1.03)
clickbench_q21 274 → 274 ms (1.00) 274 → 273 ms (1.00) 262 → 256 ms (0.98) 290 → 304 ms (1.05) 176 → 187 ms (1.06) 249 → 251 ms (1.01) 269 → 271 ms (1.01)
clickbench_q22 385 → 374 ms (0.97) 337 → 359 ms (1.06) 328 → 336 ms (1.02) 416 → 426 ms (1.03) 296 → 298 ms (1.01) 343 → 346 ms (1.01) 354 → 364 ms (1.03)
clickbench_q23 366 → 339 ms (0.93) 470 → 446 ms (0.95) 359 → 585 ms (1.63) 🔴 1.18 → 1.21 s (1.03) 160 → 159 ms (1.00) 84.3 → 80.1 ms (0.95) 247 → 241 ms (0.98)
clickbench_q24 55.5 → 69.9 ms (1.26) 🔴 62.1 → 67.0 ms (1.08) 53.1 → 42.8 ms (0.81) 🟢 87.2 → 90.2 ms (1.04) 44.3 → 45.1 ms (1.02) 35.0 → 34.2 ms (0.98) 45.3 → 49.6 ms (1.10)
clickbench_q25 63.7 → 67.1 ms (1.05) 57.4 → 61.2 ms (1.07) 47.2 → 48.5 ms (1.03) 101 → 102 ms (1.01) 50.2 → 53.6 ms (1.07) 48.5 → 49.0 ms (1.01) 61.2 → 61.2 ms (1.00)
clickbench_q26 63.3 → 73.1 ms (1.16) 🔴 64.1 → 66.2 ms (1.03) 48.0 → 48.2 ms (1.01) 89.2 → 92.6 ms (1.04) 41.6 → 47.5 ms (1.14) 🔴 31.0 → 32.6 ms (1.05) 39.3 → 45.5 ms (1.16) 🔴
clickbench_q27 258 → 255 ms (0.99) 245 → 242 ms (0.99) 225 → 231 ms (1.03) 280 → 294 ms (1.05) 170 → 177 ms (1.04) 212 → 215 ms (1.01) 219 → 226 ms (1.03)
clickbench_q28 640 → 640 ms (1.00) 622 → 619 ms (0.99) 991 → 914 ms (0.92) 702 → 706 ms (1.01) 550 → 562 ms (1.02) 559 → 562 ms (1.00) 938 → 973 ms (1.04)
clickbench_q29 59.2 → 60.9 ms (1.03) 56.2 → 56.5 ms (1.01) 46.9 → 46.4 ms (0.99) 90.1 → 91.9 ms (1.02) 41.1 → 45.2 ms (1.10) 40.0 → 38.2 ms (0.96) 32.0 → 32.0 ms (1.00)
clickbench_q30 108 → 117 ms (1.08) 105 → 111 ms (1.05) 129 → 131 ms (1.02) 167 → 175 ms (1.05) 102 → 108 ms (1.06) 98.6 → 99.1 ms (1.01) 127 → 127 ms (1.00)
clickbench_q31 154 → 158 ms (1.02) 154 → 157 ms (1.02) 165 → 169 ms (1.03) 191 → 201 ms (1.05) 183 → 172 ms (0.94) 157 → 180 ms (1.15) 🔴 173 → 200 ms (1.15) 🔴
clickbench_q32 391 → 391 ms (1.00) 383 → 384 ms (1.00) 415 → 413 ms (1.00) 427 → 425 ms (1.00) 395 → 399 ms (1.01) 404 → 401 ms (0.99) 402 → 410 ms (1.02)
clickbench_q33 437 → 438 ms (1.00) 433 → 434 ms (1.00) 564 → 565 ms (1.00) 468 → 487 ms (1.04) 389 → 398 ms (1.02) 429 → 417 ms (0.97) 440 → 453 ms (1.03)
clickbench_q34 431 → 439 ms (1.02) 436 → 454 ms (1.04) 698 → 672 ms (0.96) 474 → 481 ms (1.01) 415 → 405 ms (0.98) 437 → 434 ms (0.99) 438 → 473 ms (1.08)
clickbench_q35 95.6 → 105 ms (1.09) 89.8 → 93.5 ms (1.04) 131 → 128 ms (0.98) 124 → 125 ms (1.00) 85.5 → 90.9 ms (1.06) 89.6 → 87.5 ms (0.98) 75.3 → 73.9 ms (0.98)
clickbench_q36 66.8 → 71.1 ms (1.06) 67.2 → 68.1 ms (1.01) 83.4 → 72.8 ms (0.87) 🟢 133 → 137 ms (1.03) 32.9 → 34.8 ms (1.06) 30.6 → 32.8 ms (1.07) 35.8 → 36.3 ms (1.02)
clickbench_q37 46.6 → 50.5 ms (1.08) 48.6 → 50.1 ms (1.03) 49.6 → 55.5 ms (1.12) 🔴 90.5 → 91.9 ms (1.02) 29.2 → 29.3 ms (1.00) 31.1 → 27.2 ms (0.88) 🟢 28.9 → 29.7 ms (1.03)
clickbench_q38 39.9 → 43.2 ms (1.08) 38.9 → 41.8 ms (1.07) 50.4 → 50.0 ms (0.99) 107 → 108 ms (1.01) 29.8 → 31.6 ms (1.06) 26.9 → 30.1 ms (1.12) 🔴 29.8 → 29.7 ms (0.99)
clickbench_q39 111 → 116 ms (1.04) 108 → 112 ms (1.03) 128 → 126 ms (0.98) 207 → 213 ms (1.03) 42.5 → 44.1 ms (1.04) 41.9 → 40.7 ms (0.97) 59.5 → 61.4 ms (1.03)
clickbench_q40 34.9 → 37.2 ms (1.07) 36.0 → 40.4 ms (1.12) 🔴 41.9 → 42.9 ms (1.02) 76.3 → 78.5 ms (1.03) 29.4 → 30.1 ms (1.03) 27.7 → 27.5 ms (0.99) 20.0 → 19.6 ms (0.98)
clickbench_q41 34.7 → 38.4 ms (1.11) 🔴 35.7 → 38.9 ms (1.09) 39.9 → 40.9 ms (1.03) 76.7 → 76.6 ms (1.00) 28.9 → 30.6 ms (1.06) 26.8 → 27.3 ms (1.02) 20.2 → 21.0 ms (1.04)
clickbench_q42 35.7 → 40.0 ms (1.12) 🔴 36.4 → 38.2 ms (1.05) 39.8 → 39.5 ms (0.99) 74.5 → 75.2 ms (1.01) 26.3 → 27.9 ms (1.06) 28.5 → 26.7 ms (0.94) 20.3 → 20.6 ms (1.02)

No file size changes detected.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant