Skip to content

Optimize shared Parquet config, and record Snappy Parquet sizes - #11720

Merged
lwwmanning merged 44 commits into
developfrom
wm/file-size-parquet-codecs
Nov 21, 2026
Merged

lwwmanning merged 44 commits into
developfrom
wm/file-size-parquet-codecs

Conversation

@lwwmanning

@lwwmanning lwwmanning commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

This changes how the benchmarks write Parquet, and which Parquet sizes they record. ClickBench and FineWeb still query their published files as parquet.

Before, each suite set its own Parquet writer settings. statpopgen and SpatialBench's zone table wrote uncompressed Parquet, and the vortex suite wrote Snappy. Now every generated Parquet file uses one config, in vortex-bench/src/utils/parquet.rs. It sets zstd level 3, dictionaries of at most 64 KiB, a page index, and pages of at most 20,000 rows and 1 MiB. Each file records a layout version, and the generators write a cached file in an older layout again.

Before, TPC-H and nested TPC-H wrote row groups of 1,048,576 rows on all storage, and TPC-DS wrote 122,880. Now every query suite writes 122,880 rows (DuckDB's default) for local NVMe and 1,048,576 rows (parquet-rs's default) for S3. In a TPC-H test at scale factor 10, the small row groups were 28% to 38% faster on NVMe. On S3, the large row groups were 14% to 21% faster on DataFusion, and 3% to 8% slower on DuckDB.

Each other setting was tested on its own, on TPC-H at scale factor 10 and on the random access datasets. 1 MiB dictionaries made files 22% larger, DataFusion 8% slower, and lookups up to 2.8 times slower. Bloom filters made files 14% larger and sped up no query. The 20,000-row page limit made lookups up to 4 times faster at the same size.

compress-bench and TPC-DS at scale factor 1 now also record the size of parquet-snappy: the same Parquet as parquet, with Snappy in place of zstd. No query reads it. TPC-DS at scale factor 1 also records the size of Arrow IPC and queries Lance on DataFusion. Lance skips q72, which overflows its 32-bit string offsets.

Parquet sizes and times change for every suite that generates Parquet. The bench side is vortex-data/vortex.dev#227.

🤖 Generated with Claude Code

capture-file-sizes.py reads each Parquet file's footer and names its
format by codec: zstd stays `parquet`, the codec the benchmarks write,
and other codecs record as `parquet-<codec>`, as `vortex` and
`vortex-compact` name Vortex's. ClickBench and FineWeb measure
downloaded files, and the statpopgen and vortex suites write without
zstd, so their sizes were recorded as zstd Parquet.

Signed-off-by: Will Manning <will@willmanning.io>
compress-bench writes each dataset once more as Snappy Parquet, untimed,
and records its size as `parquet-snappy` beside `parquet` (zstd level
3). Only sizes: Snappy adds no compression or decompression timings.

Signed-off-by: Will Manning <will@willmanning.io>
compress-bench now writes each dataset once more with the compact Vortex
strategy and records its size as vortex-compact, as the SQL benchmarks
name it. The write is not timed and does not run in GPU mode. bench can
then compare Vortex and compact Vortex on the compression datasets, as it
does on the SQL benchmarks' files.

Signed-off-by: Will Manning <will@willmanning.io>
Develop's TPC-DS SF1 run now writes three formats that no query reads,
after its queries, so their file sizes are recorded with the others:

- `data-gen --size-formats arrow-ipc,parquet-snappy` writes uncompressed
  Arrow IPC and Snappy Parquet from the base Parquet files. The Snappy
  files keep the base files' other writer settings and row groups.
- `lance-bench --data-only` writes the Lance data and exits.
- `vx-bench write-size-formats` runs both, from a new `size_formats`
  list on a matrix coverage. CI runs it before "Capture file sizes".

`capture-file-sizes.py` records the `parquet-snappy` directory, and
labels its files by codec like the `parquet` directory's.

At SF1 the writes take about 8 seconds and 3.8 GB of disk.

Signed-off-by: Will Manning <will@willmanning.io>
Develop's TPC-DS SF1 run now queries Snappy Parquet on DataFusion and
DuckDB, and Lance on DataFusion. data-gen writes Snappy Parquet as the
parquet-snappy format, from the base Parquet files with the same row
groups. Arrow IPC stays a size-only format.

Lance skips TPC-DS q72, which overflows its Utf8 offsets. bench ranks a
series that misses a query below every series that ran them all.

Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning lwwmanning added the changelog/chore A trivial change label Nov 20, 2026
@lwwmanning
lwwmanning marked this pull request as ready for review November 21, 2026 01:58
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-21T22:07:28.849251Z 3b15172 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Signed-off-by: Will Manning <will@willmanning.io>
… vortex suite

Every benchmark writes zstd level 3 Parquet, but three writers did not:
statpopgen and the SpatialBench zone rewrite passed no writer settings,
so parquet-rs wrote them uncompressed, and the vortex suite's DuckDB
COPY named no codec, so DuckDB wrote Snappy. Their Parquet sizes and
times measured a different codec from every other suite. All three now
write zstd level 3. statpopgen's CI data cache also keys on the shared
writer settings, so a change to them regenerates the cached files.

Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning lwwmanning changed the title Benchmarks record Parquet sizes by codec, and TPC-DS SF1 queries Snappy Parquet and Lance Benchmarks write and record zstd Parquet, and TPC-DS SF1 queries Snappy Parquet and Lance Nov 21, 2026
`parquet-snappy` is now Parquet as DuckDB writes it: `COPY ... (FORMAT
parquet, COMPRESSION snappy)` in the `duckdb` CLI, with DuckDB's row
groups, dictionaries, statistics, and Bloom filters. data-gen writes the
query files this way, and compress-bench records its Snappy size from the
same writer. Every benchmark runner already installs the CLI.

File sizes are named by directory again, not by codec. `parquet` is the
suite's base Parquet: zstd level 3 where the benchmarks write it, and the
dataset's own files where the suite prescribes them, as ClickBench's
Snappy files are. This keeps ClickBench's `parquet` series continuous and
keeps `parquet-snappy` for DuckDB's files.

Signed-off-by: Will Manning <will@willmanning.io>
#11676 changed the integer cast's overflow error from "value exceeds
target range" to "values exceed target range". The device batch test
looked for the old wording, so CUDA tests fail on develop's code. Match
the words both share.

Signed-off-by: Will Manning <will@willmanning.io>
The queries run on DuckDB 2.0.0, which vortex-duckdb links, but the
runners installed the 1.5.5 CLI, which writes parquet-snappy and the
native duckdb database. Install the same version. DuckDB 2.0 ships the
CLI as a tarball, so the action extracts it with tar.

Signed-off-by: Will Manning <will@willmanning.io>
lwwmanning added a commit to vortex-data/vortex.dev that referenced this pull request Nov 21, 2026
vortex-data/vortex#11720 now installs the DuckDB 2.0.0 CLI, the version
the queries run on.

Signed-off-by: Will Manning <will@willmanning.io>
2.0.1 is DuckDB 2.0's bugfix release. vortex-duckdb still links 2.0.0, so
the CLI stays on the minor version the queries run on.

Signed-off-by: Will Manning <will@willmanning.io>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 147ab29be9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread vortex-bench/src/conversions.rs Resolved
Comment thread benchmarks/compress-bench/src/main.rs Resolved
…zone files

compress-bench wrote its size-only formats, DuckDB's Snappy Parquet and
compact Vortex, between a format's timed compression and its timed
decompression. Write them after every timed run of the dataset.

vx-bench run dropped the OS caches before each query, but lance-bench
wrote its Lance files on its first query, so that query read them from the
page cache. The run now writes Lance's data before the first cache drop.

SpatialBench's zone file was left as it was when it already had GeoParquet
metadata, so a file tagged before the benchmarks wrote zstd stayed
uncompressed. It is now rewritten unless it is also zstd.

Signed-off-by: Will Manning <will@willmanning.io>
Writing them after the timed runs raised run_benchmark_for_dataset's
cognitive complexity to 27, over clippy's limit of 25.

Signed-off-by: Will Manning <will@willmanning.io>
@codspeed

codspeed Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 2036 untouched benchmarks
⏩ 7 skipped benchmarks1


Comparing wm/file-size-parquet-codecs (3b15172) with develop (b5d8235)

Open in CodSpeed

Footnotes

  1. 7 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Develop runs DataFusion and DuckDB on parquet-snappy beside zstd Parquet,
so bench can compare the two Parquet variants on local disk and on S3.

Signed-off-by: Will Manning <will@willmanning.io>
Lets this pull request's action/bench-sql run compare zstd and Snappy
Parquet on NVMe and S3. Revert before merge.

Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning lwwmanning added the action/bench-sql Run SQL benchmarks without Vortex Compact on this PR label Nov 21, 2026
@github-actions github-actions Bot removed the action/bench-sql Run SQL benchmarks without Vortex Compact on this PR label Nov 21, 2026
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 3b15172 compress-bench 1 Explore Profiling Data
Previous Runs (56)
Status Commit Job Attempt Link
🟢 Done aff33cb job-nvme 2 Explore Profiling Data
🟢 Done aff33cb tpch-s3-10 2 Explore Profiling Data
🟢 Done aff33cb string-bench 2 Explore Profiling Data
🟢 Done aff33cb random-access-bench 1 Explore Profiling Data
🟢 Done aff33cb vortex-queries 1 Explore Profiling Data
🟢 Done aff33cb compress-bench 1 Explore Profiling Data
🟢 Done aff33cb statpopgen 1 Explore Profiling Data
🟢 Done aff33cb clickbench-sorted-nvme 1 Explore Profiling Data
🟢 Done aff33cb clickbench-nvme 1 Explore Profiling Data
🟢 Done aff33cb fineweb 1 Explore Profiling Data
🟢 Done aff33cb fineweb-s3 1 Explore Profiling Data
🟢 Done aff33cb tpch-nvme-10 1 Explore Profiling Data
🟢 Done aff33cb appian-nvme 1 Explore Profiling Data
🟢 Done aff33cb polarsignals 1 Explore Profiling Data
🟢 Done aff33cb tpcds-nvme-10 1 Explore Profiling Data
🟢 Done aff33cb tpcds-nvme 1 Explore Profiling Data
🟢 Done aff33cb tpch-nvme 1 Explore Profiling Data
🟢 Done aff33cb tpch-nested-nvme 1 Explore Profiling Data
🟢 Done aff33cb tpch-s3 1 Explore Profiling Data
🟢 Done aff33cb job-nvme 1 Explore Profiling Data
🟢 Done aff33cb tpch-s3-10 1 Explore Profiling Data
🟢 Done aff33cb string-bench 1 Explore Profiling Data
🟢 Done 3d619c0 clickbench-nvme 1 Explore Profiling Data
🟢 Done 3d619c0 polarsignals 1 Explore Profiling Data
🟢 Done 3d619c0 fineweb 1 Explore Profiling Data
🟢 Done 3d619c0 tpcds-nvme-10 1 Explore Profiling Data
🟢 Done 3d619c0 tpch-nvme-10 1 Explore Profiling Data
🟢 Done 3d619c0 tpch-s3 1 Explore Profiling Data
🟢 Done 3d619c0 tpch-nvme 1 Explore Profiling Data
🟢 Done 3d619c0 tpcds-nvme 1 Explore Profiling Data
🟢 Done 6f5e01b tpch-s3-10 1 Explore Profiling Data
🟢 Done 6f5e01b tpch-nvme-10 1 Explore Profiling Data
🟢 Done 6f5e01b clickbench-nvme 1 Explore Profiling Data
🟢 Done 6f5e01b polarsignals 1 Explore Profiling Data
🟢 Done 6f5e01b tpcds-nvme-10 1 Explore Profiling Data
🟢 Done 6f5e01b fineweb 1 Explore Profiling Data
🟢 Done 6f5e01b tpch-s3 1 Explore Profiling Data
🟢 Done 6f5e01b tpch-nvme 1 Explore Profiling Data
🟢 Done 6f5e01b tpcds-nvme 1 Explore Profiling Data
🟢 Done 8ca376c tpch-nvme-10 1 Explore Profiling Data
🟢 Done 8ca376c tpch-s3-10 1 Explore Profiling Data
🟢 Done 8ca376c clickbench-nvme 1 Explore Profiling Data
🟢 Done 8ca376c tpcds-nvme-10 1 Explore Profiling Data
🟢 Done 8ca376c fineweb 1 Explore Profiling Data
🟢 Done 8ca376c tpch-nvme 1 Explore Profiling Data
🟢 Done 8ca376c tpcds-nvme 1 Explore Profiling Data
🟢 Done 8ca376c tpch-s3 1 Explore Profiling Data
🟢 Done e0e95b1 clickbench-nvme 1 Explore Profiling Data
🟢 Done e0e95b1 tpch-nvme-10 1 Explore Profiling Data
🟢 Done e0e95b1 tpch-s3-10 1 Explore Profiling Data
🟢 Done e0e95b1 tpcds-nvme-10 1 Explore Profiling Data
🟢 Done e0e95b1 polarsignals 1 Explore Profiling Data
🟢 Done e0e95b1 fineweb 1 Explore Profiling Data
🟢 Done e0e95b1 tpcds-nvme 1 Explore Profiling Data
🟢 Done e0e95b1 tpch-nvme 1 Explore Profiling Data
🟢 Done e0e95b1 tpch-s3 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟡 In Progress e0e95b1 tpch-nvme 1 Explore Profiling Data

Powered by Polar Signals Cloud

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e0e95b1de1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread vortex-bench/sql/vortex/init.sql Resolved
compress-bench now records zstd Parquet's whole file, footer included,
as it does for Vortex and DuckDB's Snappy Parquet. The Vortex query suite
and statpopgen write their Parquet again when a cached file is not zstd.

Signed-off-by: Will Manning <will@willmanning.io>
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-DS SF=10 on NVME 📖

Commits: PR 3d619c0e vs base c84406eb
Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.1%
Engines: DataFusion No clear signal (+0.3%, low confidence) · DuckDB No clear signal (-0.1%, low confidence)
Vortex (geomean): hot 0.991x ➖ · cold 0.988x ➖
Parquet (geomean): hot 0.990x ➖ · cold 0.992x ➖
Shifts: Parquet (control) -1.0% · Median polish -0.9%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.991x ➖ · 99 queries × 4 targets (0↑ 0↓)

Each cell is develop c84406eb → this PR 3d619c0e (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:parquet
geomean 54.1 → 53.9 ms (1.00) 85.1 → 84.5 ms (0.99) 40.5 → 40.0 ms (0.99) 57.2 → 56.5 ms (0.99)
tpcds_q01 35.2 → 35.4 ms (1.01) 59.4 → 58.6 ms (0.99) 32.8 → 32.4 ms (0.99) 41.9 → 41.0 ms (0.98)
tpcds_q02 50.8 → 50.4 ms (0.99) 68.7 → 68.2 ms (0.99) 30.2 → 30.7 ms (1.02) 40.1 → 38.2 ms (0.95)
tpcds_q03 21.1 → 21.7 ms (1.03) 29.2 → 28.3 ms (0.97) 15.3 → 15.2 ms (0.99) 25.2 → 25.3 ms (1.00)
tpcds_q04 296 → 294 ms (1.00) 645 → 640 ms (0.99) 130 → 130 ms (1.00) 203 → 204 ms (1.01)
tpcds_q05 73.7 → 72.6 ms (0.99) 81.5 → 81.9 ms (1.00) 56.7 → 55.5 ms (0.98) 70.2 → 68.4 ms (0.97)
tpcds_q06 44.6 → 45.2 ms (1.01) 90.1 → 91.5 ms (1.02) 59.4 → 58.6 ms (0.99) 75.8 → 73.4 ms (0.97)
tpcds_q07 42.5 → 43.0 ms (1.01) 102 → 100 ms (0.99) 31.8 → 32.1 ms (1.01) 59.8 → 60.5 ms (1.01)
tpcds_q08 30.5 → 31.4 ms (1.03) 48.4 → 47.5 ms (0.98) 33.0 → 32.1 ms (0.97) 40.7 → 37.4 ms (0.92)
tpcds_q09 43.5 → 41.8 ms (0.96) 75.7 → 74.6 ms (0.98) 56.5 → 56.2 ms (0.99) 85.1 → 85.4 ms (1.00)
tpcds_q10 40.7 → 40.4 ms (0.99) 58.5 → 57.9 ms (0.99) 41.1 → 39.6 ms (0.96) 53.2 → 51.9 ms (0.98)
tpcds_q11 177 → 177 ms (1.00) 417 → 409 ms (0.98) 97.4 → 96.1 ms (0.99) 158 → 157 ms (0.99)
tpcds_q12 17.7 → 17.6 ms (0.99) 40.5 → 39.7 ms (0.98) 17.8 → 17.6 ms (0.99) 39.7 → 40.4 ms (1.02)
tpcds_q13 39.4 → 39.8 ms (1.01) 86.0 → 84.2 ms (0.98) 40.5 → 39.3 ms (0.97) 72.2 → 69.0 ms (0.96)
tpcds_q14 238 → 240 ms (1.01) 383 → 371 ms (0.97) 128 → 124 ms (0.97) 188 → 186 ms (0.99)
tpcds_q15 41.2 → 40.7 ms (0.99) 50.4 → 50.6 ms (1.00) 61.0 → 61.1 ms (1.00) 61.4 → 60.4 ms (0.98)
tpcds_q16 23.9 → 24.0 ms (1.00) 45.5 → 44.7 ms (0.98) 34.6 → 33.9 ms (0.98) 38.1 → 37.5 ms (0.98)
tpcds_q17 68.9 → 66.2 ms (0.96) 138 → 136 ms (0.98) 56.9 → 55.9 ms (0.98) 83.0 → 82.9 ms (1.00)
tpcds_q18 68.1 → 68.1 ms (1.00) 110 → 108 ms (0.99) 83.6 → 82.8 ms (0.99) 88.7 → 86.6 ms (0.98)
tpcds_q19 25.3 → 25.9 ms (1.02) 47.8 → 47.3 ms (0.99) 46.2 → 45.2 ms (0.98) 47.6 → 49.0 ms (1.03)
tpcds_q20 23.7 → 23.2 ms (0.98) 41.5 → 41.0 ms (0.99) 22.6 → 22.2 ms (0.98) 42.5 → 41.9 ms (0.99)
tpcds_q21 47.5 → 46.1 ms (0.97) 61.3 → 60.8 ms (0.99) 35.8 → 35.2 ms (0.98) 24.0 → 23.3 ms (0.97)
tpcds_q22 450 → 445 ms (0.99) 462 → 461 ms (1.00) 158 → 157 ms (1.00) 163 → 164 ms (1.00)
tpcds_q23 296 → 298 ms (1.01) 358 → 369 ms (1.03) 184 → 186 ms (1.01) 182 → 183 ms (1.01)
tpcds_q24 122 → 121 ms (1.00) 241 → 238 ms (0.99) 52.2 → 51.8 ms (0.99) 71.2 → 70.6 ms (0.99)
tpcds_q25 64.2 → 64.7 ms (1.01) 147 → 143 ms (0.97) 52.4 → 52.5 ms (1.00) 51.9 → 50.3 ms (0.97)
tpcds_q26 36.9 → 37.2 ms (1.01) 54.8 → 55.3 ms (1.01) 27.3 → 26.5 ms (0.97) 43.6 → 42.1 ms (0.97)
tpcds_q27 73.5 → 76.5 ms (1.04) 206 → 200 ms (0.97) 34.1 → 33.4 ms (0.98) 57.9 → 55.6 ms (0.96)
tpcds_q28 42.9 → 44.0 ms (1.03) 89.4 → 86.2 ms (0.96) 43.4 → 43.4 ms (1.00) 93.0 → 93.5 ms (1.01)
tpcds_q29 68.4 → 64.5 ms (0.94) 141 → 141 ms (1.00) 51.6 → 51.2 ms (0.99) 84.6 → 84.2 ms (1.00)
tpcds_q30 51.9 → 51.1 ms (0.98) 97.1 → 98.7 ms (1.02) 38.5 → 38.8 ms (1.01) 71.9 → 71.8 ms (1.00)
tpcds_q31 209 → 205 ms (0.98) 320 → 318 ms (0.99) 39.5 → 39.4 ms (1.00) 98.8 → 97.8 ms (0.99)
tpcds_q32 15.3 → 15.1 ms (0.99) 29.2 → 29.2 ms (1.00) 14.5 → 13.9 ms (0.96) 18.3 → 18.1 ms (0.99)
tpcds_q33 35.0 → 37.7 ms (1.08) 53.7 → 52.1 ms (0.97) 35.8 → 35.9 ms (1.00) 61.6 → 60.7 ms (0.98)
tpcds_q34 45.4 → 43.4 ms (0.96) 63.5 → 64.1 ms (1.01) 25.4 → 24.2 ms (0.95) 33.1 → 31.5 ms (0.95)
tpcds_q35 94.8 → 93.7 ms (0.99) 125 → 124 ms (0.99) 69.6 → 67.9 ms (0.98) 77.6 → 76.4 ms (0.99)
tpcds_q36 69.5 → 70.6 ms (1.02) 97.3 → 97.6 ms (1.00) 30.8 → 30.8 ms (1.00) 97.0 → 98.5 ms (1.02)
tpcds_q37 25.5 → 23.7 ms (0.93) 50.8 → 49.8 ms (0.98) 27.8 → 26.8 ms (0.96) 41.4 → 40.8 ms (0.98)
tpcds_q38 80.5 → 81.6 ms (1.01) 121 → 121 ms (1.00) 64.4 → 64.1 ms (1.00) 77.6 → 74.1 ms (0.96)
tpcds_q39 827 → 818 ms (0.99) 847 → 847 ms (1.00) 58.1 → 57.4 ms (0.99) 61.2 → 61.0 ms (1.00)
tpcds_q40 28.7 → 28.3 ms (0.99) 35.3 → 35.0 ms (0.99) 18.9 → 19.1 ms (1.01) 27.9 → 27.8 ms (1.00)
tpcds_q41 19.3 → 20.0 ms (1.04) 32.4 → 31.0 ms (0.96) 19.5 → 19.2 ms (0.99) 20.8 → 20.1 ms (0.97)
tpcds_q42 17.1 → 16.4 ms (0.96) 25.8 → 26.1 ms (1.01) 15.9 → 16.4 ms (1.03) 25.9 → 26.1 ms (1.01)
tpcds_q43 18.7 → 18.8 ms (1.01) 28.3 → 26.0 ms (0.92) 21.7 → 21.4 ms (0.99) 28.2 → 25.7 ms (0.91)
tpcds_q44 43.0 → 42.9 ms (1.00) 64.8 → 65.6 ms (1.01) 28.1 → 27.6 ms (0.98) 59.9 → 59.9 ms (1.00)
tpcds_q45 43.4 → 42.1 ms (0.97) 65.8 → 66.1 ms (1.01) 46.6 → 46.8 ms (1.00) 60.7 → 60.6 ms (1.00)
tpcds_q46 76.2 → 77.1 ms (1.01) 108 → 108 ms (1.00) 51.9 → 52.1 ms (1.00) 71.2 → 72.4 ms (1.02)
tpcds_q47 182 → 182 ms (1.00) 236 → 235 ms (0.99) 82.6 → 81.8 ms (0.99) 90.7 → 89.7 ms (0.99)
tpcds_q48 37.4 → 36.8 ms (0.98) 78.0 → 76.7 ms (0.98) 55.4 → 55.0 ms (0.99) 67.6 → 66.3 ms (0.98)
tpcds_q49 81.4 → 80.2 ms (0.99) 112 → 117 ms (1.05) 40.5 → 39.1 ms (0.97) 68.6 → 66.9 ms (0.98)
tpcds_q50 57.4 → 58.2 ms (1.01) 89.8 → 90.8 ms (1.01) 34.5 → 33.8 ms (0.98) 44.6 → 41.9 ms (0.94)
tpcds_q51 246 → 248 ms (1.01) 256 → 253 ms (0.99) 208 → 205 ms (0.99) 216 → 216 ms (1.00)
tpcds_q52 17.7 → 17.2 ms (0.97) 26.5 → 26.8 ms (1.01) 15.9 → 16.2 ms (1.02) 27.8 → 26.6 ms (0.96)
tpcds_q53 24.1 → 24.1 ms (1.00) 36.2 → 36.1 ms (1.00) 24.0 → 23.6 ms (0.99) 34.7 → 34.4 ms (0.99)
tpcds_q54 59.5 → 59.7 ms (1.00) 86.7 → 85.3 ms (0.98) 46.2 → 45.1 ms (0.98) 55.3 → 54.1 ms (0.98)
tpcds_q55 16.9 → 17.4 ms (1.03) 26.2 → 25.8 ms (0.98) 16.4 → 16.1 ms (0.98) 27.2 → 26.5 ms (0.97)
tpcds_q56 34.8 → 34.5 ms (0.99) 52.3 → 52.3 ms (1.00) 35.5 → 34.1 ms (0.96) 70.5 → 68.2 ms (0.97)
tpcds_q57 132 → 129 ms (0.98) 172 → 171 ms (1.00) 64.7 → 63.5 ms (0.98) 68.5 → 67.8 ms (0.99)
tpcds_q58 50.6 → 50.7 ms (1.00) 98.0 → 96.8 ms (0.99) 38.8 → 38.9 ms (1.00) 55.2 → 55.9 ms (1.01)
tpcds_q59 57.3 → 57.4 ms (1.00) 104 → 104 ms (0.99) 56.4 → 56.5 ms (1.00) 60.3 → 60.1 ms (1.00)
tpcds_q60 36.1 → 35.4 ms (0.98) 54.6 → 54.5 ms (1.00) 36.6 → 35.5 ms (0.97) 81.4 → 81.5 ms (1.00)
tpcds_q61 51.9 → 51.6 ms (0.99) 76.3 → 76.1 ms (1.00) 36.5 → 36.5 ms (1.00) 80.4 → 80.1 ms (1.00)
tpcds_q62 19.0 → 19.2 ms (1.01) 27.7 → 27.0 ms (0.98) 15.3 → 15.0 ms (0.98) 20.9 → 20.6 ms (0.99)
tpcds_q63 24.8 → 24.1 ms (0.97) 37.2 → 36.2 ms (0.97) 23.4 → 22.6 ms (0.97) 33.7 → 33.0 ms (0.98)
tpcds_q64 335 → 331 ms (0.99) 665 → 660 ms (0.99) 119 → 121 ms (1.01) 136 → 134 ms (0.99)
tpcds_q65 77.4 → 76.1 ms (0.98) 101 → 101 ms (1.00) 53.7 → 53.9 ms (1.00) 93.1 → 91.2 ms (0.98)
tpcds_q66 52.4 → 52.3 ms (1.00) 76.0 → 75.2 ms (0.99) 42.4 → 43.6 ms (1.03) 47.6 → 47.0 ms (0.99)
tpcds_q67 424 → 423 ms (1.00) 434 → 430 ms (0.99) 275 → 276 ms (1.00) 293 → 289 ms (0.99)
tpcds_q68 67.4 → 72.0 ms (1.07) 109 → 109 ms (1.00) 45.9 → 45.0 ms (0.98) 60.3 → 62.6 ms (1.04)
tpcds_q69 39.8 → 40.8 ms (1.03) 59.8 → 57.4 ms (0.96) 39.9 → 39.3 ms (0.99) 65.8 → 64.9 ms (0.99)
tpcds_q70 195 → 197 ms (1.01) 211 → 210 ms (0.99) 27.8 → 27.6 ms (0.99) 41.5 → 42.1 ms (1.01)
tpcds_q71 30.0 → 27.9 ms (0.93) 39.2 → 37.2 ms (0.95) 29.7 → 29.5 ms (1.00) 40.7 → 40.5 ms (1.00)
tpcds_q72 1.58 → 1.60 s (1.01) 1.77 → 1.78 s (1.01) 140 → 138 ms (0.98) 174 → 170 ms (0.98)
tpcds_q73 41.2 → 40.3 ms (0.98) 62.2 → 62.2 ms (1.00) 23.0 → 21.7 ms (0.94) 27.4 → 27.9 ms (1.02)
tpcds_q74 137 → 137 ms (1.00) 295 → 292 ms (0.99) 87.7 → 84.1 ms (0.96) 214 → 213 ms (0.99)
tpcds_q75 166 → 160 ms (0.96) 194 → 193 ms (1.00) 114 → 110 ms (0.96) 133 → 132 ms (0.99)
tpcds_q76 35.1 → 34.3 ms (0.98) 50.9 → 51.1 ms (1.01) 29.1 → 28.2 ms (0.97) 50.1 → 52.0 ms (1.04)
tpcds_q77 57.5 → 57.0 ms (0.99) 72.5 → 70.0 ms (0.97) 38.0 → 37.4 ms (0.98) 55.4 → 53.3 ms (0.96)
tpcds_q78 142 → 142 ms (1.00) 211 → 210 ms (1.00) 137 → 137 ms (1.00) 150 → 150 ms (1.00)
tpcds_q79 57.0 → 55.9 ms (0.98) 77.8 → 78.4 ms (1.01) 39.3 → 38.1 ms (0.97) 51.2 → 50.1 ms (0.98)
tpcds_q80 82.5 → 82.7 ms (1.00) 144 → 142 ms (0.99) 59.5 → 59.8 ms (1.01) 83.2 → 81.8 ms (0.98)
tpcds_q81 56.0 → 55.6 ms (0.99) 107 → 108 ms (1.01) 57.6 → 56.9 ms (0.99) 97.1 → 96.1 ms (0.99)
tpcds_q82 31.7 → 30.7 ms (0.97) 54.4 → 54.1 ms (0.99) 28.6 → 28.6 ms (1.00) 45.8 → 45.1 ms (0.99)
tpcds_q83 42.4 → 42.4 ms (1.00) 65.3 → 64.7 ms (0.99) 22.2 → 21.7 ms (0.97) 28.2 → 28.2 ms (1.00)
tpcds_q84 13.1 → 12.7 ms (0.97) 35.2 → 35.8 ms (1.01) 13.6 → 13.4 ms (0.99) 22.7 → 22.5 ms (0.99)
tpcds_q85 60.7 → 60.5 ms (1.00) 106 → 103 ms (0.98) 37.3 → 36.0 ms (0.97) 51.8 → 50.8 ms (0.98)
tpcds_q86 18.5 → 18.4 ms (1.00) 21.8 → 21.7 ms (0.99) 17.7 → 16.4 ms (0.93) 25.0 → 25.3 ms (1.01)
tpcds_q87 80.8 → 80.1 ms (0.99) 121 → 121 ms (1.00) 81.1 → 74.0 ms (0.91) 85.4 → 87.7 ms (1.03)
tpcds_q88 65.8 → 66.7 ms (1.01) 85.5 → 84.8 ms (0.99) 40.4 → 39.0 ms (0.97) 107 → 102 ms (0.96)
tpcds_q89 28.5 → 28.6 ms (1.00) 39.1 → 40.3 ms (1.03) 26.2 → 26.9 ms (1.03) 37.3 → 36.8 ms (0.99)
tpcds_q90 13.0 → 13.0 ms (1.00) 17.6 → 17.0 ms (0.97) 10.5 → 9.76 ms (0.93) 15.3 → 15.5 ms (1.01)
tpcds_q91 19.6 → 19.7 ms (1.01) 42.4 → 42.5 ms (1.00) 19.5 → 19.2 ms (0.99) 25.8 → 25.0 ms (0.97)
tpcds_q92 15.4 → 15.0 ms (0.97) 28.8 → 28.7 ms (1.00) 13.5 → 13.6 ms (1.01) 21.9 → 22.2 ms (1.01)
tpcds_q93 55.1 → 54.5 ms (0.99) 61.7 → 62.3 ms (1.01) 24.9 → 25.2 ms (1.02) 36.3 → 37.0 ms (1.02)
tpcds_q94 21.6 → 21.3 ms (0.99) 38.9 → 38.9 ms (1.00) 32.0 → 31.8 ms (1.00) 39.5 → 38.0 ms (0.96)
tpcds_q95 95.5 → 98.7 ms (1.03) 128 → 127 ms (1.00) 95.3 → 93.9 ms (0.99) 81.7 → 82.7 ms (1.01)
tpcds_q96 13.3 → 13.0 ms (0.98) 21.0 → 20.6 ms (0.98) 13.9 → 13.7 ms (0.98) 17.2 → 17.2 ms (1.00)
tpcds_q97 39.8 → 39.6 ms (1.00) 54.5 → 54.5 ms (1.00) 64.2 → 62.1 ms (0.97) 73.3 → 71.7 ms (0.98)
tpcds_q98 23.1 → 22.7 ms (0.98) 61.6 → 61.4 ms (1.00) 28.8 → 28.8 ms (1.00) 54.8 → 54.5 ms (0.99)
tpcds_q99 30.0 → 29.7 ms (0.99) 42.8 → 42.8 ms (1.00) 23.0 → 24.3 ms (1.06) 24.9 → 24.3 ms (0.98)
Cold runs (first iteration): geomean 0.990x ➖ · 99 queries × 4 targets (5↑ 3↓)
query datafusion:vortex [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:parquet
geomean 87.8 → 86.8 ms (0.99) 116 → 114 ms (0.99) 60.8 → 60.0 ms (0.99) 68.9 → 68.6 ms (1.00)
tpcds_q01 60.2 → 59.3 ms (0.98) 81.3 → 78.5 ms (0.97) 44.4 → 45.3 ms (1.02) 50.4 → 50.1 ms (0.99)
tpcds_q02 76.3 → 75.7 ms (0.99) 90.8 → 92.2 ms (1.02) 43.2 → 42.3 ms (0.98) 52.3 → 47.5 ms (0.91)
tpcds_q03 48.8 → 47.3 ms (0.97) 48.4 → 50.4 ms (1.04) 31.1 → 30.6 ms (0.98) 36.1 → 37.7 ms (1.04)
tpcds_q04 358 → 352 ms (0.98) 708 → 704 ms (0.99) 152 → 153 ms (1.00) 218 → 233 ms (1.07)
tpcds_q05 103 → 113 ms (1.09) 114 → 115 ms (1.01) 82.8 → 71.1 ms (0.86) 🟢 87.6 → 89.0 ms (1.02)
tpcds_q06 70.7 → 70.9 ms (1.00) 117 → 116 ms (0.99) 80.3 → 78.6 ms (0.98) 86.9 → 87.5 ms (1.01)
tpcds_q07 79.7 → 77.8 ms (0.98) 135 → 133 ms (0.99) 55.1 → 52.9 ms (0.96) 76.7 → 76.6 ms (1.00)
tpcds_q08 55.6 → 53.8 ms (0.97) 75.2 → 74.5 ms (0.99) 43.2 → 43.8 ms (1.01) 47.8 → 44.8 ms (0.94)
tpcds_q09 77.1 → 77.5 ms (1.00) 105 → 103 ms (0.98) 80.5 → 78.7 ms (0.98) 97.1 → 94.3 ms (0.97)
tpcds_q10 67.6 → 72.1 ms (1.07) 76.4 → 78.4 ms (1.03) 59.2 → 59.9 ms (1.01) 63.2 → 62.0 ms (0.98)
tpcds_q11 216 → 216 ms (1.00) 453 → 454 ms (1.00) 110 → 113 ms (1.02) 167 → 171 ms (1.03)
tpcds_q12 38.5 → 36.2 ms (0.94) 57.5 → 56.8 ms (0.99) 30.1 → 31.1 ms (1.03) 47.1 → 48.4 ms (1.03)
tpcds_q13 84.0 → 85.1 ms (1.01) 131 → 131 ms (1.00) 68.3 → 70.1 ms (1.03) 89.0 → 86.4 ms (0.97)
tpcds_q14 284 → 263 ms (0.93) 419 → 444 ms (1.06) 146 → 144 ms (0.99) 199 → 201 ms (1.01)
tpcds_q15 60.0 → 59.5 ms (0.99) 72.7 → 69.7 ms (0.96) 74.1 → 76.5 ms (1.03) 68.6 → 68.7 ms (1.00)
tpcds_q16 42.6 → 42.1 ms (0.99) 63.1 → 62.5 ms (0.99) 52.8 → 51.0 ms (0.97) 47.0 → 46.5 ms (0.99)
tpcds_q17 89.1 → 96.8 ms (1.09) 177 → 169 ms (0.96) 80.3 → 81.6 ms (1.02) 89.9 → 91.2 ms (1.01)
tpcds_q18 93.1 → 92.4 ms (0.99) 145 → 141 ms (0.98) 105 → 102 ms (0.97) 98.6 → 96.5 ms (0.98)
tpcds_q19 54.3 → 54.7 ms (1.01) 80.4 → 77.4 ms (0.96) 65.3 → 66.9 ms (1.02) 62.0 → 59.8 ms (0.96)
tpcds_q20 42.0 → 39.6 ms (0.94) 55.3 → 54.2 ms (0.98) 37.0 → 37.0 ms (1.00) 48.1 → 50.4 ms (1.05)
tpcds_q21 63.6 → 70.0 ms (1.10) 🔴 74.5 → 72.6 ms (0.97) 55.2 → 57.3 ms (1.04) 30.2 → 30.9 ms (1.02)
tpcds_q22 478 → 467 ms (0.98) 477 → 476 ms (1.00) 169 → 168 ms (0.99) 164 → 166 ms (1.01)
tpcds_q23 351 → 344 ms (0.98) 405 → 416 ms (1.03) 202 → 204 ms (1.01) 194 → 194 ms (1.00)
tpcds_q24 160 → 157 ms (0.98) 275 → 284 ms (1.03) 73.8 → 72.8 ms (0.99) 83.5 → 81.5 ms (0.98)
tpcds_q25 95.6 → 105 ms (1.10) 183 → 187 ms (1.02) 75.1 → 74.4 ms (0.99) 61.2 → 61.3 ms (1.00)
tpcds_q26 61.7 → 60.3 ms (0.98) 77.4 → 76.7 ms (0.99) 49.3 → 40.3 ms (0.82) 🟢 52.3 → 51.6 ms (0.99)
tpcds_q27 107 → 108 ms (1.01) 237 → 246 ms (1.04) 54.2 → 54.8 ms (1.01) 75.7 → 75.5 ms (1.00)
tpcds_q28 91.9 → 88.3 ms (0.96) 120 → 112 ms (0.94) 66.5 → 62.9 ms (0.95) 111 → 107 ms (0.96)
tpcds_q29 106 → 101 ms (0.95) 179 → 172 ms (0.96) 75.1 → 74.1 ms (0.99) 98.0 → 93.8 ms (0.96)
tpcds_q30 79.0 → 80.1 ms (1.01) 123 → 117 ms (0.96) 51.0 → 50.0 ms (0.98) 80.2 → 81.3 ms (1.01)
tpcds_q31 239 → 232 ms (0.97) 351 → 343 ms (0.98) 55.1 → 55.4 ms (1.00) 116 → 108 ms (0.94)
tpcds_q32 33.7 → 32.0 ms (0.95) 44.6 → 41.8 ms (0.94) 30.5 → 30.9 ms (1.02) 26.5 → 26.3 ms (0.99)
tpcds_q33 70.6 → 67.6 ms (0.96) 76.6 → 76.7 ms (1.00) 57.6 → 56.7 ms (0.98) 73.5 → 75.4 ms (1.03)
tpcds_q34 70.9 → 69.0 ms (0.97) 85.4 → 84.4 ms (0.99) 40.5 → 41.3 ms (1.02) 42.6 → 39.2 ms (0.92)
tpcds_q35 128 → 122 ms (0.96) 152 → 152 ms (1.00) 89.7 → 89.2 ms (1.00) 86.4 → 86.2 ms (1.00)
tpcds_q36 103 → 101 ms (0.98) 129 → 125 ms (0.97) 50.3 → 50.5 ms (1.00) 113 → 112 ms (0.99)
tpcds_q37 44.6 → 45.4 ms (1.02) 68.1 → 65.6 ms (0.96) 69.0 → 66.5 ms (0.96) 46.1 → 47.3 ms (1.02)
tpcds_q38 108 → 107 ms (1.00) 144 → 145 ms (1.00) 72.9 → 72.1 ms (0.99) 83.9 → 79.4 ms (0.95)
tpcds_q39 848 → 838 ms (0.99) 867 → 853 ms (0.98) 83.3 → 78.4 ms (0.94) 67.3 → 67.1 ms (1.00)
tpcds_q40 49.0 → 48.5 ms (0.99) 57.8 → 53.8 ms (0.93) 35.2 → 34.8 ms (0.99) 36.2 → 35.2 ms (0.97)
tpcds_q41 33.7 → 31.1 ms (0.92) 42.9 → 41.8 ms (0.98) 26.4 → 27.1 ms (1.03) 26.9 → 26.7 ms (0.99)
tpcds_q42 39.8 → 40.4 ms (1.01) 47.5 → 48.8 ms (1.03) 31.7 → 32.9 ms (1.04) 39.1 → 39.6 ms (1.01)
tpcds_q43 41.4 → 40.2 ms (0.97) 49.9 → 47.3 ms (0.95) 37.4 → 36.7 ms (0.98) 36.2 → 38.4 ms (1.06)
tpcds_q44 72.5 → 73.2 ms (1.01) 93.2 → 89.5 ms (0.96) 48.9 → 47.6 ms (0.97) 77.1 → 72.5 ms (0.94)
tpcds_q45 65.7 → 65.9 ms (1.00) 94.7 → 87.0 ms (0.92) 61.9 → 58.7 ms (0.95) 67.1 → 69.3 ms (1.03)
tpcds_q46 108 → 110 ms (1.02) 153 → 149 ms (0.97) 76.6 → 72.4 ms (0.95) 92.0 → 86.7 ms (0.94)
tpcds_q47 229 → 226 ms (0.99) 275 → 277 ms (1.01) 102 → 98.8 ms (0.97) 103 → 100 ms (0.98)
tpcds_q48 69.8 → 68.0 ms (0.97) 107 → 107 ms (1.00) 81.3 → 82.1 ms (1.01) 80.8 → 81.3 ms (1.01)
tpcds_q49 120 → 121 ms (1.01) 172 → 164 ms (0.95) 77.7 → 79.1 ms (1.02) 92.0 → 91.4 ms (0.99)
tpcds_q50 84.5 → 84.8 ms (1.00) 123 → 118 ms (0.96) 57.2 → 55.9 ms (0.98) 56.7 → 53.4 ms (0.94)
tpcds_q51 277 → 274 ms (0.99) 281 → 282 ms (1.01) 220 → 222 ms (1.01) 228 → 529 ms (2.32) 🔴
tpcds_q52 40.4 → 41.8 ms (1.04) 47.8 → 47.0 ms (0.98) 33.1 → 32.8 ms (0.99) 37.9 → 35.6 ms (0.94)
tpcds_q53 49.9 → 49.6 ms (0.99) 59.0 → 59.7 ms (1.01) 40.0 → 41.5 ms (1.04) 47.1 → 48.8 ms (1.04)
tpcds_q54 90.7 → 86.6 ms (0.95) 112 → 112 ms (1.00) 66.5 → 65.3 ms (0.98) 64.7 → 64.6 ms (1.00)
tpcds_q55 39.7 → 43.7 ms (1.10) 🔴 46.4 → 46.9 ms (1.01) 33.7 → 31.1 ms (0.92) 37.9 → 35.1 ms (0.93)
tpcds_q56 63.1 → 66.1 ms (1.05) 72.8 → 74.3 ms (1.02) 57.5 → 57.4 ms (1.00) 77.7 → 80.4 ms (1.04)
tpcds_q57 162 → 158 ms (0.98) 195 → 192 ms (0.98) 76.2 → 78.2 ms (1.03) 74.2 → 73.5 ms (0.99)
tpcds_q58 86.0 → 85.7 ms (1.00) 128 → 126 ms (0.99) 61.3 → 61.7 ms (1.01) 69.0 → 67.9 ms (0.98)
tpcds_q59 83.2 → 80.5 ms (0.97) 136 → 129 ms (0.95) 68.5 → 72.6 ms (1.06) 68.8 → 69.5 ms (1.01)
tpcds_q60 67.5 → 64.4 ms (0.95) 75.5 → 76.9 ms (1.02) 58.1 → 59.4 ms (1.02) 95.6 → 91.9 ms (0.96)
tpcds_q61 69.4 → 73.8 ms (1.06) 104 → 99.2 ms (0.95) 55.2 → 54.4 ms (0.99) 94.5 → 92.3 ms (0.98)
tpcds_q62 40.3 → 39.7 ms (0.99) 44.9 → 45.5 ms (1.01) 26.7 → 27.7 ms (1.04) 26.6 → 26.7 ms (1.00)
tpcds_q63 49.9 → 49.0 ms (0.98) 57.9 → 59.3 ms (1.02) 40.4 → 40.0 ms (0.99) 42.6 → 41.6 ms (0.98)
tpcds_q64 394 → 402 ms (1.02) 736 → 745 ms (1.01) 149 → 150 ms (1.01) 150 → 151 ms (1.01)
tpcds_q65 113 → 112 ms (0.99) 137 → 135 ms (0.99) 69.2 → 64.8 ms (0.94) 102 → 107 ms (1.04)
tpcds_q66 79.3 → 74.3 ms (0.94) 102 → 99.7 ms (0.98) 59.9 → 59.0 ms (0.99) 54.9 → 53.0 ms (0.97)
tpcds_q67 522 → 516 ms (0.99) 499 → 494 ms (0.99) 280 → 292 ms (1.04) 304 → 296 ms (0.97)
tpcds_q68 97.8 → 103 ms (1.06) 157 → 157 ms (1.00) 70.4 → 69.7 ms (0.99) 78.1 → 74.6 ms (0.96)
tpcds_q69 66.7 → 63.4 ms (0.95) 77.1 → 79.6 ms (1.03) 61.2 → 59.9 ms (0.98) 75.3 → 77.9 ms (1.03)
tpcds_q70 232 → 226 ms (0.97) 248 → 240 ms (0.97) 43.2 → 43.0 ms (1.00) 54.8 → 54.1 ms (0.99)
tpcds_q71 58.4 → 61.4 ms (1.05) 65.0 → 63.8 ms (0.98) 51.4 → 49.9 ms (0.97) 55.2 → 52.5 ms (0.95)
tpcds_q72 1.62 → 1.61 s (1.00) 1.79 → 1.95 s (1.09) 159 → 156 ms (0.99) 187 → 182 ms (0.97)
tpcds_q73 66.1 → 66.3 ms (1.00) 85.9 → 83.3 ms (0.97) 40.1 → 38.0 ms (0.95) 35.7 → 35.0 ms (0.98)
tpcds_q74 170 → 171 ms (1.01) 325 → 321 ms (0.99) 103 → 98.5 ms (0.95) 222 → 222 ms (1.00)
tpcds_q75 193 → 199 ms (1.03) 239 → 237 ms (0.99) 138 → 138 ms (1.00) 148 → 149 ms (1.01)
tpcds_q76 75.6 → 73.3 ms (0.97) 80.8 → 80.0 ms (0.99) 56.9 → 58.0 ms (1.02) 67.7 → 69.1 ms (1.02)
tpcds_q77 90.5 → 88.7 ms (0.98) 102 → 103 ms (1.00) 62.6 → 60.4 ms (0.97) 69.9 → 69.6 ms (1.00)
tpcds_q78 193 → 193 ms (1.00) 269 → 275 ms (1.02) 161 → 160 ms (0.99) 160 → 165 ms (1.03)
tpcds_q79 84.3 → 80.9 ms (0.96) 115 → 111 ms (0.96) 60.9 → 57.6 ms (0.95) 69.8 → 66.1 ms (0.95)
tpcds_q80 130 → 129 ms (0.99) 197 → 191 ms (0.97) 93.0 → 93.5 ms (1.00) 104 → 98.7 ms (0.95)
tpcds_q81 83.1 → 83.6 ms (1.01) 130 → 128 ms (0.98) 70.7 → 69.6 ms (0.98) 106 → 104 ms (0.98)
tpcds_q82 62.9 → 48.8 ms (0.77) 🟢 71.4 → 71.0 ms (0.99) 70.3 → 72.1 ms (1.03) 53.4 → 52.2 ms (0.98)
tpcds_q83 68.6 → 66.6 ms (0.97) 85.1 → 83.5 ms (0.98) 33.5 → 32.5 ms (0.97) 33.4 → 33.0 ms (0.99)
tpcds_q84 32.5 → 32.0 ms (0.99) 49.6 → 51.7 ms (1.04) 25.3 → 24.5 ms (0.97) 29.9 → 29.3 ms (0.98)
tpcds_q85 91.3 → 88.5 ms (0.97) 133 → 133 ms (1.00) 52.9 → 53.4 ms (1.01) 61.1 → 61.9 ms (1.01)
tpcds_q86 39.6 → 38.3 ms (0.97) 39.6 → 39.0 ms (0.98) 29.6 → 27.7 ms (0.94) 32.0 → 31.8 ms (0.99)
tpcds_q87 109 → 107 ms (0.98) 146 → 141 ms (0.97) 91.0 → 86.3 ms (0.95) 93.5 → 92.3 ms (0.99)
tpcds_q88 105 → 102 ms (0.97) 116 → 105 ms (0.90) 51.5 → 50.6 ms (0.98) 110 → 108 ms (0.99)
tpcds_q89 54.3 → 52.8 ms (0.97) 64.6 → 63.1 ms (0.98) 43.5 → 42.5 ms (0.98) 46.8 → 45.8 ms (0.98)
tpcds_q90 33.3 → 30.7 ms (0.92) 34.3 → 30.3 ms (0.88) 🟢 21.6 → 19.7 ms (0.91) 21.8 → 21.7 ms (1.00)
tpcds_q91 41.6 → 41.5 ms (1.00) 58.9 → 57.5 ms (0.98) 30.2 → 29.7 ms (0.98) 33.5 → 32.8 ms (0.98)
tpcds_q92 37.6 → 36.6 ms (0.97) 47.1 → 47.2 ms (1.00) 25.9 → 26.0 ms (1.00) 29.6 → 32.3 ms (1.09)
tpcds_q93 90.3 → 90.5 ms (1.00) 88.6 → 89.5 ms (1.01) 51.8 → 51.1 ms (0.99) 51.1 → 46.0 ms (0.90)
tpcds_q94 46.6 → 43.9 ms (0.94) 57.4 → 57.4 ms (1.00) 45.5 → 45.5 ms (1.00) 46.7 → 44.5 ms (0.95)
tpcds_q95 126 → 124 ms (0.98) 158 → 144 ms (0.91) 105 → 109 ms (1.04) 86.4 → 88.6 ms (1.03)
tpcds_q96 34.9 → 33.2 ms (0.95) 35.1 → 36.1 ms (1.03) 27.2 → 27.9 ms (1.03) 24.0 → 23.5 ms (0.98)
tpcds_q97 68.9 → 69.6 ms (1.01) 81.4 → 79.6 ms (0.98) 83.4 → 83.3 ms (1.00) 82.4 → 80.1 ms (0.97)
tpcds_q98 48.7 → 47.9 ms (0.98) 83.3 → 83.6 ms (1.00) 47.4 → 44.8 ms (0.95) 67.1 → 68.3 ms (1.02)
tpcds_q99 48.0 → 47.2 ms (0.98) 59.1 → 58.1 ms (0.98) 36.8 → 34.8 ms (0.94) 39.9 → 32.4 ms (0.81) 🟢

No file size changes detected.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on S3 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -4.7%
Engines: DataFusion No clear signal (-4.8%, environment too noisy confidence) · DuckDB No clear signal (-4.6%, environment too noisy confidence)
Vortex (geomean): hot 0.878x ➖ · cold 0.924x ➖
Parquet (geomean): hot 0.921x ➖ · cold 0.920x ➖
Shifts: Parquet (control) -7.9% · Median polish -7.0%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.892x ➖ · 22 queries × 6 targets (9↑ 0↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±30% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet
geomean 288 → 246 ms (0.85) 255 → 239 ms (0.94) 238 → 223 ms (0.94) 360 → 292 ms (0.81) 321 → 293 ms (0.91) 345 → 311 ms (0.90)
tpch_q01 211 → 219 ms (1.03) 241 → 230 ms (0.95) 198 → 142 ms (0.72) 246 → 231 ms (0.94) 265 → 243 ms (0.92) 208 → 180 ms (0.87)
tpch_q02 339 → 334 ms (0.98) 320 → 314 ms (0.98) 282 → 270 ms (0.96) 1.56 s → 394 ms (0.25) 🟢 371 → 385 ms (1.04) 404 → 456 ms (1.13)
tpch_q03 304 → 294 ms (0.97) 478 → 295 ms (0.62) 🟢 229 → 230 ms (1.00) 360 → 356 ms (0.99) 374 → 350 ms (0.94) 364 → 390 ms (1.07)
tpch_q04 163 → 167 ms (1.02) 163 → 144 ms (0.88) 139 → 140 ms (1.01) 232 → 222 ms (0.95) 230 → 217 ms (0.94) 293 → 224 ms (0.76)
tpch_q05 396 → 335 ms (0.85) 352 → 347 ms (0.98) 324 → 327 ms (1.01) 524 → 474 ms (0.90) 532 → 470 ms (0.88) 540 → 501 ms (0.93)
tpch_q06 137 → 113 ms (0.82) 149 → 128 ms (0.86) 211 → 102 ms (0.48) 🟢 196 → 157 ms (0.80) 177 → 179 ms (1.02) 157 → 126 ms (0.81)
tpch_q07 422 → 416 ms (0.99) 413 → 395 ms (0.96) 371 → 319 ms (0.86) 392 → 388 ms (0.99) 426 → 377 ms (0.89) 449 → 382 ms (0.85)
tpch_q08 1.51 s → 414 ms (0.27) 🟢 401 → 394 ms (0.98) 391 → 398 ms (1.02) 540 → 504 ms (0.93) 680 → 510 ms (0.75) 595 → 555 ms (0.93)
tpch_q09 331 → 352 ms (1.06) 323 → 322 ms (1.00) 402 → 393 ms (0.98) 497 → 389 ms (0.78) 399 → 390 ms (0.98) 584 → 519 ms (0.89)
tpch_q10 370 → 343 ms (0.92) 336 → 327 ms (0.97) 307 → 294 ms (0.96) 502 → 488 ms (0.97) 465 → 461 ms (0.99) 579 → 518 ms (0.89)
tpch_q11 299 → 295 ms (0.99) 347 → 283 ms (0.81) 260 → 271 ms (1.04) 289 → 281 ms (0.97) 295 → 283 ms (0.96) 298 → 292 ms (0.98)
tpch_q12 218 → 188 ms (0.86) 187 → 193 ms (1.03) 150 → 170 ms (1.13) 210 → 204 ms (0.97) 210 → 194 ms (0.92) 261 → 228 ms (0.87)
tpch_q13 121 → 110 ms (0.91) 109 → 101 ms (0.92) 275 → 274 ms (0.99) 255 → 188 ms (0.74) 189 → 195 ms (1.03) 394 → 349 ms (0.89)
tpch_q14 180 → 189 ms (1.05) 195 → 198 ms (1.02) 144 → 154 ms (1.07) 223 → 202 ms (0.91) 244 → 211 ms (0.87) 249 → 228 ms (0.92)
tpch_q15 312 → 298 ms (0.95) 346 → 297 ms (0.86) 295 → 295 ms (1.00) 252 → 228 ms (0.90) 256 → 244 ms (0.95) 231 → 203 ms (0.88)
tpch_q16 148 → 142 ms (0.96) 123 → 120 ms (0.98) 105 → 105 ms (1.00) 240 → 210 ms (0.88) 207 → 194 ms (0.94) 279 → 266 ms (0.95)
tpch_q17 284 → 257 ms (0.91) 288 → 281 ms (0.98) 298 → 269 ms (0.90) 323 → 274 ms (0.85) 295 → 284 ms (0.96) 304 → 261 ms (0.86)
tpch_q18 502 → 313 ms (0.62) 🟢 265 → 241 ms (0.91) 305 → 301 ms (0.99) 620 → 298 ms (0.48) 🟢 325 → 305 ms (0.94) 367 → 324 ms (0.88)
tpch_q19 231 → 234 ms (1.01) 240 → 264 ms (1.10) 168 → 153 ms (0.91) 313 → 283 ms (0.90) 483 → 282 ms (0.58) 🟢 295 → 218 ms (0.74)
tpch_q20 449 → 251 ms (0.56) 🟢 249 → 239 ms (0.96) 242 → 231 ms (0.96) 544 → 328 ms (0.60) 🟢 348 → 318 ms (0.91) 465 → 430 ms (0.92)
tpch_q21 525 → 415 ms (0.79) 392 → 384 ms (0.98) 312 → 292 ms (0.94) 580 → 516 ms (0.89) 515 → 457 ms (0.89) 458 → 428 ms (0.94)
tpch_q22 174 → 156 ms (0.90) 154 → 155 ms (1.01) 157 → 159 ms (1.01) 243 → 220 ms (0.90) 251 → 235 ms (0.94) 281 → 281 ms (1.00)
Cold runs (first iteration): geomean 0.923x ➖ · 22 queries × 6 targets (7↑ 2↓)
query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet
geomean 504 → 456 ms (0.91) 467 → 444 ms (0.95) 420 → 394 ms (0.94) 395 → 341 ms (0.86) 356 → 351 ms (0.98) 394 → 355 ms (0.90)
tpch_q01 664 → 766 ms (1.15) 659 → 665 ms (1.01) 622 → 387 ms (0.62) 🟢 247 → 259 ms (1.05) 281 → 201 ms (0.71) 293 → 212 ms (0.72)
tpch_q02 641 → 591 ms (0.92) 635 → 781 ms (1.23) 558 → 449 ms (0.80) 832 → 427 ms (0.51) 🟢 466 → 467 ms (1.00) 401 → 495 ms (1.24)
tpch_q03 837 → 685 ms (0.82) 870 → 670 ms (0.77) 651 → 614 ms (0.94) 477 → 424 ms (0.89) 410 → 383 ms (0.93) 438 → 463 ms (1.06)
tpch_q04 577 → 400 ms (0.69) 🟢 381 → 321 ms (0.84) 292 → 259 ms (0.89) 288 → 280 ms (0.97) 256 → 242 ms (0.94) 346 → 303 ms (0.88)
tpch_q05 668 → 519 ms (0.78) 735 → 504 ms (0.69) 🟢 467 → 490 ms (1.05) 589 → 545 ms (0.93) 563 → 582 ms (1.03) 582 → 522 ms (0.90)
tpch_q06 280 → 262 ms (0.93) 331 → 311 ms (0.94) 298 → 236 ms (0.79) 195 → 183 ms (0.94) 225 → 202 ms (0.90) 208 → 218 ms (1.05)
tpch_q07 566 → 749 ms (1.32) 🔴 566 → 534 ms (0.94) 558 → 444 ms (0.80) 454 → 424 ms (0.93) 493 → 425 ms (0.86) 524 → 423 ms (0.81)
tpch_q08 730 → 546 ms (0.75) 603 → 590 ms (0.98) 576 → 536 ms (0.93) 603 → 574 ms (0.95) 578 → 537 ms (0.93) 661 → 625 ms (0.95)
tpch_q09 554 → 516 ms (0.93) 504 → 496 ms (0.98) 501 → 536 ms (1.07) 636 → 446 ms (0.70) 425 → 464 ms (1.09) 582 → 563 ms (0.97)
tpch_q10 498 → 491 ms (0.99) 470 → 486 ms (1.03) 455 → 463 ms (1.02) 553 → 514 ms (0.93) 494 → 511 ms (1.03) 598 → 565 ms (0.95)
tpch_q11 492 → 495 ms (1.01) 524 → 420 ms (0.80) 375 → 380 ms (1.01) 326 → 324 ms (0.99) 329 → 309 ms (0.94) 317 → 290 ms (0.91)
tpch_q12 342 → 387 ms (1.13) 359 → 375 ms (1.05) 303 → 357 ms (1.18) 276 → 262 ms (0.95) 234 → 226 ms (0.97) 285 → 283 ms (0.99)
tpch_q13 299 → 261 ms (0.87) 281 → 282 ms (1.01) 512 → 479 ms (0.94) 271 → 244 ms (0.90) 264 → 241 ms (0.91) 552 → 398 ms (0.72)
tpch_q14 318 → 338 ms (1.06) 343 → 332 ms (0.97) 283 → 324 ms (1.14) 291 → 249 ms (0.85) 277 → 360 ms (1.30) 303 → 266 ms (0.88)
tpch_q15 489 → 508 ms (1.04) 509 → 521 ms (1.02) 551 → 520 ms (0.94) 265 → 273 ms (1.03) 273 → 274 ms (1.00) 254 → 223 ms (0.88)
tpch_q16 326 → 292 ms (0.90) 309 → 286 ms (0.93) 212 → 229 ms (1.08) 291 → 263 ms (0.90) 243 → 231 ms (0.95) 303 → 275 ms (0.91)
tpch_q17 462 → 401 ms (0.87) 416 → 427 ms (1.03) 469 → 419 ms (0.89) 378 → 327 ms (0.87) 359 → 297 ms (0.83) 355 → 300 ms (0.85)
tpch_q18 1.05 s → 476 ms (0.45) 🟢 496 → 412 ms (0.83) 461 → 444 ms (0.96) 515 → 308 ms (0.60) 🟢 341 → 516 ms (1.51) 🔴 357 → 353 ms (0.99)
tpch_q19 366 → 385 ms (1.05) 366 → 439 ms (1.20) 301 → 276 ms (0.92) 329 → 315 ms (0.96) 407 → 497 ms (1.22) 355 → 246 ms (0.69) 🟢
tpch_q20 491 → 403 ms (0.82) 437 → 399 ms (0.91) 401 → 345 ms (0.86) 592 → 422 ms (0.71) 409 → 410 ms (1.00) 538 → 451 ms (0.84)
tpch_q21 676 → 629 ms (0.93) 631 → 543 ms (0.86) 440 → 431 ms (0.98) 676 → 535 ms (0.79) 631 → 554 ms (0.88) 557 → 487 ms (0.88)
tpch_q22 426 → 383 ms (0.90) 330 → 347 ms (1.05) 344 → 349 ms (1.02) 298 → 265 ms (0.89) 276 → 265 ms (0.96) 309 → 305 ms (0.99)

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=10 on NVME 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +54.0%
Engines: DataFusion No clear signal (+43.1%, environment too noisy confidence) · DuckDB No clear signal (+64.6%, environment too noisy confidence)
Vortex (geomean): hot 1.001x ➖ · cold 1.041x ➖
Parquet (geomean): hot 0.661x ✅ · cold 0.718x ✅
Shifts: Parquet (control) -33.9% · Median polish +0.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.899x 🟢 · 22 queries × 7 targets (39↑ 7↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 60.9 → 60.7 ms (1.00) 68.0 → 67.7 ms (1.00) 143 → 99.9 ms (0.70) 🟢 45.3 → 45.5 ms (1.01) 51.3 → 51.5 ms (1.00) 112 → 70.1 ms (0.63) 🟢 39.6 → 42.9 ms (1.08)
tpch_q01 94.4 → 95.1 ms (1.01) 101 → 100 ms (1.00) 86.8 → 88.4 ms (1.02) 36.5 → 36.7 ms (1.01) 45.4 → 45.6 ms (1.00) 67.8 → 51.6 ms (0.76) 🟢 23.8 → 24.3 ms (1.02)
tpch_q02 65.3 → 64.6 ms (0.99) 65.6 → 65.5 ms (1.00) 173 → 96.8 ms (0.56) 🟢 28.2 → 28.9 ms (1.02) 31.7 → 31.5 ms (1.00) 87.2 → 52.4 ms (0.60) 🟢 20.7 → 22.2 ms (1.07)
tpch_q03 62.7 → 63.6 ms (1.01) 69.4 → 68.5 ms (0.99) 135 → 110 ms (0.81) 🟢 52.2 → 53.2 ms (1.02) 58.6 → 58.8 ms (1.00) 111 → 74.4 ms (0.67) 🟢 45.8 → 48.8 ms (1.07)
tpch_q04 34.6 → 34.3 ms (0.99) 39.2 → 38.4 ms (0.98) 57.6 → 47.3 ms (0.82) 🟢 49.3 → 48.8 ms (0.99) 54.5 → 54.4 ms (1.00) 77.3 → 73.0 ms (0.94) 56.6 → 58.1 ms (1.03)
tpch_q05 119 → 121 ms (1.01) 120 → 121 ms (1.01) 167 → 155 ms (0.93) 70.4 → 70.3 ms (1.00) 73.1 → 73.7 ms (1.01) 123 → 91.9 ms (0.75) 🟢 59.1 → 65.0 ms (1.10)
tpch_q06 10.2 → 9.48 ms (0.93) 15.5 → 15.6 ms (1.01) 36.5 → 31.7 ms (0.87) 🟢 15.0 → 15.5 ms (1.03) 20.0 → 19.5 ms (0.97) 35.4 → 29.8 ms (0.84) 🟢 14.4 → 17.4 ms (1.20) 🔴
tpch_q07 96.2 → 96.7 ms (1.01) 106 → 104 ms (0.98) 214 → 184 ms (0.86) 🟢 53.0 → 55.1 ms (1.04) 64.4 → 64.1 ms (1.00) 120 → 83.3 ms (0.69) 🟢 42.6 → 47.7 ms (1.12) 🔴
tpch_q08 144 → 141 ms (0.99) 148 → 146 ms (0.99) 219 → 186 ms (0.85) 🟢 87.6 → 86.5 ms (0.99) 90.4 → 90.1 ms (1.00) 149 → 116 ms (0.78) 🟢 66.2 → 70.7 ms (1.07)
tpch_q09 185 → 185 ms (1.00) 187 → 189 ms (1.01) 309 → 210 ms (0.68) 🟢 106 → 105 ms (0.99) 112 → 114 ms (1.02) 238 → 135 ms (0.57) 🟢 77.6 → 100 ms (1.29) 🔴
tpch_q10 70.2 → 70.6 ms (1.01) 77.0 → 77.4 ms (1.00) 313 → 147 ms (0.47) 🟢 96.0 → 96.8 ms (1.01) 102 → 103 ms (1.00) 313 → 132 ms (0.42) 🟢 91.9 → 90.1 ms (0.98)
tpch_q11 46.3 → 45.9 ms (0.99) 46.1 → 45.7 ms (0.99) 127 → 54.1 ms (0.43) 🟢 13.3 → 13.9 ms (1.05) 14.5 → 14.6 ms (1.01) 43.6 → 22.0 ms (0.50) 🟢 9.73 → 10.4 ms (1.07)
tpch_q12 36.2 → 36.0 ms (0.99) 48.8 → 49.1 ms (1.01) 82.5 → 58.3 ms (0.71) 🟢 46.4 → 45.3 ms (0.98) 58.6 → 59.8 ms (1.02) 71.6 → 58.5 ms (0.82) 🟢 39.5 → 41.7 ms (1.06)
tpch_q13 50.0 → 49.4 ms (0.99) 57.3 → 57.1 ms (1.00) 211 → 82.9 ms (0.39) 🟢 70.2 → 71.3 ms (1.01) 77.0 → 76.6 ms (0.99) 294 → 103 ms (0.35) 🟢 100 → 104 ms (1.04)
tpch_q14 21.0 → 20.9 ms (0.99) 25.4 → 25.6 ms (1.01) 87.0 → 55.6 ms (0.64) 🟢 27.8 → 27.5 ms (0.99) 33.1 → 32.8 ms (0.99) 100 → 56.9 ms (0.57) 🟢 32.3 → 34.3 ms (1.06)
tpch_q15 32.8 → 33.0 ms (1.01) 47.1 → 47.6 ms (1.01) 107 → 92.1 ms (0.86) 🟢 24.0 → 23.8 ms (0.99) 31.3 → 31.5 ms (1.01) 58.1 → 50.7 ms (0.87) 🟢 24.7 → 28.4 ms (1.15) 🔴
tpch_q16 37.1 → 36.8 ms (0.99) 38.1 → 37.5 ms (0.99) 97.0 → 53.0 ms (0.55) 🟢 49.4 → 49.1 ms (0.99) 50.4 → 50.1 ms (0.99) 99.6 → 54.1 ms (0.54) 🟢 44.9 → 46.6 ms (1.04)
tpch_q17 180 → 182 ms (1.01) 180 → 179 ms (0.99) 218 → 206 ms (0.95) 39.0 → 38.5 ms (0.99) 40.5 → 40.4 ms (1.00) 82.1 → 60.8 ms (0.74) 🟢 26.2 → 30.1 ms (1.15) 🔴
tpch_q18 203 → 203 ms (1.00) 211 → 209 ms (0.99) 268 → 211 ms (0.79) 🟢 106 → 108 ms (1.02) 104 → 109 ms (1.04) 155 → 103 ms (0.66) 🟢 71.5 → 73.4 ms (1.03)
tpch_q19 28.4 → 29.0 ms (1.02) 35.9 → 34.4 ms (0.96) 109 → 88.9 ms (0.82) 🟢 27.3 → 26.6 ms (0.97) 34.5 → 34.7 ms (1.01) 96.6 → 62.1 ms (0.64) 🟢 29.4 → 32.6 ms (1.11) 🔴
tpch_q20 60.5 → 61.3 ms (1.01) 67.4 → 68.3 ms (1.01) 186 → 126 ms (0.68) 🟢 40.8 → 41.4 ms (1.01) 51.9 → 52.1 ms (1.00) 147 → 71.3 ms (0.48) 🟢 37.6 → 42.5 ms (1.13) 🔴
tpch_q21 179 → 182 ms (1.01) 188 → 187 ms (1.00) 244 → 243 ms (1.00) 92.1 → 94.7 ms (1.03) 104 → 106 ms (1.02) 254 → 204 ms (0.80) 🟢 62.7 → 66.7 ms (1.06)
tpch_q22 25.7 → 25.3 ms (0.99) 26.7 → 27.4 ms (1.02) 150 → 50.7 ms (0.34) 🟢 33.4 → 33.3 ms (1.00) 35.3 → 35.4 ms (1.00) 168 → 50.6 ms (0.30) 🟢 35.2 → 37.3 ms (1.06)
Cold runs (first iteration): geomean 0.949x ➖ · 22 queries × 7 targets (31↑ 32↓)
query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 112 → 114 ms (1.02) 114 → 116 ms (1.02) 189 → 154 ms (0.81) 🟢 76.8 → 82.0 ms (1.07) 79.2 → 84.0 ms (1.06) 137 → 87.0 ms (0.63) 🟢 65.3 → 75.0 ms (1.15) 🔴
tpch_q01 152 → 154 ms (1.01) 146 → 155 ms (1.06) 120 → 136 ms (1.13) 🔴 65.7 → 75.7 ms (1.15) 🔴 69.9 → 83.0 ms (1.19) 🔴 89.2 → 62.9 ms (0.70) 🟢 51.8 → 56.9 ms (1.10)
tpch_q02 94.5 → 97.7 ms (1.03) 95.7 → 93.6 ms (0.98) 198 → 121 ms (0.61) 🟢 44.1 → 44.8 ms (1.02) 46.4 → 48.0 ms (1.03) 97.3 → 61.6 ms (0.63) 🟢 31.5 → 35.2 ms (1.12) 🔴
tpch_q03 108 → 115 ms (1.07) 111 → 116 ms (1.04) 173 → 156 ms (0.90) 87.4 → 97.2 ms (1.11) 🔴 91.0 → 96.8 ms (1.06) 140 → 90.1 ms (0.64) 🟢 75.3 → 82.5 ms (1.10)
tpch_q04 63.8 → 66.3 ms (1.04) 64.3 → 64.9 ms (1.01) 86.7 → 86.4 ms (1.00) 77.6 → 81.1 ms (1.05) 80.0 → 84.0 ms (1.05) 98.2 → 90.0 ms (0.92) 78.8 → 87.3 ms (1.11) 🔴
tpch_q05 172 → 175 ms (1.02) 175 → 172 ms (0.98) 217 → 207 ms (0.96) 106 → 106 ms (1.00) 103 → 109 ms (1.06) 155 → 116 ms (0.75) 🟢 91.3 → 102 ms (1.11) 🔴
tpch_q06 51.4 → 54.7 ms (1.06) 49.4 → 52.8 ms (1.07) 71.9 → 111 ms (1.55) 🔴 44.7 → 51.6 ms (1.16) 🔴 45.4 → 52.3 ms (1.15) 🔴 56.9 → 45.6 ms (0.80) 🟢 41.0 → 49.5 ms (1.21) 🔴
tpch_q07 155 → 161 ms (1.04) 158 → 165 ms (1.04) 266 → 242 ms (0.91) 96.0 → 104 ms (1.09) 98.1 → 107 ms (1.09) 154 → 104 ms (0.68) 🟢 78.0 → 89.9 ms (1.15) 🔴
tpch_q08 213 → 205 ms (0.96) 214 → 203 ms (0.95) 281 → 246 ms (0.87) 🟢 121 → 124 ms (1.03) 126 → 126 ms (1.00) 188 → 132 ms (0.70) 🟢 105 → 121 ms (1.15) 🔴
tpch_q09 264 → 270 ms (1.02) 264 → 262 ms (0.99) 370 → 277 ms (0.75) 🟢 146 → 161 ms (1.10) 🔴 149 → 154 ms (1.04) 276 → 154 ms (0.56) 🟢 111 → 156 ms (1.40) 🔴
tpch_q10 125 → 125 ms (1.00) 128 → 133 ms (1.04) 360 → 202 ms (0.56) 🟢 130 → 146 ms (1.12) 🔴 134 → 150 ms (1.12) 🔴 336 → 157 ms (0.47) 🟢 117 → 129 ms (1.10) 🔴
tpch_q11 70.7 → 72.6 ms (1.03) 72.1 → 71.0 ms (0.98) 147 → 80.3 ms (0.55) 🟢 28.5 → 30.5 ms (1.07) 29.9 → 29.8 ms (0.99) 50.2 → 33.3 ms (0.66) 🟢 16.6 → 17.9 ms (1.08)
tpch_q12 72.7 → 76.8 ms (1.06) 75.9 → 87.3 ms (1.15) 🔴 116 → 137 ms (1.19) 🔴 76.3 → 78.3 ms (1.03) 80.9 → 86.7 ms (1.07) 90.8 → 76.6 ms (0.84) 🟢 69.6 → 76.0 ms (1.09)
tpch_q13 74.0 → 78.5 ms (1.06) 90.2 → 87.8 ms (0.97) 238 → 114 ms (0.48) 🟢 92.0 → 99.5 ms (1.08) 98.5 → 103 ms (1.05) 311 → 118 ms (0.38) 🟢 111 → 117 ms (1.06)
tpch_q14 82.6 → 84.3 ms (1.02) 92.1 → 91.2 ms (0.99) 183 → 110 ms (0.60) 🟢 68.1 → 72.8 ms (1.07) 65.1 → 71.8 ms (1.10) 🔴 135 → 74.2 ms (0.55) 🟢 66.0 → 73.0 ms (1.11) 🔴
tpch_q15 91.6 → 89.1 ms (0.97) 96.4 → 100 ms (1.04) 152 → 139 ms (0.92) 56.0 → 63.4 ms (1.13) 🔴 61.4 → 68.2 ms (1.11) 🔴 84.0 → 72.2 ms (0.86) 🟢 54.2 → 67.2 ms (1.24) 🔴
tpch_q16 65.6 → 66.2 ms (1.01) 62.9 → 64.8 ms (1.03) 115 → 79.4 ms (0.69) 🟢 59.7 → 58.2 ms (0.97) 63.5 → 59.6 ms (0.94) 106 → 56.7 ms (0.53) 🟢 52.2 → 53.7 ms (1.03)
tpch_q17 246 → 230 ms (0.93) 221 → 224 ms (1.01) 262 → 255 ms (0.97) 65.4 → 69.8 ms (1.07) 67.2 → 68.1 ms (1.01) 117 → 80.3 ms (0.69) 🟢 55.3 → 67.8 ms (1.22) 🔴
tpch_q18 263 → 268 ms (1.02) 268 → 265 ms (0.99) 310 → 285 ms (0.92) 119 → 120 ms (1.01) 118 → 122 ms (1.04) 166 → 122 ms (0.73) 🟢 88.6 → 97.5 ms (1.10) 🔴
tpch_q19 80.3 → 79.6 ms (0.99) 82.0 → 83.7 ms (1.02) 166 → 158 ms (0.95) 70.9 → 79.2 ms (1.12) 🔴 71.6 → 75.6 ms (1.06) 128 → 77.1 ms (0.60) 🟢 63.3 → 84.4 ms (1.33) 🔴
tpch_q20 110 → 112 ms (1.02) 111 → 115 ms (1.03) 239 → 179 ms (0.75) 🟢 87.5 → 94.4 ms (1.08) 89.2 → 97.7 ms (1.09) 184 → 88.0 ms (0.48) 🟢 74.8 → 91.5 ms (1.22) 🔴
tpch_q21 228 → 230 ms (1.01) 236 → 233 ms (0.99) 307 → 325 ms (1.06) 136 → 140 ms (1.03) 144 → 149 ms (1.04) 272 → 229 ms (0.84) 🟢 89.3 → 104 ms (1.16) 🔴
tpch_q22 50.5 → 52.6 ms (1.04) 53.7 → 55.0 ms (1.03) 170 → 75.6 ms (0.45) 🟢 49.2 → 51.0 ms (1.04) 49.8 → 52.3 ms (1.05) 179 → 61.2 ms (0.34) 🟢 44.6 → 50.5 ms (1.13) 🔴

File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
File Scale Format Base HEAD Change %
region_0001.vortex 10.0 vortex-file-compressed 6.33 KB 6.36 KB +32 B +0.5%

Totals:

  • vortex-compact: 1.92 GB → 1.92 GB (0.0%)
  • vortex-file-compressed: 2.17 GB → 2.17 GB (+0.0%)

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: PolarSignals Profiling 📖

Commits: PR aff33cb1 vs base 967bcb67
Vortex (geomean): hot 1.001x ➖ · cold 1.030x ➖


Hot runs: geomean 1.001x ➖ · 13 queries × 1 target (0↑ 0↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v1]
geomean 25.1 → 25.1 ms (1.00)
polarsignals_q00 81.5 → 81.7 ms (1.00)
polarsignals_q01 89.2 → 89.7 ms (1.01)
polarsignals_q02 14.6 → 14.2 ms (0.97)
polarsignals_q03 90.3 → 92.3 ms (1.02)
polarsignals_q04 9.30 → 9.20 ms (0.99)
polarsignals_q05 11.3 → 11.6 ms (1.02)
polarsignals_q06 22.9 → 22.9 ms (1.00)
polarsignals_q07 10.5 → 10.9 ms (1.03)
polarsignals_q08 133 → 134 ms (1.01)
polarsignals_q09 10.1 → 9.85 ms (0.98)
polarsignals_q10 17.4 → 17.4 ms (1.00)
polarsignals_q11 16.1 → 16.1 ms (1.00)
polarsignals_q12 16.8 → 16.6 ms (0.99)
Cold runs (first iteration): geomean 1.030x ➖ · 13 queries × 1 target (0↑ 1↓)
query datafusion:vortex [file, v1]
geomean 67.8 → 69.8 ms (1.03)
polarsignals_q00 125 → 124 ms (0.99)
polarsignals_q01 146 → 149 ms (1.02)
polarsignals_q02 48.0 → 49.2 ms (1.02)
polarsignals_q03 154 → 150 ms (0.97)
polarsignals_q04 41.1 → 42.3 ms (1.03)
polarsignals_q05 45.3 → 46.4 ms (1.03)
polarsignals_q06 56.3 → 56.5 ms (1.00)
polarsignals_q07 44.6 → 43.6 ms (0.98)
polarsignals_q08 195 → 259 ms (1.33) 🔴
polarsignals_q09 40.8 → 41.9 ms (1.03)
polarsignals_q10 52.1 → 53.0 ms (1.02)
polarsignals_q11 50.5 → 49.2 ms (0.97)
polarsignals_q12 48.2 → 50.0 ms (1.04)

No file size changes detected.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb S3 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -1.8%
Engines: DataFusion No clear signal (-8.9%, environment too noisy confidence) · DuckDB No clear signal (+5.9%, environment too noisy confidence)
Vortex (geomean): hot 0.956x ➖ · cold 0.884x ➖
Parquet (geomean): hot 0.974x ➖ · cold 0.963x ➖
Shifts: Parquet (control) -2.6% · Median polish -2.4%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.962x ➖ · 9 queries × 6 targets (2↑ 1↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±30% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet
geomean 308 → 282 ms (0.92) 337 → 318 ms (0.94) 688 → 701 ms (1.02) 433 → 434 ms (1.00) 463 → 447 ms (0.96) 583 → 542 ms (0.93)
fineweb_q00 42.3 → 46.5 ms (1.10) 35.9 → 38.2 ms (1.06) 347 → 403 ms (1.16) 67.0 → 60.8 ms (0.91) 67.2 → 66.0 ms (0.98) 308 → 307 ms (1.00)
fineweb_q01 656 → 355 ms (0.54) 🟢 352 → 450 ms (1.28) 737 → 715 ms (0.97) 269 → 273 ms (1.01) 338 → 312 ms (0.92) 538 → 527 ms (0.98)
fineweb_q02 218 → 256 ms (1.17) 339 → 333 ms (0.98) 665 → 798 ms (1.20) 275 → 306 ms (1.11) 377 → 346 ms (0.92) 557 → 500 ms (0.90)
fineweb_q03 492 → 412 ms (0.84) 1.00 s → 505 ms (0.50) 🟢 730 → 706 ms (0.97) 901 → 878 ms (0.98) 871 → 835 ms (0.96) 799 → 680 ms (0.85)
fineweb_q04 570 → 429 ms (0.75) 529 → 508 ms (0.96) 810 → 746 ms (0.92) 877 → 893 ms (1.02) 974 → 899 ms (0.92) 863 → 816 ms (0.95)
fineweb_q05 408 → 393 ms (0.96) 490 → 464 ms (0.95) 675 → 718 ms (1.06) 844 → 872 ms (1.03) 884 → 938 ms (1.06) 733 → 670 ms (0.91)
fineweb_q06 490 → 508 ms (1.04) 712 → 718 ms (1.01) 751 → 978 ms (1.30) 🔴 946 → 913 ms (0.97) 883 → 854 ms (0.97) 731 → 707 ms (0.97)
fineweb_q07 455 → 444 ms (0.98) 470 → 478 ms (1.02) 875 → 673 ms (0.77) 852 → 816 ms (0.96) 823 → 845 ms (1.03) 775 → 701 ms (0.90)
fineweb_q08 163 → 171 ms (1.05) 151 → 142 ms (0.94) 773 → 716 ms (0.93) 200 → 212 ms (1.06) 210 → 196 ms (0.93) 295 → 271 ms (0.92)
Cold runs (first iteration): geomean 0.910x ➖ · 9 queries × 6 targets (5↑ 2↓)
query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet
geomean 419 → 329 ms (0.78) 469 → 415 ms (0.88) 726 → 742 ms (1.02) 489 → 496 ms (1.01) 569 → 494 ms (0.87) 639 → 580 ms (0.91)
fineweb_q00 157 → 71.7 ms (0.46) 🟢 112 → 104 ms (0.93) 697 → 733 ms (1.05) 98.4 → 87.9 ms (0.89) 94.3 → 88.6 ms (0.94) 438 → 383 ms (0.87)
fineweb_q01 1.28 s → 473 ms (0.37) 🟢 882 → 509 ms (0.58) 🟢 730 → 625 ms (0.86) 320 → 369 ms (1.15) 379 → 362 ms (0.96) 589 → 546 ms (0.93)
fineweb_q02 314 → 249 ms (0.79) 455 → 355 ms (0.78) 668 → 812 ms (1.22) 307 → 349 ms (1.14) 577 → 372 ms (0.64) 🟢 576 → 538 ms (0.93)
fineweb_q03 545 → 510 ms (0.94) 1.12 s → 557 ms (0.50) 🟢 723 → 783 ms (1.08) 997 → 962 ms (0.96) 982 → 919 ms (0.94) 777 → 748 ms (0.96)
fineweb_q04 562 → 427 ms (0.76) 550 → 506 ms (0.92) 831 → 634 ms (0.76) 1.00 → 1.01 s (1.01) 1.21 s → 987 ms (0.82) 805 → 814 ms (1.01)
fineweb_q05 448 → 437 ms (0.98) 551 → 495 ms (0.90) 660 → 679 ms (1.03) 943 → 927 ms (0.98) 1.02 s → 960 ms (0.94) 812 → 642 ms (0.79)
fineweb_q06 490 → 560 ms (1.14) 642 ms → 1.32 s (2.05) 🔴 747 → 991 ms (1.33) 🔴 1.04 s → 997 ms (0.96) 968 → 884 ms (0.91) 776 → 644 ms (0.83)
fineweb_q07 515 → 536 ms (1.04) 518 → 596 ms (1.15) 696 → 707 ms (1.02) 928 → 867 ms (0.93) 1.10 s → 905 ms (0.82) 860 → 776 ms (0.90)
fineweb_q08 184 → 187 ms (1.02) 215 → 176 ms (0.82) 795 → 773 ms (0.97) 183 → 205 ms (1.12) 237 → 213 ms (0.90) 355 → 339 ms (0.95)

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on NVME 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: Likely regression (environment too noisy confidence)
Attributed Vortex impact: +135.3%
Engines: DataFusion Likely regression (+117.6%, environment too noisy confidence) · DuckDB Likely regression (+153.8%, environment too noisy confidence)
Vortex (geomean): hot 1.000x ➖ · cold 0.990x ➖
Parquet (geomean): hot 0.426x ✅ · cold 0.519x ✅
Shifts: Parquet (control) -57.4% · Median polish -0.3%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.784x 🟢 · 22 queries × 7 targets (45↑ 2↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 20.2 → 20.1 ms (1.00) 21.4 → 21.2 ms (0.99) 77.5 → 35.4 ms (0.46) 🟢 15.5 → 15.5 ms (1.00) 16.9 → 17.1 ms (1.01) 55.9 → 22.1 ms (0.40) 🟢 13.3 → 13.4 ms (1.01)
tpch_q01 15.6 → 15.2 ms (0.98) 16.8 → 16.2 ms (0.96) 71.3 → 18.7 ms (0.26) 🟢 9.55 → 9.19 ms (0.96) 9.36 → 11.3 ms (1.21) 🔴 47.4 → 12.0 ms (0.25) 🟢 6.28 → 6.38 ms (1.02)
tpch_q02 17.5 → 17.5 ms (1.00) 19.6 → 19.4 ms (0.99) 77.2 → 36.2 ms (0.47) 🟢 15.2 → 15.1 ms (1.00) 18.2 → 17.6 ms (0.97) 48.4 → 19.6 ms (0.40) 🟢 8.85 → 8.62 ms (0.97)
tpch_q03 19.2 → 19.4 ms (1.01) 19.8 → 20.0 ms (1.01) 81.7 → 38.6 ms (0.47) 🟢 18.5 → 18.7 ms (1.01) 19.5 → 19.4 ms (1.00) 60.4 → 24.6 ms (0.41) 🟢 15.4 → 15.5 ms (1.01)
tpch_q04 10.6 → 11.0 ms (1.04) 11.0 → 10.9 ms (0.99) 39.5 → 19.1 ms (0.48) 🟢 12.3 → 12.2 ms (0.99) 13.2 → 13.4 ms (1.01) 34.8 → 16.4 ms (0.47) 🟢 11.5 → 11.2 ms (0.97)
tpch_q05 37.3 → 37.4 ms (1.00) 38.2 → 38.3 ms (1.00) 110 → 55.3 ms (0.50) 🟢 20.1 → 20.0 ms (0.99) 21.2 → 21.2 ms (1.00) 67.3 → 26.5 ms (0.39) 🟢 15.5 → 15.4 ms (1.00)
tpch_q06 4.56 → 4.34 ms (0.95) 5.39 → 5.00 ms (0.93) 31.4 → 10.3 ms (0.33) 🟢 4.88 → 4.91 ms (1.01) 5.20 → 5.32 ms (1.02) 21.9 → 8.72 ms (0.40) 🟢 3.21 → 3.22 ms (1.00)
tpch_q07 53.6 → 52.8 ms (0.99) 53.9 → 54.2 ms (1.01) 113 → 64.8 ms (0.57) 🟢 24.0 → 24.3 ms (1.01) 25.8 → 25.8 ms (1.00) 63.9 → 26.5 ms (0.41) 🟢 18.5 → 18.2 ms (0.98)
tpch_q08 52.2 → 52.0 ms (1.00) 52.4 → 53.0 ms (1.01) 117 → 66.7 ms (0.57) 🟢 21.3 → 21.5 ms (1.01) 22.4 → 22.0 ms (0.98) 75.7 → 27.4 ms (0.36) 🟢 17.5 → 17.1 ms (0.97)
tpch_q09 47.2 → 47.6 ms (1.01) 48.3 → 49.0 ms (1.01) 137 → 67.0 ms (0.49) 🟢 21.7 → 22.1 ms (1.02) 24.5 → 23.9 ms (0.98) 108 → 38.6 ms (0.36) 🟢 23.9 → 33.6 ms (1.40) 🔴
tpch_q10 17.8 → 17.5 ms (0.98) 19.1 → 19.6 ms (1.02) 104 → 46.6 ms (0.45) 🟢 27.7 → 27.6 ms (0.99) 31.0 → 30.8 ms (0.99) 106 → 53.0 ms (0.50) 🟢 29.0 → 31.9 ms (1.10)
tpch_q11 14.6 → 14.6 ms (1.00) 15.2 → 15.3 ms (1.01) 61.7 → 28.7 ms (0.47) 🟢 8.11 → 8.03 ms (0.99) 8.59 → 8.57 ms (1.00) 29.5 → 10.1 ms (0.34) 🟢 5.80 → 5.84 ms (1.01)
tpch_q12 18.5 → 18.7 ms (1.01) 20.0 → 20.1 ms (1.01) 63.2 → 30.9 ms (0.49) 🟢 14.3 → 14.7 ms (1.03) 17.0 → 17.2 ms (1.01) 37.2 → 16.8 ms (0.45) 🟢 10.7 → 10.4 ms (0.97)
tpch_q13 18.8 → 19.8 ms (1.05) 21.4 → 20.8 ms (0.97) 143 → 41.1 ms (0.29) 🟢 19.6 → 19.4 ms (0.99) 20.0 → 20.9 ms (1.04) 175 → 40.9 ms (0.23) 🟢 32.1 → 34.1 ms (1.06)
tpch_q14 14.4 → 14.2 ms (0.98) 15.4 → 15.6 ms (1.01) 47.8 → 27.0 ms (0.56) 🟢 11.1 → 11.4 ms (1.02) 12.3 → 12.5 ms (1.02) 42.3 → 17.9 ms (0.42) 🟢 12.4 → 12.2 ms (0.98)
tpch_q15 18.5 → 18.8 ms (1.01) 20.5 → 19.9 ms (0.97) 74.2 → 28.3 ms (0.38) 🟢 8.32 → 8.40 ms (1.01) 9.58 → 9.66 ms (1.01) 32.6 → 12.9 ms (0.39) 🟢 6.93 → 6.81 ms (0.98)
tpch_q16 18.4 → 18.4 ms (1.00) 18.6 → 18.1 ms (0.97) 43.0 → 26.4 ms (0.61) 🟢 16.6 → 16.4 ms (0.99) 17.1 → 17.3 ms (1.01) 47.1 → 22.0 ms (0.47) 🟢 19.2 → 18.8 ms (0.98)
tpch_q17 29.4 → 29.5 ms (1.00) 30.6 → 30.3 ms (0.99) 123 → 46.8 ms (0.38) 🟢 13.0 → 12.7 ms (0.97) 13.8 → 13.9 ms (1.00) 48.0 → 18.6 ms (0.39) 🟢 8.53 → 8.57 ms (1.00)
tpch_q18 36.7 → 36.7 ms (1.00) 36.6 → 36.8 ms (1.01) 120 → 50.9 ms (0.42) 🟢 24.0 → 24.7 ms (1.03) 25.2 → 25.3 ms (1.00) 73.9 → 28.1 ms (0.38) 🟢 18.3 → 17.9 ms (0.98)
tpch_q19 10.9 → 9.98 ms (0.91) 12.4 → 11.2 ms (0.90) 56.6 → 22.2 ms (0.39) 🟢 11.4 → 11.4 ms (1.01) 14.5 → 14.6 ms (1.01) 46.1 → 18.3 ms (0.40) 🟢 12.7 → 9.78 ms (0.77) 🟢
tpch_q20 20.8 → 20.6 ms (0.99) 22.2 → 22.3 ms (1.00) 77.8 → 44.6 ms (0.57) 🟢 14.5 → 14.4 ms (0.99) 17.1 → 17.1 ms (1.00) 63.4 → 25.1 ms (0.40) 🟢 14.4 → 15.1 ms (1.05)
tpch_q21 35.5 → 35.3 ms (0.99) 37.2 → 37.9 ms (1.02) 113 → 55.8 ms (0.49) 🟢 48.8 → 47.9 ms (0.98) 49.2 → 49.3 ms (1.00) 107 → 53.1 ms (0.50) 🟢 21.6 → 21.5 ms (0.99)
tpch_q22 10.9 → 11.0 ms (1.00) 11.1 → 11.4 ms (1.02) 51.2 → 32.6 ms (0.64) 🟢 16.8 → 17.0 ms (1.01) 19.6 → 19.6 ms (1.00) 50.4 → 27.3 ms (0.54) 🟢 23.4 → 24.8 ms (1.06)
Cold runs (first iteration): geomean 0.821x 🟢 · 22 queries × 7 targets (48↑ 1↓)
query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 46.7 → 46.3 ms (0.99) 47.2 → 46.8 ms (0.99) 100 → 59.1 ms (0.59) 🟢 29.1 → 28.8 ms (0.99) 30.7 → 30.4 ms (0.99) 65.0 → 29.8 ms (0.46) 🟢 23.4 → 22.7 ms (0.97)
tpch_q01 41.3 → 43.1 ms (1.04) 44.2 → 45.8 ms (1.04) 87.9 → 41.8 ms (0.48) 🟢 20.6 → 19.8 ms (0.96) 23.0 → 22.2 ms (0.96) 53.9 → 18.4 ms (0.34) 🟢 14.4 → 14.3 ms (0.99)
tpch_q02 38.7 → 37.4 ms (0.96) 37.7 → 38.6 ms (1.03) 92.0 → 52.8 ms (0.57) 🟢 25.4 → 24.8 ms (0.97) 27.4 → 26.8 ms (0.98) 55.3 → 26.6 ms (0.48) 🟢 16.4 → 16.0 ms (0.97)
tpch_q03 44.4 → 44.4 ms (1.00) 45.8 → 43.5 ms (0.95) 105 → 62.5 ms (0.59) 🟢 34.4 → 35.0 ms (1.02) 35.9 → 35.7 ms (0.99) 68.6 → 31.2 ms (0.45) 🟢 25.1 → 24.6 ms (0.98)
tpch_q04 34.2 → 33.9 ms (0.99) 34.8 → 34.4 ms (0.99) 59.5 → 38.7 ms (0.65) 🟢 25.0 → 24.8 ms (0.99) 25.1 → 27.1 ms (1.08) 43.0 → 23.7 ms (0.55) 🟢 22.4 → 20.5 ms (0.91)
tpch_q05 64.4 → 65.6 ms (1.02) 69.7 → 67.5 ms (0.97) 139 → 81.5 ms (0.59) 🟢 37.6 → 37.8 ms (1.00) 36.9 → 36.3 ms (0.98) 77.1 → 33.5 ms (0.43) 🟢 27.4 → 25.3 ms (0.92)
tpch_q06 25.4 → 25.8 ms (1.02) 26.8 → 26.0 ms (0.97) 48.2 → 32.9 ms (0.68) 🟢 14.8 → 14.9 ms (1.00) 15.6 → 15.9 ms (1.02) 27.8 → 13.2 ms (0.48) 🟢 9.84 → 9.77 ms (0.99)
tpch_q07 81.8 → 79.8 ms (0.97) 82.0 → 79.4 ms (0.97) 139 → 90.1 ms (0.65) 🟢 41.1 → 39.3 ms (0.96) 41.3 → 42.3 ms (1.02) 72.5 → 36.0 ms (0.50) 🟢 29.8 → 27.2 ms (0.91)
tpch_q08 81.0 → 81.5 ms (1.01) 78.2 → 80.8 ms (1.03) 145 → 96.4 ms (0.66) 🟢 37.3 → 36.9 ms (0.99) 39.0 → 38.8 ms (1.00) 86.9 → 36.0 ms (0.41) 🟢 29.8 → 28.7 ms (0.96)
tpch_q09 76.4 → 79.5 ms (1.04) 79.7 → 77.4 ms (0.97) 166 → 96.6 ms (0.58) 🟢 37.7 → 37.1 ms (0.98) 40.2 → 37.7 ms (0.94) 120 → 46.1 ms (0.38) 🟢 37.1 → 45.7 ms (1.23) 🔴
tpch_q10 50.6 → 48.6 ms (0.96) 49.7 → 48.8 ms (0.98) 130 → 69.3 ms (0.53) 🟢 45.9 → 45.0 ms (0.98) 48.2 → 46.1 ms (0.96) 118 → 60.4 ms (0.51) 🟢 43.1 → 44.3 ms (1.03)
tpch_q11 34.2 → 35.2 ms (1.03) 35.4 → 35.8 ms (1.01) 81.6 → 46.8 ms (0.57) 🟢 17.1 → 17.7 ms (1.03) 18.5 → 17.9 ms (0.97) 34.9 → 15.5 ms (0.44) 🟢 11.9 → 11.7 ms (0.99)
tpch_q12 48.6 → 42.2 ms (0.87) 🟢 41.4 → 42.0 ms (1.02) 84.5 → 50.8 ms (0.60) 🟢 25.4 → 24.6 ms (0.97) 29.4 → 28.3 ms (0.96) 44.4 → 23.1 ms (0.52) 🟢 21.1 → 18.6 ms (0.88) 🟢
tpch_q13 40.8 → 37.6 ms (0.92) 39.0 → 38.4 ms (0.98) 164 → 58.4 ms (0.36) 🟢 31.8 → 33.1 ms (1.04) 32.8 → 32.7 ms (1.00) 183 → 47.5 ms (0.26) 🟢 41.1 → 41.1 ms (1.00)
tpch_q14 38.7 → 37.6 ms (0.97) 38.7 → 36.5 ms (0.94) 71.1 → 48.6 ms (0.68) 🟢 24.9 → 24.8 ms (1.00) 27.5 → 25.6 ms (0.93) 51.0 → 26.2 ms (0.51) 🟢 21.1 → 19.8 ms (0.94)
tpch_q15 45.7 → 46.8 ms (1.02) 47.1 → 46.2 ms (0.98) 94.7 → 48.1 ms (0.51) 🟢 21.6 → 20.4 ms (0.95) 22.5 → 22.6 ms (1.00) 40.7 → 21.1 ms (0.52) 🟢 16.9 → 14.8 ms (0.88) 🟢
tpch_q16 39.0 → 40.1 ms (1.03) 38.7 → 39.0 ms (1.01) 61.1 → 43.4 ms (0.71) 🟢 23.5 → 24.6 ms (1.05) 23.8 → 24.4 ms (1.02) 52.9 → 27.0 ms (0.51) 🟢 25.2 → 25.2 ms (1.00)
tpch_q17 58.4 → 57.1 ms (0.98) 58.1 → 58.7 ms (1.01) 144 → 75.9 ms (0.53) 🟢 28.1 → 27.5 ms (0.98) 29.4 → 29.2 ms (0.99) 57.4 → 26.2 ms (0.46) 🟢 17.3 → 17.2 ms (0.99)
tpch_q18 58.8 → 57.5 ms (0.98) 59.6 → 60.4 ms (1.01) 141 → 71.8 ms (0.51) 🟢 35.1 → 34.2 ms (0.97) 34.5 → 34.0 ms (0.99) 81.6 → 35.7 ms (0.44) 🟢 25.8 → 25.4 ms (0.98)
tpch_q19 35.9 → 36.4 ms (1.02) 39.0 → 37.9 ms (0.97) 83.1 → 49.8 ms (0.60) 🟢 25.9 → 25.3 ms (0.98) 31.4 → 30.6 ms (0.98) 60.6 → 26.0 ms (0.43) 🟢 22.4 → 18.9 ms (0.84) 🟢
tpch_q20 44.9 → 44.4 ms (0.99) 48.0 → 45.9 ms (0.96) 100 → 69.3 ms (0.69) 🟢 31.3 → 30.0 ms (0.96) 32.2 → 31.8 ms (0.99) 73.9 → 34.4 ms (0.47) 🟢 27.1 → 27.2 ms (1.01)
tpch_q21 64.7 → 66.6 ms (1.03) 66.3 → 66.2 ms (1.00) 138 → 84.1 ms (0.61) 🟢 63.5 → 63.6 ms (1.00) 64.8 → 65.0 ms (1.00) 116 → 62.3 ms (0.54) 🟢 33.2 → 31.2 ms (0.94)
tpch_q22 30.8 → 30.8 ms (1.00) 31.0 → 31.3 ms (1.01) 69.4 → 49.9 ms (0.72) 🟢 27.5 → 27.3 ms (0.99) 29.0 → 29.4 ms (1.01) 59.3 → 34.3 ms (0.58) 🟢 32.3 → 32.4 ms (1.00)

File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
File Scale Format Base HEAD Change %
region_0001.vortex 1.0 vortex-file-compressed 6.33 KB 6.36 KB +32 B +0.5%

Totals:

  • vortex-compact: 189.32 MB → 189.32 MB (0.0%)
  • vortex-file-compressed: 216.07 MB → 216.07 MB (+0.0%)

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Clickbench on NVME 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.1%
Engines: DataFusion No clear signal (+0.4%, low confidence) · DuckDB No clear signal (-0.3%, low confidence)
Vortex (geomean): hot 0.993x ➖ · cold 1.019x ➖
Parquet (geomean): hot 0.995x ➖ · cold 1.011x ➖
Shifts: Parquet (control) -0.5% · Median polish -0.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.995x ➖ · 43 queries × 7 targets (4↑ 1↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 41.5 → 41.1 ms (0.99) 48.5 → 48.4 ms (1.00) 65.5 → 64.9 ms (0.99) 36.1 → 35.5 ms (0.99) 42.7 → 42.5 ms (1.00) 67.0 → 66.8 ms (1.00) 65.6 → 65.8 ms (1.00)
clickbench_q00 1.28 → 1.27 ms (0.99) 1.27 → 1.22 ms (0.96) 1.27 → 1.24 ms (0.98) 2.27 → 2.19 ms (0.97) 2.26 → 2.15 ms (0.95) 9.85 → 9.11 ms (0.92) 11.0 → 11.9 ms (1.08)
clickbench_q01 9.02 → 9.18 ms (1.02) 7.41 → 7.12 ms (0.96) 16.0 → 16.0 ms (1.00) 7.83 → 7.06 ms (0.90) 7.03 → 7.08 ms (1.01) 15.1 → 15.4 ms (1.02) 26.4 → 26.2 ms (0.99)
clickbench_q02 13.5 → 13.5 ms (1.00) 13.1 → 12.7 ms (0.97) 18.4 → 18.7 ms (1.02) 9.21 → 9.29 ms (1.01) 12.6 → 12.2 ms (0.97) 22.1 → 21.8 ms (0.99) 22.2 → 23.0 ms (1.04)
clickbench_q03 13.9 → 14.1 ms (1.01) 13.0 → 12.2 ms (0.94) 16.8 → 17.0 ms (1.01) 13.9 → 13.9 ms (1.00) 15.0 → 14.8 ms (0.99) 22.8 → 22.9 ms (1.01) 21.3 → 22.0 ms (1.03)
clickbench_q04 58.5 → 58.8 ms (1.00) 59.1 → 59.4 ms (1.01) 64.2 → 63.5 ms (0.99) 57.3 → 56.1 ms (0.98) 57.6 → 57.4 ms (1.00) 61.5 → 61.0 ms (0.99) 74.4 → 74.8 ms (1.01)
clickbench_q05 80.7 → 80.0 ms (0.99) 87.8 → 83.1 ms (0.95) 88.9 → 88.6 ms (1.00) 63.8 → 64.4 ms (1.01) 68.8 → 68.2 ms (0.99) 83.6 → 83.0 ms (0.99) 79.4 → 83.5 ms (1.05)
clickbench_q06 1.25 → 1.25 ms (1.01) 1.28 → 1.23 ms (0.96) 1.26 → 1.25 ms (0.99) 2.76 → 2.63 ms (0.95) 2.64 → 2.60 ms (0.98) 20.3 → 20.2 ms (1.00) 21.3 → 21.6 ms (1.01)
clickbench_q07 12.2 → 12.0 ms (0.98) 11.9 → 12.4 ms (1.04) 16.8 → 16.9 ms (1.01) 9.71 → 8.84 ms (0.91) 9.76 → 9.20 ms (0.94) 16.8 → 16.8 ms (1.00) 17.7 → 18.3 ms (1.03)
clickbench_q08 82.3 → 79.4 ms (0.96) 81.3 → 79.8 ms (0.98) 87.7 → 85.1 ms (0.97) 71.5 → 68.3 ms (0.95) 75.2 → 74.5 ms (0.99) 76.6 → 73.9 ms (0.96) 88.2 → 86.6 ms (0.98)
clickbench_q09 173 → 172 ms (0.99) 179 → 173 ms (0.97) 186 → 188 ms (1.01) 91.3 → 93.3 ms (1.02) 102 → 98.9 ms (0.97) 103 → 102 ms (0.99) 97.7 → 98.4 ms (1.01)
clickbench_q10 24.9 → 24.9 ms (1.00) 26.6 → 26.6 ms (1.00) 37.0 → 35.6 ms (0.96) 28.3 → 27.7 ms (0.98) 32.2 → 32.4 ms (1.01) 41.1 → 42.2 ms (1.03) 52.4 → 51.5 ms (0.98)
clickbench_q11 27.5 → 26.7 ms (0.97) 29.1 → 29.0 ms (1.00) 43.1 → 43.0 ms (1.00) 30.5 → 30.2 ms (0.99) 35.3 → 35.4 ms (1.00) 47.1 → 46.3 ms (0.98) 55.3 → 56.2 ms (1.02)
clickbench_q12 74.0 → 74.5 ms (1.01) 85.5 → 88.1 ms (1.03) 96.6 → 97.1 ms (1.00) 70.6 → 68.5 ms (0.97) 83.2 → 83.9 ms (1.01) 92.1 → 92.4 ms (1.00) 81.2 → 88.1 ms (1.09)
clickbench_q13 121 → 120 ms (0.99) 130 → 133 ms (1.02) 148 → 147 ms (0.99) 118 → 117 ms (0.99) 145 → 144 ms (0.99) 150 → 152 ms (1.01) 146 → 148 ms (1.01)
clickbench_q14 76.8 → 77.9 ms (1.01) 94.6 → 92.0 ms (0.97) 102 → 101 ms (0.99) 72.1 → 72.4 ms (1.00) 88.5 → 89.5 ms (1.01) 100 → 101 ms (1.00) 90.4 → 98.3 ms (1.09)
clickbench_q15 67.6 → 67.7 ms (1.00) 62.6 → 64.0 ms (1.02) 72.9 → 71.0 ms (0.97) 76.8 → 75.6 ms (0.98) 77.6 → 76.9 ms (0.99) 86.7 → 85.7 ms (0.99) 83.6 → 83.1 ms (0.99)
clickbench_q16 161 → 158 ms (0.98) 163 → 163 ms (1.00) 157 → 156 ms (1.00) 138 → 136 ms (0.99) 147 → 144 ms (0.98) 163 → 162 ms (0.99) 167 → 158 ms (0.94)
clickbench_q17 158 → 156 ms (0.98) 161 → 161 ms (1.00) 159 → 156 ms (0.99) 136 → 138 ms (1.01) 151 → 148 ms (0.98) 165 → 173 ms (1.05) 152 → 158 ms (1.04)
clickbench_q18 298 → 306 ms (1.02) 291 → 295 ms (1.01) 311 → 311 ms (1.00) 258 → 253 ms (0.98) 262 → 253 ms (0.96) 294 → 294 ms (1.00) 303 → 311 ms (1.03)
clickbench_q19 9.99 → 9.92 ms (0.99) 9.83 → 10.0 ms (1.02) 19.2 → 18.9 ms (0.98) 8.98 → 9.01 ms (1.00) 9.67 → 9.66 ms (1.00) 19.3 → 19.4 ms (1.01) 17.6 → 18.4 ms (1.04)
clickbench_q20 47.9 → 46.8 ms (0.98) 84.8 → 86.7 ms (1.02) 179 → 178 ms (1.00) 51.3 → 51.7 ms (1.01) 79.1 → 78.6 ms (0.99) 196 → 185 ms (0.94) 189 → 147 ms (0.78) 🟢
clickbench_q21 57.5 → 57.2 ms (0.99) 114 → 117 ms (1.03) 174 → 173 ms (0.99) 56.6 → 57.1 ms (1.01) 106 → 107 ms (1.02) 220 → 216 ms (0.98) 171 → 137 ms (0.80) 🟢
clickbench_q22 78.5 → 74.7 ms (0.95) 224 → 226 ms (1.01) 263 → 260 ms (0.99) 87.9 → 90.7 ms (1.03) 238 → 236 ms (0.99) 308 → 311 ms (1.01) 182 → 163 ms (0.89) 🟢
clickbench_q23 98.0 → 97.7 ms (1.00) 163 → 179 ms (1.10) 1.00 → 1.01 s (1.00) 62.2 → 61.5 ms (0.99) 71.8 → 74.1 ms (1.03) 164 → 177 ms (1.08) 73.7 → 68.6 ms (0.93)
clickbench_q24 9.87 → 9.13 ms (0.93) 18.2 → 19.8 ms (1.09) 30.2 → 29.8 ms (0.99) 17.2 → 16.8 ms (0.98) 22.6 → 22.0 ms (0.97) 29.5 → 29.7 ms (1.01) 30.3 → 32.5 ms (1.07)
clickbench_q25 20.4 → 20.4 ms (1.00) 36.7 → 37.1 ms (1.01) 37.5 → 36.9 ms (0.98) 19.6 → 18.9 ms (0.97) 41.6 → 41.5 ms (1.00) 41.7 → 41.6 ms (1.00) 41.7 → 45.5 ms (1.09)
clickbench_q26 15.5 → 15.4 ms (0.99) 32.6 → 31.9 ms (0.98) 30.6 → 30.6 ms (1.00) 14.8 → 14.5 ms (0.98) 21.2 → 23.0 ms (1.09) 31.2 → 31.0 ms (0.99) 30.5 → 32.9 ms (1.08)
clickbench_q27 105 → 104 ms (0.99) 165 → 164 ms (1.00) 195 → 192 ms (0.99) 52.1 → 51.6 ms (0.99) 131 → 131 ms (1.00) 188 → 199 ms (1.06) 154 → 125 ms (0.81) 🟢
clickbench_q28 559 → 564 ms (1.01) 603 → 607 ms (1.01) 623 → 619 ms (0.99) 520 → 515 ms (0.99) 567 → 569 ms (1.00) 967 → 945 ms (0.98) 827 → 760 ms (0.92)
clickbench_q29 26.2 → 26.3 ms (1.00) 25.9 → 26.0 ms (1.01) 29.7 → 28.9 ms (0.97) 17.2 → 17.2 ms (1.00) 19.4 → 19.1 ms (0.99) 24.5 → 24.5 ms (1.00) 27.2 → 28.1 ms (1.03)
clickbench_q30 62.5 → 59.0 ms (0.94) 70.4 → 70.7 ms (1.00) 91.8 → 89.3 ms (0.97) 60.4 → 60.0 ms (0.99) 77.9 → 77.2 ms (0.99) 105 → 105 ms (1.00) 100 → 106 ms (1.06)
clickbench_q31 69.9 → 69.5 ms (0.99) 76.6 → 76.1 ms (0.99) 105 → 103 ms (0.98) 110 → 110 ms (1.00) 117 → 127 ms (1.08) 134 → 145 ms (1.08) 160 → 175 ms (1.10)
clickbench_q32 294 → 292 ms (0.99) 286 → 288 ms (1.01) 308 → 307 ms (1.00) 386 → 367 ms (0.95) 379 → 375 ms (0.99) 391 → 377 ms (0.96) 463 → 461 ms (1.00)
clickbench_q33 330 → 330 ms (1.00) 336 → 334 ms (0.99) 383 → 375 ms (0.98) 342 → 341 ms (1.00) 345 → 343 ms (1.00) 471 → 472 ms (1.00) 450 → 433 ms (0.96)
clickbench_q34 324 → 325 ms (1.00) 333 → 329 ms (0.99) 379 → 377 ms (0.99) 354 → 352 ms (0.99) 354 → 362 ms (1.02) 472 → 472 ms (1.00) 454 → 443 ms (0.98)
clickbench_q35 61.1 → 61.2 ms (1.00) 59.0 → 60.4 ms (1.02) 64.6 → 64.6 ms (1.00) 60.4 → 57.5 ms (0.95) 59.8 → 59.5 ms (1.00) 61.5 → 61.3 ms (1.00) 74.4 → 74.7 ms (1.00)
clickbench_q36 37.2 → 37.7 ms (1.01) 43.1 → 43.5 ms (1.01) 74.1 → 74.2 ms (1.00) 11.3 → 11.4 ms (1.01) 12.5 → 12.3 ms (0.98) 30.3 → 29.7 ms (0.98) 32.2 → 30.3 ms (0.94)
clickbench_q37 21.5 → 20.9 ms (0.97) 22.8 → 22.7 ms (1.00) 36.1 → 35.5 ms (0.99) 7.88 → 7.78 ms (0.99) 9.66 → 9.56 ms (0.99) 24.1 → 23.5 ms (0.98) 16.7 → 18.0 ms (1.08)
clickbench_q38 13.0 → 12.9 ms (0.99) 19.1 → 18.4 ms (0.96) 48.0 → 47.8 ms (1.00) 8.51 → 8.19 ms (0.96) 8.82 → 8.84 ms (1.00) 23.7 → 23.7 ms (1.00) 18.6 → 22.6 ms (1.22) 🔴
clickbench_q39 78.9 → 78.3 ms (0.99) 83.7 → 83.3 ms (1.00) 144 → 143 ms (0.99) 17.7 → 18.0 ms (1.01) 19.3 → 19.1 ms (0.99) 53.2 → 53.2 ms (1.00) 54.8 → 50.9 ms (0.93)
clickbench_q40 10.2 → 10.3 ms (1.01) 12.4 → 11.9 ms (0.96) 20.3 → 20.0 ms (0.99) 6.99 → 6.93 ms (0.99) 7.73 → 7.61 ms (0.98) 15.0 → 14.7 ms (0.98) 16.1 → 16.9 ms (1.05)
clickbench_q41 9.49 → 9.56 ms (1.01) 10.2 → 10.7 ms (1.05) 18.9 → 18.6 ms (0.98) 7.87 → 7.74 ms (0.98) 8.20 → 8.18 ms (1.00) 15.9 → 15.3 ms (0.97) 16.0 → 16.7 ms (1.04)
clickbench_q42 9.94 → 9.93 ms (1.00) 10.5 → 10.1 ms (0.97) 19.4 → 19.3 ms (1.00) 7.04 → 6.97 ms (0.99) 7.22 → 7.21 ms (1.00) 14.7 → 14.0 ms (0.96) 16.0 → 16.7 ms (1.04)
Cold runs (first iteration): geomean 1.014x ➖ · 43 queries × 7 targets (5↑ 17↓)
query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 105 → 108 ms (1.03) 105 → 106 ms (1.01) 156 → 159 ms (1.02) 82.8 → 85.2 ms (1.03) 87.0 → 88.1 ms (1.01) 82.3 → 82.6 ms (1.00) 98.7 → 99.1 ms (1.00)
clickbench_q00 22.3 → 24.3 ms (1.09) 21.9 → 21.2 ms (0.97) 49.9 → 50.7 ms (1.02) 25.3 → 24.9 ms (0.98) 27.3 → 25.6 ms (0.94) 14.7 → 14.6 ms (0.99) 13.6 → 14.5 ms (1.06)
clickbench_q01 32.5 → 36.9 ms (1.14) 🔴 34.4 → 33.9 ms (0.99) 67.2 → 68.4 ms (1.02) 28.2 → 28.1 ms (1.00) 30.1 → 30.4 ms (1.01) 20.6 → 21.3 ms (1.03) 43.8 → 44.9 ms (1.03)
clickbench_q02 47.3 → 47.0 ms (0.99) 44.5 → 45.2 ms (1.01) 80.0 → 79.8 ms (1.00) 33.3 → 34.5 ms (1.03) 36.8 → 38.6 ms (1.05) 31.0 → 31.6 ms (1.02) 46.2 → 49.6 ms (1.07)
clickbench_q03 49.7 → 54.0 ms (1.09) 48.7 → 48.6 ms (1.00) 77.6 → 81.6 ms (1.05) 44.1 → 46.3 ms (1.05) 42.2 → 45.6 ms (1.08) 36.6 → 33.5 ms (0.92) 44.0 → 46.1 ms (1.05)
clickbench_q04 104 → 104 ms (1.00) 101 → 101 ms (1.00) 130 → 126 ms (0.97) 87.8 → 89.3 ms (1.02) 88.0 → 89.4 ms (1.02) 73.7 → 74.8 ms (1.01) 94.1 → 98.1 ms (1.04)
clickbench_q05 131 → 136 ms (1.03) 134 → 133 ms (1.00) 154 → 157 ms (1.02) 92.2 → 94.4 ms (1.02) 96.8 → 96.3 ms (0.99) 97.5 → 97.2 ms (1.00) 100.0 → 104 ms (1.04)
clickbench_q06 22.6 → 23.9 ms (1.06) 22.0 → 21.7 ms (0.98) 49.6 → 50.0 ms (1.01) 33.5 → 33.1 ms (0.99) 35.6 → 32.1 ms (0.90) 28.6 → 28.6 ms (1.00) 27.5 → 29.1 ms (1.06)
clickbench_q07 37.5 → 37.8 ms (1.01) 38.8 → 35.8 ms (0.92) 67.7 → 68.5 ms (1.01) 29.7 → 29.0 ms (0.98) 33.3 → 30.8 ms (0.93) 22.9 → 23.6 ms (1.03) 32.1 → 37.7 ms (1.18) 🔴
clickbench_q08 130 → 127 ms (0.97) 123 → 126 ms (1.02) 156 → 163 ms (1.04) 99.1 → 101 ms (1.02) 108 → 110 ms (1.02) 88.6 → 89.4 ms (1.01) 115 → 118 ms (1.03)
clickbench_q09 215 → 218 ms (1.01) 218 → 215 ms (0.99) 259 → 267 ms (1.03) 122 → 124 ms (1.01) 137 → 136 ms (0.99) 117 → 119 ms (1.02) 131 → 140 ms (1.06)
clickbench_q10 64.5 → 67.4 ms (1.04) 62.1 → 61.9 ms (1.00) 100 → 101 ms (1.01) 61.3 → 67.8 ms (1.11) 🔴 66.1 → 68.8 ms (1.04) 53.5 → 54.4 ms (1.02) 87.1 → 89.8 ms (1.03)
clickbench_q11 68.2 → 68.6 ms (1.01) 63.0 → 62.0 ms (0.98) 109 → 111 ms (1.02) 65.7 → 72.3 ms (1.10) 67.5 → 72.4 ms (1.07) 62.5 → 61.0 ms (0.98) 93.3 → 95.8 ms (1.03)
clickbench_q12 123 → 128 ms (1.04) 135 → 142 ms (1.06) 162 → 165 ms (1.02) 97.2 → 98.1 ms (1.01) 112 → 114 ms (1.01) 105 → 106 ms (1.01) 103 → 109 ms (1.06)
clickbench_q13 189 → 189 ms (1.00) 191 → 195 ms (1.02) 225 → 229 ms (1.02) 156 → 163 ms (1.04) 182 → 185 ms (1.02) 167 → 167 ms (1.00) 180 → 181 ms (1.00)
clickbench_q14 131 → 133 ms (1.02) 141 → 140 ms (0.99) 172 → 173 ms (1.01) 98.8 → 101 ms (1.02) 117 → 120 ms (1.02) 117 → 116 ms (1.00) 111 → 119 ms (1.06)
clickbench_q15 99.0 → 103 ms (1.05) 101 → 96.6 ms (0.96) 130 → 130 ms (1.00) 108 → 106 ms (0.98) 107 → 108 ms (1.01) 102 → 101 ms (1.00) 103 → 105 ms (1.02)
clickbench_q16 219 → 220 ms (1.00) 211 → 222 ms (1.05) 236 → 235 ms (1.00) 171 → 170 ms (0.99) 174 → 175 ms (1.01) 175 → 175 ms (1.00) 190 → 191 ms (1.01)
clickbench_q17 228 → 228 ms (1.00) 235 → 224 ms (0.96) 237 → 237 ms (1.00) 168 → 162 ms (0.97) 172 → 176 ms (1.02) 182 → 179 ms (0.98) 185 → 189 ms (1.02)
clickbench_q18 381 → 377 ms (0.99) 388 → 384 ms (0.99) 422 → 425 ms (1.01) 305 → 306 ms (1.00) 304 → 294 ms (0.97) 318 → 314 ms (0.99) 348 → 351 ms (1.01)
clickbench_q19 45.2 → 46.4 ms (1.03) 36.3 → 39.6 ms (1.09) 76.1 → 76.0 ms (1.00) 33.2 → 36.7 ms (1.11) 🔴 32.4 → 34.3 ms (1.06) 27.0 → 26.2 ms (0.97) 31.3 → 37.1 ms (1.19) 🔴
clickbench_q20 233 → 232 ms (1.00) 218 → 227 ms (1.04) 277 → 285 ms (1.03) 157 → 160 ms (1.02) 140 → 148 ms (1.06) 232 → 221 ms (0.95) 459 → 310 ms (0.67) 🟢
clickbench_q21 275 → 279 ms (1.01) 263 → 255 ms (0.97) 291 → 303 ms (1.04) 175 → 179 ms (1.02) 163 → 178 ms (1.09) 263 → 265 ms (1.01) 429 → 316 ms (0.74) 🟢
clickbench_q22 379 → 382 ms (1.01) 388 → 373 ms (0.96) 417 → 431 ms (1.03) 289 → 296 ms (1.03) 313 → 324 ms (1.04) 373 → 353 ms (0.95) 469 → 327 ms (0.70) 🟢
clickbench_q23 391 → 354 ms (0.91) 267 → 262 ms (0.98) 1.18 → 1.20 s (1.02) 162 → 165 ms (1.02) 150 → 150 ms (1.00) 236 → 237 ms (1.00) 135 → 137 ms (1.01)
clickbench_q24 54.2 → 61.9 ms (1.14) 🔴 60.2 → 62.9 ms (1.04) 88.3 → 90.6 ms (1.03) 43.6 → 51.7 ms (1.19) 🔴 51.1 → 52.8 ms (1.03) 45.3 → 44.8 ms (0.99) 43.3 → 45.4 ms (1.05)
clickbench_q25 65.0 → 65.1 ms (1.00) 73.8 → 78.7 ms (1.07) 97.0 → 106 ms (1.09) 50.8 → 53.6 ms (1.06) 71.4 → 73.7 ms (1.03) 54.2 → 57.5 ms (1.06) 74.5 → 74.8 ms (1.00)
clickbench_q26 65.8 → 68.6 ms (1.04) 71.7 → 81.1 ms (1.13) 🔴 88.5 → 92.8 ms (1.05) 44.7 → 47.1 ms (1.05) 50.8 → 49.7 ms (0.98) 39.5 → 48.3 ms (1.22) 🔴 45.4 → 48.3 ms (1.06)
clickbench_q27 253 → 256 ms (1.01) 253 → 267 ms (1.05) 281 → 294 ms (1.04) 170 → 173 ms (1.02) 180 → 187 ms (1.04) 233 → 230 ms (0.99) 420 → 277 ms (0.66) 🟢
clickbench_q28 658 → 661 ms (1.01) 686 → 684 ms (1.00) 698 → 709 ms (1.02) 558 → 604 ms (1.08) 599 → 614 ms (1.02) 942 → 940 ms (1.00) 877 → 813 ms (0.93)
clickbench_q29 60.6 → 58.3 ms (0.96) 55.6 → 54.0 ms (0.97) 91.6 → 91.2 ms (1.00) 41.3 → 43.0 ms (1.04) 49.5 → 46.8 ms (0.95) 31.7 → 31.2 ms (0.98) 43.0 → 44.8 ms (1.04)
clickbench_q30 109 → 118 ms (1.08) 121 → 121 ms (1.00) 168 → 176 ms (1.04) 99.9 → 110 ms (1.10) 🔴 115 → 119 ms (1.03) 129 → 124 ms (0.96) 167 → 170 ms (1.02)
clickbench_q31 154 → 155 ms (1.01) 147 → 150 ms (1.02) 189 → 203 ms (1.07) 183 → 196 ms (1.07) 191 → 199 ms (1.04) 174 → 173 ms (0.99) 230 → 237 ms (1.03)
clickbench_q32 391 → 390 ms (1.00) 384 → 386 ms (1.01) 426 → 439 ms (1.03) 401 → 403 ms (1.00) 413 → 410 ms (0.99) 409 → 409 ms (1.00) 483 → 483 ms (1.00)
clickbench_q33 427 → 432 ms (1.01) 434 → 435 ms (1.00) 475 → 479 ms (1.01) 404 → 401 ms (0.99) 384 → 394 ms (1.03) 446 → 450 ms (1.01) 565 → 515 ms (0.91)
clickbench_q34 423 → 436 ms (1.03) 435 → 433 ms (0.99) 474 → 478 ms (1.01) 413 → 414 ms (1.00) 399 → 399 ms (1.00) 448 → 452 ms (1.01) 587 → 526 ms (0.90) 🟢
clickbench_q35 93.9 → 99.9 ms (1.06) 91.5 → 93.0 ms (1.02) 125 → 123 ms (0.99) 89.0 → 86.0 ms (0.97) 87.5 → 89.5 ms (1.02) 75.9 → 72.8 ms (0.96) 91.9 → 95.2 ms (1.04)
clickbench_q36 66.7 → 68.8 ms (1.03) 71.0 → 70.8 ms (1.00) 135 → 136 ms (1.01) 33.8 → 33.6 ms (1.00) 34.1 → 33.9 ms (0.99) 35.5 → 36.3 ms (1.02) 43.9 → 42.9 ms (0.98)
clickbench_q37 46.9 → 48.7 ms (1.04) 47.9 → 46.9 ms (0.98) 92.3 → 91.6 ms (0.99) 28.9 → 29.6 ms (1.02) 31.9 → 30.9 ms (0.97) 29.1 → 30.0 ms (1.03) 24.1 → 28.5 ms (1.18) 🔴
clickbench_q38 40.4 → 42.8 ms (1.06) 45.3 → 46.7 ms (1.03) 107 → 109 ms (1.02) 29.5 → 33.1 ms (1.12) 🔴 30.7 → 27.9 ms (0.91) 28.8 → 29.8 ms (1.03) 30.0 → 35.1 ms (1.17) 🔴
clickbench_q39 110 → 112 ms (1.01) 113 → 115 ms (1.02) 207 → 212 ms (1.03) 43.0 → 43.6 ms (1.01) 42.8 → 44.1 ms (1.03) 59.4 → 61.2 ms (1.03) 73.8 → 72.9 ms (0.99)
clickbench_q40 34.9 → 37.2 ms (1.07) 35.5 → 36.0 ms (1.01) 76.6 → 77.4 ms (1.01) 28.7 → 29.5 ms (1.03) 28.8 → 28.2 ms (0.98) 20.2 → 19.4 ms (0.96) 26.4 → 30.4 ms (1.15) 🔴
clickbench_q41 34.6 → 36.0 ms (1.04) 33.5 → 35.6 ms (1.06) 76.7 → 77.2 ms (1.01) 27.6 → 29.8 ms (1.08) 25.9 → 29.1 ms (1.12) 🔴 19.9 → 20.4 ms (1.03) 23.5 → 27.7 ms (1.18) 🔴
clickbench_q42 35.5 → 36.6 ms (1.03) 34.4 → 34.7 ms (1.01) 75.3 → 75.0 ms (1.00) 28.3 → 27.3 ms (0.96) 28.0 → 30.3 ms (1.08) 20.4 → 20.6 ms (1.01) 23.8 → 26.4 ms (1.11) 🔴

No file size changes detected.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-DS SF=1 on NVME 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.7%
Engines: DataFusion No clear signal (+0.4%, low confidence) · DuckDB No clear signal (+0.9%, low confidence)
Vortex (geomean): hot 1.004x ➖ · cold 1.025x ➖
Parquet (geomean): hot 0.995x ➖ · cold 1.013x ➖
Shifts: Parquet (control) -0.5% · Median polish +0.2%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 1.001x ➖ · 99 queries × 7 targets (11↑ 10↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 30.1 → 30.1 ms (1.00) 33.2 → 33.2 ms (1.00) 44.3 → 44.0 ms (0.99) 18.7 → 18.9 ms (1.01) 21.2 → 21.4 ms (1.01) 24.5 → 24.4 ms (1.00) 13.9 → 13.9 ms (1.00)
tpcds_q01 21.3 → 22.0 ms (1.03) 21.7 → 22.2 ms (1.02) 41.7 → 40.7 ms (0.98) 16.0 → 16.3 ms (1.02) 17.4 → 17.4 ms (1.00) 24.0 → 24.0 ms (1.00) 15.5 → 16.8 ms (1.09)
tpcds_q02 42.8 → 42.8 ms (1.00) 43.0 → 43.3 ms (1.01) 55.0 → 55.4 ms (1.01) 15.8 → 18.4 ms (1.17) 🔴 19.1 → 18.1 ms (0.94) 17.9 → 17.7 ms (0.99) 12.2 → 12.2 ms (1.00)
tpcds_q03 11.7 → 11.8 ms (1.01) 12.5 → 12.3 ms (0.99) 18.7 → 18.6 ms (1.00) 7.41 → 7.45 ms (1.00) 8.30 → 8.44 ms (1.02) 9.72 → 9.81 ms (1.01) 5.11 → 5.02 ms (0.98)
tpcds_q04 81.3 → 81.8 ms (1.01) 89.2 → 90.1 ms (1.01) 179 → 180 ms (1.01) 59.7 → 59.6 ms (1.00) 62.9 → 62.9 ms (1.00) 118 → 117 ms (0.99) 48.8 → 49.1 ms (1.01)
tpcds_q05 60.7 → 58.1 ms (0.96) 57.4 → 56.4 ms (0.98) 67.6 → 66.9 ms (0.99) 23.0 → 23.4 ms (1.02) 24.8 → 24.9 ms (1.00) 28.3 → 28.3 ms (1.00) 15.6 → 15.7 ms (1.01)
tpcds_q06 22.2 → 21.9 ms (0.98) 24.3 → 24.4 ms (1.00) 33.9 → 34.0 ms (1.00) 24.5 → 24.2 ms (0.99) 25.7 → 25.5 ms (0.99) 26.3 → 26.7 ms (1.01) 17.6 → 17.1 ms (0.97)
tpcds_q07 26.2 → 26.2 ms (1.00) 28.9 → 28.6 ms (0.99) 46.5 → 46.8 ms (1.01) 22.2 → 22.4 ms (1.01) 26.4 → 26.4 ms (1.00) 31.1 → 30.7 ms (0.99) 21.2 → 21.9 ms (1.03)
tpcds_q08 26.1 → 25.9 ms (0.99) 27.4 → 27.0 ms (0.99) 31.2 → 31.2 ms (1.00) 17.5 → 17.7 ms (1.01) 18.1 → 18.0 ms (0.99) 19.6 → 19.9 ms (1.01) 18.2 → 16.1 ms (0.89) 🟢
tpcds_q09 19.4 → 20.5 ms (1.05) 21.5 → 22.3 ms (1.03) 30.5 → 30.9 ms (1.01) 19.1 → 19.7 ms (1.03) 20.5 → 20.6 ms (1.01) 27.0 → 27.4 ms (1.02) 8.58 → 9.08 ms (1.06)
tpcds_q10 30.9 → 30.1 ms (0.97) 35.2 → 35.2 ms (1.00) 43.3 → 44.5 ms (1.03) 28.1 → 28.2 ms (1.00) 31.5 → 31.9 ms (1.01) 32.7 → 32.7 ms (1.00) 17.8 → 17.7 ms (1.00)
tpcds_q11 49.2 → 48.8 ms (0.99) 53.2 → 53.2 ms (1.00) 96.2 → 96.6 ms (1.00) 36.6 → 36.1 ms (0.99) 37.9 → 38.3 ms (1.01) 68.4 → 67.9 ms (0.99) 42.0 → 42.4 ms (1.01)
tpcds_q12 15.0 → 15.3 ms (1.02) 19.4 → 19.5 ms (1.01) 23.7 → 23.5 ms (0.99) 10.3 → 10.3 ms (1.00) 13.1 → 13.4 ms (1.02) 14.0 → 14.2 ms (1.02) 8.61 → 8.86 ms (1.03)
tpcds_q13 31.1 → 31.6 ms (1.02) 45.2 → 46.3 ms (1.02) 47.3 → 46.1 ms (0.97) 20.7 → 20.5 ms (0.99) 34.0 → 34.2 ms (1.01) 31.0 → 31.4 ms (1.01) 20.5 → 18.5 ms (0.90)
tpcds_q14 490 → 490 ms (1.00) 655 → 652 ms (1.00) 533 → 534 ms (1.00) 56.5 → 56.9 ms (1.01) 59.0 → 57.6 ms (0.98) 66.3 → 65.3 ms (0.98) 39.9 → 40.0 ms (1.00)
tpcds_q15 12.4 → 12.6 ms (1.02) 13.5 → 13.6 ms (1.01) 23.6 → 20.4 ms (0.86) 🟢 11.3 → 11.2 ms (1.00) 12.6 → 12.9 ms (1.02) 16.6 → 16.1 ms (0.97) 8.26 → 10.0 ms (1.21) 🔴
tpcds_q16 18.9 → 18.7 ms (0.99) 20.6 → 21.0 ms (1.02) 37.3 → 36.8 ms (0.99) 24.3 → 24.5 ms (1.01) 26.4 → 27.6 ms (1.05) 30.2 → 29.8 ms (0.99) 12.6 → 11.9 ms (0.95)
tpcds_q17 53.1 → 53.4 ms (1.01) 56.5 → 57.5 ms (1.02) 75.2 → 74.1 ms (0.99) 19.4 → 20.1 ms (1.03) 23.9 → 23.7 ms (0.99) 34.5 → 34.7 ms (1.01) 17.7 → 19.6 ms (1.10) 🔴
tpcds_q18 63.6 → 63.5 ms (1.00) 63.2 → 57.0 ms (0.90) 69.3 → 66.8 ms (0.96) 31.9 → 32.6 ms (1.02) 39.3 → 38.2 ms (0.97) 47.0 → 46.9 ms (1.00) 32.0 → 31.7 ms (0.99)
tpcds_q19 17.1 → 17.1 ms (1.00) 19.6 → 19.9 ms (1.01) 29.5 → 29.3 ms (0.99) 20.8 → 21.1 ms (1.01) 23.8 → 24.2 ms (1.01) 18.8 → 20.3 ms (1.08) 10.5 → 10.5 ms (1.00)
tpcds_q20 17.0 → 16.7 ms (0.98) 21.2 → 20.6 ms (0.97) 24.9 → 22.7 ms (0.91) 10.8 → 11.1 ms (1.03) 14.0 → 13.9 ms (0.99) 14.9 → 14.4 ms (0.97) 9.08 → 9.38 ms (1.03)
tpcds_q21 15.2 → 15.7 ms (1.03) 17.5 → 17.7 ms (1.01) 20.1 → 20.5 ms (1.02) 10.3 → 10.9 ms (1.06) 12.4 → 12.4 ms (1.00) 10.3 → 10.4 ms (1.01) 5.96 → 6.28 ms (1.05)
tpcds_q22 49.3 → 49.7 ms (1.01) 50.0 → 51.4 ms (1.03) 75.9 → 77.9 ms (1.03) 18.2 → 18.3 ms (1.00) 24.9 → 20.4 ms (0.82) 🟢 20.8 → 20.8 ms (1.00) 17.5 → 17.6 ms (1.01)
tpcds_q23 72.2 → 72.7 ms (1.01) 75.2 → 74.1 ms (0.98) 91.3 → 93.9 ms (1.03) 37.4 → 38.0 ms (1.02) 41.9 → 42.7 ms (1.02) 42.9 → 45.3 ms (1.06) 26.7 → 27.0 ms (1.01)
tpcds_q24 63.2 → 63.6 ms (1.01) 69.3 → 69.6 ms (1.00) 90.2 → 90.6 ms (1.01) 26.3 → 26.3 ms (1.00) 28.1 → 28.4 ms (1.01) 30.6 → 29.7 ms (0.97) 15.1 → 13.6 ms (0.91)
tpcds_q25 44.9 → 45.4 ms (1.01) 50.0 → 49.8 ms (1.00) 77.2 → 76.2 ms (0.99) 19.2 → 19.1 ms (1.00) 22.0 → 22.3 ms (1.01) 30.3 → 30.0 ms (0.99) 14.0 → 14.5 ms (1.04)
tpcds_q26 26.7 → 26.9 ms (1.01) 28.6 → 28.4 ms (0.99) 43.1 → 43.0 ms (1.00) 21.8 → 22.0 ms (1.01) 26.7 → 27.0 ms (1.01) 28.7 → 28.5 ms (0.99) 18.0 → 17.9 ms (1.00)
tpcds_q27 51.3 → 52.7 ms (1.03) 54.0 → 52.9 ms (0.98) 74.2 → 77.2 ms (1.04) 24.9 → 25.0 ms (1.01) 29.2 → 29.3 ms (1.00) 34.8 → 32.8 ms (0.94) 23.1 → 23.9 ms (1.03)
tpcds_q28 19.9 → 19.5 ms (0.98) 23.8 → 23.3 ms (0.98) 28.4 → 27.9 ms (0.98) 14.9 → 15.3 ms (1.03) 20.6 → 20.8 ms (1.01) 25.9 → 25.4 ms (0.98) 11.9 → 12.9 ms (1.09)
tpcds_q29 52.9 → 52.6 ms (0.99) 55.8 → 57.8 ms (1.04) 75.3 → 76.5 ms (1.02) 18.5 → 18.3 ms (0.99) 22.3 → 22.1 ms (0.99) 33.6 → 32.5 ms (0.96) 13.9 → 14.6 ms (1.05)
tpcds_q30 27.8 → 27.4 ms (0.99) 29.3 → 29.9 ms (1.02) 45.0 → 44.3 ms (0.98) 18.6 → 18.6 ms (1.00) 21.7 → 22.0 ms (1.02) 35.7 → 34.7 ms (0.97) 17.6 → 19.0 ms (1.08)
tpcds_q31 64.4 → 63.5 ms (0.99) 68.2 → 67.8 ms (1.00) 81.5 → 79.6 ms (0.98) 18.2 → 18.7 ms (1.03) 20.2 → 20.0 ms (0.99) 35.6 → 35.5 ms (1.00) 30.1 → 27.5 ms (0.92)
tpcds_q32 12.9 → 13.0 ms (1.01) 14.0 → 13.9 ms (0.99) 26.2 → 23.8 ms (0.91) 10.2 → 10.5 ms (1.03) 11.1 → 11.1 ms (1.00) 14.1 → 13.2 ms (0.93) 5.30 → 5.46 ms (1.03)
tpcds_q33 29.0 → 28.6 ms (0.99) 29.5 → 29.4 ms (1.00) 38.3 → 38.7 ms (1.01) 18.4 → 18.9 ms (1.03) 20.5 → 20.0 ms (0.98) 23.3 → 23.5 ms (1.01) 8.95 → 10.1 ms (1.12) 🔴
tpcds_q34 17.9 → 18.0 ms (1.00) 20.2 → 19.8 ms (0.98) 24.2 → 24.6 ms (1.02) 12.7 → 12.9 ms (1.02) 14.2 → 14.8 ms (1.04) 19.9 → 17.0 ms (0.85) 🟢 10.5 → 8.82 ms (0.84) 🟢
tpcds_q35 31.4 → 31.1 ms (0.99) 38.1 → 38.1 ms (1.00) 50.7 → 50.6 ms (1.00) 30.6 → 31.2 ms (1.02) 33.4 → 34.2 ms (1.02) 35.1 → 34.9 ms (1.00) 24.3 → 23.1 ms (0.95)
tpcds_q36 47.1 → 47.9 ms (1.02) 51.2 → 52.1 ms (1.02) 53.5 → 52.6 ms (0.98) 14.4 → 14.6 ms (1.02) 15.6 → 15.7 ms (1.00) 39.5 → 39.6 ms (1.00) 12.7 → 12.6 ms (0.99)
tpcds_q37 13.3 → 12.7 ms (0.96) 15.1 → 14.0 ms (0.93) 21.0 → 21.3 ms (1.01) 9.77 → 9.83 ms (1.01) 13.0 → 12.9 ms (0.99) 12.5 → 12.7 ms (1.02) 5.62 → 6.31 ms (1.12) 🔴
tpcds_q38 26.6 → 26.6 ms (1.00) 28.1 → 27.7 ms (0.98) 45.2 → 44.8 ms (0.99) 24.5 → 24.5 ms (1.00) 25.4 → 25.6 ms (1.01) 29.5 → 30.0 ms (1.01) 22.4 → 22.1 ms (0.99)
tpcds_q39 66.0 → 64.0 ms (0.97) 67.3 → 67.1 ms (1.00) 68.5 → 68.3 ms (1.00) 19.0 → 19.0 ms (1.00) 20.5 → 20.2 ms (0.99) 23.3 → 23.2 ms (1.00) 19.3 → 19.2 ms (1.00)
tpcds_q40 23.4 → 23.1 ms (0.98) 23.9 → 23.9 ms (1.00) 31.0 → 27.4 ms (0.88) 🟢 11.3 → 11.3 ms (1.00) 12.7 → 12.6 ms (1.00) 15.8 → 15.5 ms (0.98) 8.30 → 8.44 ms (1.02)
tpcds_q41 20.3 → 19.5 ms (0.96) 21.0 → 20.6 ms (0.98) 20.2 → 20.1 ms (1.00) 17.3 → 17.5 ms (1.02) 19.4 → 19.3 ms (1.00) 11.8 → 12.0 ms (1.01) 10.5 → 9.79 ms (0.93)
tpcds_q42 11.6 → 11.6 ms (1.01) 12.2 → 12.0 ms (0.98) 18.2 → 17.9 ms (0.98) 7.26 → 7.38 ms (1.02) 8.10 → 8.30 ms (1.03) 9.53 → 9.45 ms (0.99) 4.79 → 4.84 ms (1.01)
tpcds_q43 13.2 → 13.2 ms (1.00) 14.7 → 14.2 ms (0.97) 19.8 → 19.5 ms (0.98) 10.2 → 9.90 ms (0.97) 10.9 → 11.0 ms (1.01) 12.0 → 12.1 ms (1.01) 7.40 → 7.37 ms (1.00)
tpcds_q44 26.9 → 27.2 ms (1.01) 27.7 → 26.7 ms (0.96) 35.9 → 34.8 ms (0.97) 12.7 → 12.9 ms (1.01) 14.8 → 14.8 ms (1.00) 21.3 → 20.3 ms (0.95) 9.21 → 9.78 ms (1.06)
tpcds_q45 18.0 → 17.3 ms (0.96) 18.0 → 19.1 ms (1.06) 27.7 → 28.2 ms (1.02) 23.7 → 24.1 ms (1.02) 25.2 → 25.6 ms (1.02) 21.1 → 20.9 ms (0.99) 13.4 → 12.7 ms (0.95)
tpcds_q46 23.3 → 23.7 ms (1.01) 26.1 → 26.5 ms (1.02) 29.0 → 28.8 ms (0.99) 31.1 → 31.2 ms (1.00) 33.0 → 32.8 ms (0.99) 36.8 → 35.7 ms (0.97) 25.0 → 23.8 ms (0.95)
tpcds_q47 76.3 → 76.3 ms (1.00) 80.3 → 81.0 ms (1.01) 97.1 → 97.5 ms (1.00) 31.1 → 31.6 ms (1.02) 32.6 → 34.6 ms (1.06) 34.5 → 34.6 ms (1.00) 33.5 → 33.3 ms (1.00)
tpcds_q48 28.9 → 29.0 ms (1.00) 42.4 → 43.0 ms (1.01) 41.6 → 42.7 ms (1.03) 23.0 → 22.7 ms (0.99) 35.3 → 35.5 ms (1.01) 30.2 → 29.8 ms (0.99) 27.5 → 18.0 ms (0.65) 🟢
tpcds_q49 57.4 → 60.8 ms (1.06) 61.7 → 57.8 ms (0.94) 72.3 → 70.9 ms (0.98) 18.5 → 19.2 ms (1.04) 21.6 → 22.9 ms (1.06) 29.1 → 29.0 ms (1.00) 14.4 → 14.6 ms (1.01)
tpcds_q50 30.9 → 30.7 ms (1.00) 32.7 → 32.8 ms (1.01) 46.3 → 47.1 ms (1.02) 14.8 → 15.2 ms (1.03) 16.3 → 16.5 ms (1.01) 19.6 → 19.5 ms (0.99) 9.86 → 11.5 ms (1.17) 🔴
tpcds_q51 67.3 → 67.0 ms (1.00) 68.4 → 67.6 ms (0.99) 74.7 → 74.5 ms (1.00) 49.0 → 49.2 ms (1.00) 50.1 → 50.1 ms (1.00) 52.6 → 52.1 ms (0.99) 46.8 → 46.7 ms (1.00)
tpcds_q52 12.2 → 12.1 ms (0.99) 12.4 → 13.2 ms (1.06) 18.7 → 18.3 ms (0.98) 7.47 → 7.58 ms (1.01) 8.46 → 8.39 ms (0.99) 9.58 → 9.69 ms (1.01) 5.05 → 5.12 ms (1.01)
tpcds_q53 18.8 → 18.6 ms (0.99) 21.9 → 22.0 ms (1.00) 22.5 → 22.7 ms (1.01) 11.7 → 11.8 ms (1.01) 13.4 → 13.7 ms (1.02) 13.8 → 14.3 ms (1.04) 9.40 → 8.46 ms (0.90)
tpcds_q54 32.1 → 32.3 ms (1.00) 34.6 → 33.6 ms (0.97) 54.2 → 52.3 ms (0.97) 19.8 → 20.0 ms (1.01) 21.0 → 21.3 ms (1.01) 24.8 → 24.6 ms (0.99) 12.1 → 11.4 ms (0.95)
tpcds_q55 11.8 → 11.5 ms (0.98) 11.9 → 11.8 ms (0.99) 18.2 → 18.2 ms (1.00) 7.24 → 7.33 ms (1.01) 7.95 → 8.09 ms (1.02) 9.43 → 9.42 ms (1.00) 4.74 → 4.89 ms (1.03)
tpcds_q56 26.9 → 26.8 ms (1.00) 27.7 → 27.5 ms (0.99) 40.6 → 41.7 ms (1.03) 18.0 → 18.3 ms (1.01) 19.6 → 20.2 ms (1.03) 26.7 → 26.2 ms (0.98) 10.1 → 11.9 ms (1.17) 🔴
tpcds_q57 62.3 → 63.0 ms (1.01) 67.3 → 67.5 ms (1.00) 101 → 102 ms (1.01) 25.8 → 26.0 ms (1.01) 26.9 → 27.2 ms (1.01) 30.4 → 30.7 ms (1.01) 27.6 → 27.8 ms (1.01)
tpcds_q58 41.8 → 40.9 ms (0.98) 45.0 → 45.0 ms (1.00) 65.2 → 65.4 ms (1.00) 20.7 → 21.0 ms (1.01) 21.3 → 21.7 ms (1.02) 24.6 → 23.8 ms (0.97) 10.7 → 10.7 ms (1.00)
tpcds_q59 27.6 → 27.4 ms (0.99) 29.9 → 29.8 ms (1.00) 49.5 → 48.5 ms (0.98) 19.7 → 19.9 ms (1.01) 20.8 → 22.2 ms (1.07) 22.5 → 22.8 ms (1.02) 16.8 → 16.6 ms (0.99)
tpcds_q60 26.7 → 26.9 ms (1.01) 27.2 → 28.0 ms (1.03) 42.2 → 40.8 ms (0.97) 18.6 → 19.6 ms (1.05) 23.3 → 21.6 ms (0.93) 27.6 → 27.8 ms (1.01) 11.0 → 11.7 ms (1.07)
tpcds_q61 21.0 → 21.1 ms (1.00) 24.9 → 24.6 ms (0.99) 35.5 → 35.6 ms (1.00) 19.9 → 20.1 ms (1.01) 21.8 → 22.0 ms (1.01) 24.2 → 24.7 ms (1.02) 9.92 → 10.5 ms (1.05)
tpcds_q62 16.4 → 16.7 ms (1.02) 16.9 → 17.0 ms (1.00) 25.3 → 25.5 ms (1.01) 11.1 → 11.6 ms (1.04) 13.0 → 13.1 ms (1.00) 12.8 → 12.9 ms (1.01) 8.20 → 8.32 ms (1.01)
tpcds_q63 18.4 → 18.5 ms (1.00) 22.3 → 21.7 ms (0.97) 22.2 → 22.5 ms (1.01) 11.2 → 11.3 ms (1.01) 12.9 → 13.1 ms (1.02) 13.4 → 13.7 ms (1.03) 8.98 → 7.77 ms (0.87) 🟢
tpcds_q64 190 → 190 ms (1.00) 283 → 284 ms (1.00) 288 → 288 ms (1.00) 53.1 → 53.3 ms (1.00) 58.6 → 59.8 ms (1.02) 70.4 → 70.5 ms (1.00) 38.0 → 37.8 ms (0.99)
tpcds_q65 74.5 → 74.0 ms (0.99) 76.8 → 77.4 ms (1.01) 120 → 122 ms (1.02) 13.3 → 13.4 ms (1.01) 14.3 → 14.5 ms (1.02) 19.2 → 19.7 ms (1.03) 9.57 → 9.72 ms (1.02)
tpcds_q66 50.1 → 50.0 ms (1.00) 51.0 → 52.7 ms (1.03) 61.0 → 59.3 ms (0.97) 31.8 → 31.3 ms (0.99) 35.1 → 38.1 ms (1.08) 33.0 → 33.1 ms (1.00) 24.3 → 24.4 ms (1.01)
tpcds_q67 51.8 → 51.1 ms (0.99) 53.4 → 53.2 ms (1.00) 68.6 → 69.1 ms (1.01) 55.6 → 56.2 ms (1.01) 57.2 → 66.9 ms (1.17) 🔴 61.1 → 61.1 ms (1.00) 54.9 → 55.4 ms (1.01)
tpcds_q68 23.0 → 23.3 ms (1.02) 27.2 → 27.0 ms (0.99) 28.9 → 29.3 ms (1.01) 22.3 → 22.6 ms (1.01) 24.7 → 24.6 ms (1.00) 29.5 → 29.2 ms (0.99) 16.4 → 14.8 ms (0.90)
tpcds_q69 27.1 → 28.5 ms (1.05) 34.9 → 33.9 ms (0.97) 43.6 → 43.8 ms (1.00) 22.7 → 23.4 ms (1.03) 26.9 → 27.2 ms (1.01) 26.0 → 25.7 ms (0.99) 14.9 → 14.9 ms (1.00)
tpcds_q70 73.6 → 73.8 ms (1.00) 74.7 → 74.2 ms (0.99) 80.5 → 80.6 ms (1.00) 15.0 → 15.0 ms (1.00) 15.9 → 15.7 ms (0.99) 19.4 → 19.3 ms (0.99) 9.97 → 10.3 ms (1.03)
tpcds_q71 19.9 → 20.8 ms (1.05) 19.7 → 20.1 ms (1.02) 27.1 → 26.4 ms (0.98) 13.8 → 13.4 ms (0.97) 15.0 → 14.4 ms (0.96) 17.0 → 17.0 ms (1.00) 7.88 → 8.08 ms (1.03)
tpcds_q72 261 → 262 ms (1.01) 272 → 271 ms (1.00) 530 → 470 ms (0.89) 🟢 43.1 → 43.6 ms (1.01) 48.7 → 49.0 ms (1.01) 52.7 → 52.1 ms (0.99) 35.8 → 35.7 ms (1.00)
tpcds_q73 16.8 → 16.7 ms (0.99) 20.1 → 19.8 ms (0.99) 23.7 → 23.8 ms (1.00) 11.6 → 11.8 ms (1.02) 12.9 → 13.2 ms (1.03) 15.4 → 14.7 ms (0.95) 7.91 → 7.54 ms (0.95)
tpcds_q74 40.6 → 41.2 ms (1.01) 42.7 → 42.7 ms (1.00) 70.2 → 70.7 ms (1.01) 31.4 → 31.5 ms (1.00) 32.9 → 33.5 ms (1.02) 84.5 → 83.7 ms (0.99) 35.8 → 34.9 ms (0.97)
tpcds_q75 139 → 136 ms (0.97) 137 → 142 ms (1.04) 153 → 152 ms (0.99) 37.5 → 38.1 ms (1.02) 40.4 → 40.1 ms (0.99) 44.6 → 44.0 ms (0.99) 30.0 → 30.3 ms (1.01)
tpcds_q76 29.6 → 30.2 ms (1.02) 29.7 → 30.6 ms (1.03) 33.8 → 33.8 ms (1.00) 13.2 → 13.3 ms (1.01) 13.6 → 13.8 ms (1.01) 17.0 → 19.4 ms (1.14) 🔴 8.11 → 8.13 ms (1.00)
tpcds_q77 45.6 → 45.7 ms (1.00) 46.7 → 46.4 ms (0.99) 56.5 → 58.2 ms (1.03) 21.4 → 21.4 ms (1.00) 23.3 → 23.7 ms (1.02) 26.6 → 25.8 ms (0.97) 10.4 → 10.4 ms (1.00)
tpcds_q78 60.5 → 60.7 ms (1.00) 63.6 → 64.2 ms (1.01) 87.0 → 86.8 ms (1.00) 43.6 → 43.4 ms (1.00) 47.3 → 47.4 ms (1.00) 77.8 → 78.6 ms (1.01) 65.1 → 65.1 ms (1.00)
tpcds_q79 20.0 → 20.3 ms (1.01) 23.3 → 23.3 ms (1.00) 26.7 → 27.1 ms (1.02) 16.4 → 16.5 ms (1.01) 17.5 → 18.0 ms (1.03) 23.6 → 24.1 ms (1.02) 12.1 → 11.8 ms (0.98)
tpcds_q80 69.7 → 68.8 ms (0.99) 72.2 → 71.0 ms (0.98) 80.7 → 81.4 ms (1.01) 29.5 → 30.0 ms (1.02) 32.8 → 32.5 ms (0.99) 37.8 → 35.4 ms (0.94) 16.8 → 17.0 ms (1.01)
tpcds_q81 27.6 → 27.7 ms (1.00) 29.6 → 29.2 ms (0.99) 37.0 → 37.9 ms (1.03) 21.4 → 21.4 ms (1.00) 23.9 → 24.0 ms (1.00) 36.3 → 35.1 ms (0.97) 25.5 → 22.3 ms (0.88) 🟢
tpcds_q82 12.8 → 12.6 ms (0.98) 15.6 → 14.9 ms (0.95) 22.4 → 22.8 ms (1.02) 10.1 → 9.98 ms (0.98) 13.6 → 14.3 ms (1.06) 13.4 → 13.6 ms (1.02) 6.01 → 6.98 ms (1.16) 🔴
tpcds_q83 35.5 → 35.3 ms (1.00) 36.5 → 36.3 ms (0.99) 58.6 → 58.3 ms (0.99) 17.5 → 17.9 ms (1.02) 18.6 → 18.9 ms (1.02) 20.3 → 20.1 ms (0.99) 8.90 → 9.04 ms (1.01)
tpcds_q84 11.0 → 10.7 ms (0.97) 12.9 → 12.4 ms (0.96) 25.8 → 26.1 ms (1.01) 15.6 → 15.9 ms (1.02) 17.8 → 17.9 ms (1.00) 22.8 → 22.4 ms (0.98) 9.22 → 8.44 ms (0.92)
tpcds_q85 52.7 → 52.3 ms (0.99) 95.8 → 95.6 ms (1.00) 79.7 → 80.1 ms (1.01) 25.3 → 25.5 ms (1.01) 39.1 → 39.4 ms (1.01) 32.2 → 31.8 ms (0.99) 21.4 → 18.2 ms (0.85) 🟢
tpcds_q86 16.6 → 16.5 ms (0.99) 16.7 → 16.5 ms (0.99) 20.4 → 21.3 ms (1.04) 10.0 → 10.3 ms (1.02) 11.1 → 11.2 ms (1.01) 11.7 → 11.8 ms (1.00) 8.49 → 8.43 ms (0.99)
tpcds_q87 26.9 → 26.4 ms (0.98) 27.4 → 27.8 ms (1.01) 45.7 → 44.5 ms (0.97) 26.0 → 26.4 ms (1.02) 27.0 → 27.2 ms (1.01) 31.3 → 31.4 ms (1.00) 24.7 → 24.0 ms (0.97)
tpcds_q88 33.3 → 33.1 ms (0.99) 45.4 → 45.3 ms (1.00) 37.4 → 37.0 ms (0.99) 30.3 → 31.2 ms (1.03) 31.1 → 32.2 ms (1.04) 38.9 → 39.1 ms (1.00) 18.8 → 19.0 ms (1.01)
tpcds_q89 21.6 → 21.8 ms (1.01) 23.6 → 23.9 ms (1.01) 25.4 → 25.8 ms (1.01) 12.3 → 12.3 ms (1.00) 13.8 → 14.9 ms (1.08) 14.3 → 14.4 ms (1.01) 10.3 → 9.45 ms (0.92)
tpcds_q90 11.1 → 11.2 ms (1.01) 12.0 → 11.7 ms (0.97) 15.9 → 15.8 ms (1.00) 7.93 → 8.12 ms (1.02) 8.32 → 8.52 ms (1.02) 10.0 → 10.2 ms (1.02) 4.94 → 4.93 ms (1.00)
tpcds_q91 16.3 → 16.7 ms (1.02) 26.0 → 25.8 ms (0.99) 31.8 → 31.4 ms (0.99) 14.9 → 15.3 ms (1.02) 23.7 → 23.8 ms (1.00) 21.0 → 20.6 ms (0.98) 8.64 → 9.44 ms (1.09)
tpcds_q92 13.4 → 13.3 ms (0.99) 14.8 → 14.8 ms (1.00) 26.9 → 26.8 ms (1.00) 10.9 → 10.9 ms (1.00) 11.6 → 11.9 ms (1.02) 15.6 → 15.1 ms (0.97) 6.69 → 6.99 ms (1.04)
tpcds_q93 20.3 → 20.0 ms (0.99) 21.1 → 20.7 ms (0.98) 27.3 → 26.8 ms (0.98) 9.80 → 9.51 ms (0.97) 11.8 → 11.7 ms (1.00) 14.2 → 13.9 ms (0.98) 6.98 → 7.11 ms (1.02)
tpcds_q94 19.2 → 19.3 ms (1.00) 19.8 → 20.3 ms (1.03) 35.2 → 35.1 ms (1.00) 22.1 → 22.2 ms (1.01) 24.5 → 25.3 ms (1.03) 25.4 → 25.1 ms (0.99) 11.9 → 11.8 ms (0.99)
tpcds_q95 41.5 → 42.2 ms (1.02) 43.3 → 43.1 ms (1.00) 64.9 → 63.5 ms (0.98) 59.6 → 59.9 ms (1.01) 62.6 → 62.9 ms (1.00) 43.6 → 43.7 ms (1.00) 36.1 → 34.1 ms (0.95)
tpcds_q96 7.89 → 7.86 ms (1.00) 10.4 → 10.4 ms (0.99) 12.8 → 12.8 ms (1.00) 6.71 → 6.91 ms (1.03) 8.22 → 7.71 ms (0.94) 8.21 → 8.36 ms (1.02) 3.79 → 3.71 ms (0.98)
tpcds_q97 17.8 → 17.9 ms (1.01) 18.2 → 18.2 ms (1.00) 35.2 → 35.6 ms (1.01) 23.3 → 23.2 ms (1.00) 23.8 → 23.7 ms (1.00) 27.4 → 27.5 ms (1.00) 21.5 → 21.5 ms (1.00)
tpcds_q98 16.4 → 16.4 ms (1.00) 20.4 → 20.7 ms (1.01) 24.6 → 24.7 ms (1.01) 11.5 → 11.7 ms (1.02) 14.3 → 14.6 ms (1.02) 16.5 → 16.5 ms (1.00) 9.94 → 10.3 ms (1.03)
tpcds_q99 16.6 → 16.4 ms (0.99) 17.6 → 17.7 ms (1.00) 29.0 → 30.0 ms (1.03) 14.7 → 14.5 ms (0.99) 16.3 → 16.6 ms (1.02) 18.7 → 19.1 ms (1.02) 12.9 → 12.7 ms (0.99)
Cold runs (first iteration): geomean 1.023x ➖ · 99 queries × 7 targets (6↑ 28↓)
query datafusion:vortex [file, v2] datafusion:vortex-compact [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:vortex-compact [file, v2] duckdb:parquet duckdb:duckdb
geomean 52.1 → 53.7 ms (1.03) 55.4 → 57.4 ms (1.04) 60.7 → 61.6 ms (1.02) 29.3 → 29.9 ms (1.02) 31.6 → 32.1 ms (1.01) 31.3 → 31.6 ms (1.01) 22.6 → 23.4 ms (1.03)
tpcds_q01 39.5 → 40.4 ms (1.02) 37.3 → 41.3 ms (1.11) 🔴 60.3 → 59.4 ms (0.98) 24.5 → 25.2 ms (1.03) 26.5 → 26.5 ms (1.00) 30.8 → 31.0 ms (1.01) 24.5 → 25.9 ms (1.06)
tpcds_q02 63.2 → 63.6 ms (1.01) 62.7 → 63.3 ms (1.01) 68.3 → 71.1 ms (1.04) 23.2 → 23.2 ms (1.00) 24.2 → 23.6 ms (0.97) 23.9 → 24.6 ms (1.03) 19.6 → 18.0 ms (0.92)
tpcds_q03 28.7 → 28.8 ms (1.00) 31.3 → 32.3 ms (1.03) 30.7 → 30.7 ms (1.00) 16.2 → 16.6 ms (1.03) 16.7 → 17.4 ms (1.05) 15.4 → 15.8 ms (1.02) 11.4 → 12.4 ms (1.08)
tpcds_q04 112 → 114 ms (1.02) 118 → 120 ms (1.02) 208 → 205 ms (0.99) 70.0 → 70.6 ms (1.01) 75.0 → 75.5 ms (1.01) 125 → 124 ms (0.99) 56.2 → 57.2 ms (1.02)
tpcds_q05 83.5 → 78.7 ms (0.94) 78.4 → 84.6 ms (1.08) 87.6 → 91.1 ms (1.04) 34.0 → 32.7 ms (0.96) 34.8 → 34.8 ms (1.00) 33.0 → 32.9 ms (1.00) 21.0 → 23.3 ms (1.11) 🔴
tpcds_q06 41.9 → 41.2 ms (0.98) 43.5 → 45.4 ms (1.04) 47.5 → 46.7 ms (0.98) 37.7 → 38.8 ms (1.03) 40.4 → 40.3 ms (1.00) 35.4 → 36.0 ms (1.02) 27.4 → 27.9 ms (1.02)
tpcds_q07 46.7 → 50.5 ms (1.08) 46.7 → 50.0 ms (1.07) 64.4 → 66.0 ms (1.03) 34.6 → 35.4 ms (1.02) 38.2 → 38.9 ms (1.02) 38.6 → 38.7 ms (1.00) 30.0 → 32.1 ms (1.07)
tpcds_q08 42.8 → 46.1 ms (1.08) 46.1 → 47.7 ms (1.03) 47.2 → 48.6 ms (1.03) 25.9 → 26.2 ms (1.01) 26.1 → 26.1 ms (1.00) 24.8 → 26.4 ms (1.07) 27.2 → 26.1 ms (0.96)
tpcds_q09 36.5 → 37.1 ms (1.02) 38.9 → 39.6 ms (1.02) 36.8 → 42.2 ms (1.15) 🔴 29.2 → 30.3 ms (1.04) 31.1 → 31.1 ms (1.00) 30.0 → 30.5 ms (1.02) 13.3 → 14.0 ms (1.05)
tpcds_q10 52.5 → 52.0 ms (0.99) 60.6 → 59.4 ms (0.98) 56.3 → 55.9 ms (0.99) 41.7 → 42.9 ms (1.03) 45.8 → 46.0 ms (1.00) 39.8 → 41.2 ms (1.03) 27.3 → 27.8 ms (1.02)
tpcds_q11 74.7 → 78.9 ms (1.06) 76.9 → 79.3 ms (1.03) 118 → 118 ms (1.00) 45.8 → 45.6 ms (1.00) 48.5 → 49.0 ms (1.01) 76.0 → 75.3 ms (0.99) 50.2 → 50.7 ms (1.01)
tpcds_q12 32.7 → 33.4 ms (1.02) 37.0 → 38.3 ms (1.04) 35.9 → 37.1 ms (1.03) 17.7 → 17.6 ms (1.00) 20.9 → 21.1 ms (1.01) 18.8 → 19.6 ms (1.04) 15.8 → 16.7 ms (1.06)
tpcds_q13 56.4 → 58.2 ms (1.03) 63.8 → 68.9 ms (1.08) 62.7 → 66.2 ms (1.06) 32.6 → 35.3 ms (1.08) 45.0 → 46.0 ms (1.02) 39.3 → 39.4 ms (1.00) 31.6 → 29.8 ms (0.94)
tpcds_q14 549 → 542 ms (0.99) 712 → 708 ms (1.00) 582 → 586 ms (1.01) 67.0 → 67.0 ms (1.00) 69.4 → 72.3 ms (1.04) 67.8 → 68.8 ms (1.02) 46.1 → 45.2 ms (0.98)
tpcds_q15 31.0 → 31.4 ms (1.02) 30.9 → 32.5 ms (1.05) 36.3 → 34.7 ms (0.96) 17.9 → 18.3 ms (1.02) 20.3 → 20.8 ms (1.03) 22.3 → 22.3 ms (1.00) 13.9 → 18.0 ms (1.29) 🔴
tpcds_q16 36.3 → 38.4 ms (1.06) 39.1 → 40.0 ms (1.02) 50.7 → 50.5 ms (1.00) 34.3 → 35.7 ms (1.04) 36.6 → 37.5 ms (1.02) 36.8 → 37.1 ms (1.01) 21.0 → 20.9 ms (1.00)
tpcds_q17 75.1 → 76.2 ms (1.01) 78.7 → 78.9 ms (1.00) 95.1 → 96.2 ms (1.01) 32.2 → 33.4 ms (1.04) 35.5 → 36.3 ms (1.02) 39.9 → 40.4 ms (1.01) 28.0 → 31.4 ms (1.12) 🔴
tpcds_q18 88.2 → 71.8 ms (0.81) 🟢 72.0 → 86.7 ms (1.20) 🔴 92.8 → 88.2 ms (0.95) 43.7 → 45.5 ms (1.04) 50.0 → 50.0 ms (1.00) 55.1 → 54.4 ms (0.99) 44.4 → 45.9 ms (1.03)
tpcds_q19 37.0 → 38.7 ms (1.05) 40.8 → 42.2 ms (1.04) 42.2 → 43.2 ms (1.02) 29.5 → 29.9 ms (1.01) 30.5 → 31.6 ms (1.03) 25.3 → 26.1 ms (1.03) 19.1 → 20.7 ms (1.08)
tpcds_q20 34.0 → 34.4 ms (1.01) 38.3 → 40.1 ms (1.05) 40.0 → 34.0 ms (0.85) 🟢 18.4 → 18.3 ms (0.99) 21.6 → 22.0 ms (1.02) 19.6 → 19.1 ms (0.97) 16.0 → 16.6 ms (1.04)
tpcds_q21 32.9 → 36.2 ms (1.10) 34.8 → 36.7 ms (1.05) 30.5 → 31.6 ms (1.04) 22.8 → 23.8 ms (1.04) 24.1 → 24.6 ms (1.02) 16.5 → 17.0 ms (1.03) 10.7 → 11.9 ms (1.11) 🔴
tpcds_q22 70.5 → 72.4 ms (1.03) 70.0 → 72.4 ms (1.04) 88.5 → 89.1 ms (1.01) 30.9 → 30.4 ms (0.99) 32.7 → 33.7 ms (1.03) 26.7 → 26.9 ms (1.01) 23.2 → 23.3 ms (1.00)
tpcds_q23 98.3 → 99.5 ms (1.01) 99.8 → 98.4 ms (0.99) 112 → 114 ms (1.02) 48.8 → 49.7 ms (1.02) 52.1 → 54.3 ms (1.04) 51.5 → 50.3 ms (0.98) 34.0 → 34.1 ms (1.00)
tpcds_q24 85.8 → 85.0 ms (0.99) 91.4 → 92.2 ms (1.01) 113 → 116 ms (1.02) 35.9 → 36.4 ms (1.01) 37.5 → 38.4 ms (1.02) 36.2 → 36.1 ms (1.00) 27.9 → 27.5 ms (0.99)
tpcds_q25 67.8 → 73.2 ms (1.08) 77.6 → 79.0 ms (1.02) 99.6 → 101 ms (1.01) 31.3 → 32.0 ms (1.02) 33.9 → 34.7 ms (1.02) 36.7 → 37.0 ms (1.01) 25.1 → 27.8 ms (1.11) 🔴
tpcds_q26 42.3 → 46.7 ms (1.10) 🔴 45.2 → 47.5 ms (1.05) 58.5 → 60.6 ms (1.04) 31.8 → 32.5 ms (1.02) 38.4 → 38.5 ms (1.00) 34.7 → 34.9 ms (1.01) 26.5 → 26.4 ms (1.00)
tpcds_q27 73.6 → 72.0 ms (0.98) 73.0 → 84.3 ms (1.15) 🔴 97.6 → 97.2 ms (1.00) 37.2 → 37.2 ms (1.00) 40.5 → 39.9 ms (0.99) 40.7 → 41.4 ms (1.02) 33.6 → 35.4 ms (1.06)
tpcds_q28 39.2 → 41.0 ms (1.05) 41.6 → 44.2 ms (1.06) 40.5 → 40.6 ms (1.00) 26.7 → 26.2 ms (0.98) 30.2 → 29.3 ms (0.97) 33.5 → 30.0 ms (0.90) 🟢 17.3 → 18.9 ms (1.09)
tpcds_q29 67.2 → 66.8 ms (0.99) 68.3 → 78.8 ms (1.15) 🔴 99.2 → 98.4 ms (0.99) 29.3 → 29.8 ms (1.02) 33.2 → 33.5 ms (1.01) 38.5 → 39.6 ms (1.03) 24.2 → 27.4 ms (1.13) 🔴
tpcds_q30 44.5 → 47.7 ms (1.07) 46.9 → 48.7 ms (1.04) 57.9 → 58.4 ms (1.01) 27.0 → 27.4 ms (1.01) 31.1 → 30.7 ms (0.99) 44.2 → 44.0 ms (1.00) 29.9 → 32.2 ms (1.08)
tpcds_q31 88.7 → 90.9 ms (1.02) 93.9 → 94.2 ms (1.00) 96.4 → 97.0 ms (1.01) 26.7 → 27.1 ms (1.01) 27.1 → 28.1 ms (1.04) 42.2 → 43.3 ms (1.02) 38.0 → 36.1 ms (0.95)
tpcds_q32 29.3 → 31.9 ms (1.09) 32.0 → 34.8 ms (1.09) 39.7 → 40.2 ms (1.01) 18.5 → 18.6 ms (1.01) 19.3 → 19.1 ms (0.99) 20.6 → 20.1 ms (0.97) 12.1 → 13.5 ms (1.11) 🔴
tpcds_q33 50.2 → 51.1 ms (1.02) 52.3 → 53.8 ms (1.03) 54.0 → 55.5 ms (1.03) 26.9 → 27.3 ms (1.01) 29.4 → 30.6 ms (1.04) 28.7 → 29.9 ms (1.04) 15.4 → 17.9 ms (1.16) 🔴
tpcds_q34 36.4 → 37.3 ms (1.03) 37.4 → 40.7 ms (1.09) 35.0 → 35.7 ms (1.02) 22.1 → 23.7 ms (1.07) 24.5 → 24.7 ms (1.01) 23.7 → 24.5 ms (1.03) 18.7 → 17.0 ms (0.91)
tpcds_q35 56.6 → 54.7 ms (0.97) 60.7 → 61.7 ms (1.02) 61.6 → 65.1 ms (1.06) 44.7 → 45.2 ms (1.01) 46.2 → 47.1 ms (1.02) 43.5 → 44.2 ms (1.02) 33.2 → 32.9 ms (0.99)
tpcds_q36 66.1 → 68.8 ms (1.04) 69.0 → 74.1 ms (1.07) 69.9 → 69.8 ms (1.00) 24.2 → 25.1 ms (1.04) 25.7 → 26.1 ms (1.02) 47.5 → 47.8 ms (1.01) 22.1 → 21.6 ms (0.98)
tpcds_q37 32.6 → 34.6 ms (1.06) 36.3 → 36.4 ms (1.00) 31.4 → 34.0 ms (1.08) 24.5 → 25.6 ms (1.04) 26.8 → 28.8 ms (1.07) 19.1 → 19.6 ms (1.03) 13.5 → 14.4 ms (1.06)
tpcds_q38 45.7 → 47.7 ms (1.04) 47.9 → 50.4 ms (1.05) 63.0 → 64.6 ms (1.03) 32.3 → 32.3 ms (1.00) 34.1 → 34.6 ms (1.01) 33.4 → 34.5 ms (1.03) 29.3 → 27.9 ms (0.95)
tpcds_q39 86.1 → 87.4 ms (1.01) 86.7 → 86.6 ms (1.00) 85.2 → 83.7 ms (0.98) 32.6 → 32.2 ms (0.99) 33.7 → 34.4 ms (1.02) 29.9 → 30.3 ms (1.01) 24.5 → 24.5 ms (1.00)
tpcds_q40 39.6 → 40.0 ms (1.01) 42.3 → 45.0 ms (1.06) 50.1 → 46.5 ms (0.93) 19.6 → 19.5 ms (1.00) 21.6 → 21.8 ms (1.01) 20.8 → 20.8 ms (1.00) 15.1 → 16.8 ms (1.12) 🔴
tpcds_q41 33.2 → 34.0 ms (1.03) 33.8 → 35.0 ms (1.04) 31.9 → 32.0 ms (1.01) 24.8 → 24.6 ms (0.99) 26.4 → 26.3 ms (0.99) 18.3 → 18.1 ms (0.99) 17.2 → 17.1 ms (0.99)
tpcds_q42 28.4 → 28.5 ms (1.00) 30.6 → 32.0 ms (1.04) 29.2 → 32.9 ms (1.12) 🔴 16.2 → 16.8 ms (1.04) 16.2 → 16.4 ms (1.01) 15.7 → 15.7 ms (1.00) 11.4 → 12.0 ms (1.06)
tpcds_q43 30.6 → 31.6 ms (1.03) 31.6 → 33.7 ms (1.07) 31.0 → 31.6 ms (1.02) 18.7 → 19.4 ms (1.04) 19.4 → 21.0 ms (1.08) 18.0 → 18.1 ms (1.01) 14.7 → 13.8 ms (0.94)
tpcds_q44 48.6 → 51.1 ms (1.05) 46.6 → 48.3 ms (1.04) 50.4 → 51.2 ms (1.01) 24.1 → 25.6 ms (1.06) 26.3 → 27.2 ms (1.03) 28.8 → 26.9 ms (0.93) 17.7 → 19.5 ms (1.11) 🔴
tpcds_q45 34.4 → 36.1 ms (1.05) 37.4 → 38.4 ms (1.03) 39.1 → 40.4 ms (1.03) 30.2 → 31.3 ms (1.04) 32.9 → 34.1 ms (1.04) 27.6 → 28.6 ms (1.04) 21.7 → 22.1 ms (1.02)
tpcds_q46 45.5 → 47.3 ms (1.04) 46.7 → 48.4 ms (1.04) 44.4 → 45.7 ms (1.03) 42.9 → 44.5 ms (1.04) 44.6 → 45.6 ms (1.02) 43.2 → 43.3 ms (1.00) 34.2 → 34.0 ms (0.99)
tpcds_q47 105 → 108 ms (1.04) 112 → 113 ms (1.01) 121 → 119 ms (0.98) 40.6 → 42.6 ms (1.05) 42.9 → 42.8 ms (1.00) 41.2 → 41.9 ms (1.02) 42.0 → 41.5 ms (0.99)
tpcds_q48 51.2 → 51.8 ms (1.01) 68.6 → 60.6 ms (0.88) 🟢 58.2 → 59.2 ms (1.02) 35.2 → 35.6 ms (1.01) 47.0 → 47.0 ms (1.00) 36.2 → 37.0 ms (1.02) 36.6 → 28.2 ms (0.77) 🟢
tpcds_q49 79.4 → 79.8 ms (1.01) 77.0 → 79.9 ms (1.04) 90.0 → 92.9 ms (1.03) 31.1 → 31.7 ms (1.02) 33.3 → 33.7 ms (1.01) 36.5 → 35.8 ms (0.98) 23.8 → 24.8 ms (1.04)
tpcds_q50 48.9 → 53.1 ms (1.09) 51.0 → 53.1 ms (1.04) 65.4 → 65.7 ms (1.00) 25.6 → 25.4 ms (0.99) 25.9 → 26.9 ms (1.04) 25.2 → 25.7 ms (1.02) 17.3 → 20.6 ms (1.19) 🔴
tpcds_q51 87.5 → 91.9 ms (1.05) 86.5 → 89.3 ms (1.03) 93.3 → 92.8 ms (0.99) 77.9 → 78.6 ms (1.01) 78.3 → 59.9 ms (0.76) 🟢 60.2 → 60.4 ms (1.00) 55.3 → 55.1 ms (1.00)
tpcds_q52 27.9 → 29.1 ms (1.05) 31.1 → 32.4 ms (1.04) 28.5 → 29.7 ms (1.04) 15.9 → 16.6 ms (1.05) 16.9 → 17.2 ms (1.02) 15.3 → 15.8 ms (1.03) 11.2 → 12.1 ms (1.08)
tpcds_q53 37.7 → 39.6 ms (1.05) 41.0 → 43.2 ms (1.06) 36.2 → 37.6 ms (1.04) 20.8 → 21.4 ms (1.03) 23.7 → 23.3 ms (0.98) 20.8 → 21.1 ms (1.01) 17.4 → 16.8 ms (0.97)
tpcds_q54 54.4 → 55.9 ms (1.03) 57.6 → 58.6 ms (1.02) 62.9 → 68.0 ms (1.08) 31.7 → 30.7 ms (0.97) 31.2 → 32.4 ms (1.04) 31.8 → 32.0 ms (1.01) 21.6 → 21.9 ms (1.01)
tpcds_q55 27.1 → 27.8 ms (1.02) 30.6 → 30.8 ms (1.01) 29.1 → 29.8 ms (1.02) 16.4 → 16.2 ms (0.99) 16.3 → 16.6 ms (1.02) 15.4 → 15.8 ms (1.02) 11.3 → 12.0 ms (1.06)
tpcds_q56 47.2 → 50.2 ms (1.07) 49.8 → 51.6 ms (1.04) 53.1 → 51.5 ms (0.97) 28.7 → 29.5 ms (1.03) 30.4 → 31.2 ms (1.03) 30.7 → 31.5 ms (1.03) 17.3 → 19.8 ms (1.15) 🔴
tpcds_q57 86.5 → 89.9 ms (1.04) 93.6 → 96.2 ms (1.03) 116 → 113 ms (0.98) 33.4 → 35.2 ms (1.05) 35.4 → 35.9 ms (1.01) 36.2 → 36.7 ms (1.02) 34.7 → 34.9 ms (1.01)
tpcds_q58 67.2 → 68.1 ms (1.01) 72.6 → 74.0 ms (1.02) 81.2 → 83.6 ms (1.03) 30.4 → 30.7 ms (1.01) 32.6 → 32.0 ms (0.98) 29.7 → 29.5 ms (0.99) 17.7 → 18.6 ms (1.05)
tpcds_q59 51.8 → 52.5 ms (1.01) 52.3 → 54.3 ms (1.04) 65.2 → 64.0 ms (0.98) 26.7 → 28.6 ms (1.07) 28.2 → 28.9 ms (1.02) 28.4 → 29.0 ms (1.02) 23.5 → 24.1 ms (1.03)
tpcds_q60 46.5 → 50.5 ms (1.09) 49.4 → 50.8 ms (1.03) 50.5 → 56.2 ms (1.11) 🔴 28.6 → 29.1 ms (1.02) 30.4 → 31.6 ms (1.04) 31.9 → 32.6 ms (1.02) 16.9 → 19.3 ms (1.14) 🔴
tpcds_q61 39.7 → 41.7 ms (1.05) 42.5 → 44.2 ms (1.04) 49.8 → 51.0 ms (1.02) 29.6 → 30.6 ms (1.03) 32.7 → 32.2 ms (0.98) 28.8 → 29.6 ms (1.03) 18.4 → 19.9 ms (1.08)
tpcds_q62 32.3 → 33.9 ms (1.05) 33.6 → 34.4 ms (1.02) 37.6 → 38.1 ms (1.01) 17.3 → 17.8 ms (1.03) 19.5 → 19.8 ms (1.02) 18.0 → 18.2 ms (1.01) 14.6 → 16.0 ms (1.10)
tpcds_q63 37.0 → 40.3 ms (1.09) 41.5 → 42.3 ms (1.02) 36.5 → 39.0 ms (1.07) 20.3 → 20.4 ms (1.01) 22.5 → 22.8 ms (1.01) 20.0 → 20.0 ms (1.00) 16.5 → 16.0 ms (0.97)
tpcds_q64 219 → 222 ms (1.02) 313 → 312 ms (1.00) 323 → 325 ms (1.00) 67.6 → 71.9 ms (1.06) 74.5 → 75.9 ms (1.02) 80.1 → 80.8 ms (1.01) 54.5 → 56.6 ms (1.04)
tpcds_q65 93.7 → 94.6 ms (1.01) 101 → 101 ms (1.01) 138 → 138 ms (1.00) 22.9 → 23.9 ms (1.04) 23.6 → 24.7 ms (1.05) 25.9 → 26.1 ms (1.00) 18.3 → 19.3 ms (1.05)
tpcds_q66 67.9 → 69.7 ms (1.03) 70.3 → 72.1 ms (1.03) 73.9 → 75.9 ms (1.03) 38.5 → 38.7 ms (1.00) 41.2 → 43.5 ms (1.05) 38.4 → 38.2 ms (1.00) 30.9 → 32.1 ms (1.04)
tpcds_q67 82.7 → 84.8 ms (1.03) 83.8 → 85.5 ms (1.02) 91.6 → 92.7 ms (1.01) 65.1 → 67.4 ms (1.03) 67.0 → 67.7 ms (1.01) 66.7 → 67.7 ms (1.01) 64.8 → 65.9 ms (1.02)
tpcds_q68 45.1 → 46.0 ms (1.02) 45.3 → 48.9 ms (1.08) 46.8 → 48.3 ms (1.03) 34.6 → 36.1 ms (1.04) 36.2 → 37.3 ms (1.03) 35.8 → 36.2 ms (1.01) 25.9 → 25.6 ms (0.99)
tpcds_q69 46.7 → 49.6 ms (1.06) 58.9 → 57.2 ms (0.97) 55.0 → 54.5 ms (0.99) 37.6 → 38.1 ms (1.01) 41.7 → 43.0 ms (1.03) 34.2 → 34.9 ms (1.02) 24.0 → 24.3 ms (1.01)
tpcds_q70 95.5 → 95.8 ms (1.00) 92.9 → 95.9 ms (1.03) 96.0 → 98.1 ms (1.02) 25.1 → 26.7 ms (1.06) 26.2 → 26.9 ms (1.02) 26.4 → 27.5 ms (1.04) 20.3 → 20.1 ms (0.99)
tpcds_q71 39.7 → 39.8 ms (1.00) 39.9 → 40.5 ms (1.01) 37.2 → 38.7 ms (1.04) 23.2 → 24.0 ms (1.03) 24.9 → 25.6 ms (1.03) 23.8 → 24.7 ms (1.04) 14.7 → 16.0 ms (1.09)
tpcds_q72 276 → 277 ms (1.00) 276 → 283 ms (1.02) 549 → 497 ms (0.91) 57.3 → 56.7 ms (0.99) 60.8 → 60.4 ms (0.99) 59.2 → 59.1 ms (1.00) 46.4 → 45.4 ms (0.98)
tpcds_q73 35.8 → 37.4 ms (1.05) 38.7 → 40.7 ms (1.05) 35.2 → 36.4 ms (1.03) 21.4 → 21.5 ms (1.01) 23.9 → 23.7 ms (0.99) 21.6 → 21.9 ms (1.02) 16.0 → 15.5 ms (0.96)
tpcds_q74 65.2 → 67.3 ms (1.03) 66.6 → 70.8 ms (1.06) 87.0 → 91.0 ms (1.05) 39.8 → 41.5 ms (1.04) 41.4 → 42.3 ms (1.02) 91.9 → 89.6 ms (0.97) 44.4 → 42.2 ms (0.95)
tpcds_q75 157 → 160 ms (1.02) 172 → 166 ms (0.96) 178 → 177 ms (0.99) 49.8 → 49.7 ms (1.00) 51.4 → 52.6 ms (1.02) 49.8 → 50.7 ms (1.02) 40.0 → 41.7 ms (1.04)
tpcds_q76 48.3 → 49.6 ms (1.03) 48.2 → 52.6 ms (1.09) 45.8 → 47.4 ms (1.03) 23.6 → 22.9 ms (0.97) 23.2 → 24.1 ms (1.04) 22.9 → 24.4 ms (1.07) 16.1 → 16.0 ms (0.99)
tpcds_q77 65.2 → 70.0 ms (1.07) 68.9 → 70.1 ms (1.02) 69.1 → 73.7 ms (1.07) 32.3 → 32.3 ms (1.00) 32.6 → 33.2 ms (1.02) 32.3 → 31.5 ms (0.97) 19.2 → 20.1 ms (1.05)
tpcds_q78 86.7 → 89.6 ms (1.03) 88.2 → 89.8 ms (1.02) 116 → 118 ms (1.02) 53.9 → 53.8 ms (1.00) 55.4 → 55.3 ms (1.00) 83.6 → 84.8 ms (1.01) 73.7 → 76.2 ms (1.03)
tpcds_q79 39.2 → 41.3 ms (1.05) 42.6 → 43.6 ms (1.02) 41.2 → 42.0 ms (1.02) 26.8 → 27.6 ms (1.03) 28.5 → 29.2 ms (1.02) 28.2 → 29.2 ms (1.04) 20.7 → 20.8 ms (1.00)
tpcds_q80 94.7 → 96.6 ms (1.02) 94.5 → 97.1 ms (1.03) 108 → 111 ms (1.03) 44.1 → 44.4 ms (1.01) 46.1 → 45.9 ms (1.00) 43.2 → 43.9 ms (1.02) 28.4 → 30.2 ms (1.07)
tpcds_q81 46.1 → 47.2 ms (1.02) 46.1 → 47.8 ms (1.04) 51.8 → 53.2 ms (1.03) 29.6 → 30.5 ms (1.03) 32.5 → 32.6 ms (1.00) 44.5 → 44.1 ms (0.99) 37.7 → 36.7 ms (0.97)
tpcds_q82 34.2 → 37.1 ms (1.08) 36.6 → 39.8 ms (1.09) 33.5 → 33.7 ms (1.00) 24.6 → 25.9 ms (1.05) 29.3 → 29.6 ms (1.01) 20.3 → 20.8 ms (1.02) 14.1 → 15.4 ms (1.09)
tpcds_q83 53.1 → 55.9 ms (1.05) 57.4 → 57.2 ms (1.00) 74.3 → 77.4 ms (1.04) 25.3 → 25.9 ms (1.02) 25.3 → 27.6 ms (1.09) 25.1 → 25.2 ms (1.00) 14.6 → 15.4 ms (1.05)
tpcds_q84 26.0 → 26.7 ms (1.03) 27.9 → 28.8 ms (1.03) 38.9 → 39.6 ms (1.02) 24.2 → 24.6 ms (1.02) 24.9 → 24.2 ms (0.97) 28.6 → 28.1 ms (0.98) 17.4 → 17.9 ms (1.03)
tpcds_q85 76.9 → 77.4 ms (1.01) 117 → 120 ms (1.03) 102 → 102 ms (1.00) 37.0 → 37.1 ms (1.00) 51.2 → 52.2 ms (1.02) 39.2 → 38.9 ms (0.99) 34.7 → 31.9 ms (0.92)
tpcds_q86 34.4 → 35.2 ms (1.02) 33.4 → 36.0 ms (1.08) 32.0 → 33.1 ms (1.03) 16.9 → 17.3 ms (1.03) 17.5 → 18.2 ms (1.04) 16.6 → 17.4 ms (1.04) 14.6 → 15.4 ms (1.05)
tpcds_q87 47.9 → 49.6 ms (1.03) 46.8 → 50.1 ms (1.07) 60.4 → 61.7 ms (1.02) 34.3 → 34.0 ms (0.99) 34.8 → 36.5 ms (1.05) 35.9 → 35.3 ms (0.98) 31.3 → 30.4 ms (0.97)
tpcds_q88 55.2 → 55.7 ms (1.01) 69.2 → 69.2 ms (1.00) 49.4 → 53.8 ms (1.09) 38.2 → 38.9 ms (1.02) 38.7 → 39.0 ms (1.01) 44.2 → 46.1 ms (1.04) 24.3 → 24.5 ms (1.01)
tpcds_q89 41.2 → 43.2 ms (1.05) 43.6 → 46.6 ms (1.07) 41.3 → 42.0 ms (1.02) 21.3 → 22.5 ms (1.06) 22.9 → 23.0 ms (1.00) 21.4 → 22.1 ms (1.03) 18.1 → 17.6 ms (0.97)
tpcds_q90 24.8 → 25.9 ms (1.05) 26.7 → 28.3 ms (1.06) 26.1 → 25.8 ms (0.99) 14.9 → 15.1 ms (1.02) 15.2 → 15.0 ms (0.99) 15.4 → 15.2 ms (0.99) 10.1 → 11.7 ms (1.16) 🔴
tpcds_q91 34.3 → 36.8 ms (1.08) 44.2 → 45.6 ms (1.03) 45.5 → 44.6 ms (0.98) 24.0 → 24.5 ms (1.02) 33.1 → 33.3 ms (1.01) 28.5 → 27.7 ms (0.97) 17.1 → 20.0 ms (1.17) 🔴
tpcds_q92 30.4 → 31.7 ms (1.05) 32.2 → 32.7 ms (1.02) 40.6 → 42.4 ms (1.05) 18.9 → 19.2 ms (1.02) 19.6 → 19.5 ms (1.00) 22.2 → 22.2 ms (1.00) 14.0 → 15.6 ms (1.11) 🔴
tpcds_q93 40.6 → 40.9 ms (1.01) 40.2 → 41.8 ms (1.04) 45.2 → 46.3 ms (1.03) 22.5 → 23.9 ms (1.06) 24.1 → 26.2 ms (1.09) 20.1 → 20.1 ms (1.00) 13.8 → 16.4 ms (1.19) 🔴
tpcds_q94 34.9 → 38.5 ms (1.10) 🔴 37.8 → 39.7 ms (1.05) 47.8 → 49.5 ms (1.04) 31.0 → 31.8 ms (1.03) 34.0 → 34.3 ms (1.01) 32.6 → 32.4 ms (0.99) 21.8 → 23.6 ms (1.08)
tpcds_q95 61.5 → 62.8 ms (1.02) 61.6 → 63.5 ms (1.03) 77.9 → 81.8 ms (1.05) 66.7 → 67.3 ms (1.01) 70.3 → 70.4 ms (1.00) 49.5 → 49.3 ms (1.00) 43.8 → 43.1 ms (0.98)
tpcds_q96 25.1 → 25.6 ms (1.02) 27.8 → 26.9 ms (0.97) 24.1 → 24.4 ms (1.02) 15.2 → 15.7 ms (1.03) 15.6 → 16.2 ms (1.04) 13.9 → 14.0 ms (1.01) 8.86 → 8.63 ms (0.97)
tpcds_q97 36.3 → 40.3 ms (1.11) 🔴 36.9 → 39.2 ms (1.06) 50.5 → 51.5 ms (1.02) 31.9 → 31.8 ms (1.00) 33.1 → 34.6 ms (1.05) 33.7 → 34.1 ms (1.01) 27.2 → 27.7 ms (1.02)
tpcds_q98 34.1 → 34.5 ms (1.01) 40.4 → 40.4 ms (1.00) 38.1 → 39.6 ms (1.04) 20.2 → 20.4 ms (1.01) 23.9 → 23.9 ms (1.00) 22.9 → 23.2 ms (1.01) 17.8 → 20.0 ms (1.12) 🔴
tpcds_q99 33.2 → 33.6 ms (1.01) 34.3 → 36.2 ms (1.06) 42.1 → 41.0 ms (0.97) 22.6 → 22.9 ms (1.01) 25.5 → 25.5 ms (1.00) 24.7 → 25.3 ms (1.03) 19.2 → 19.5 ms (1.02)

No file size changes detected.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Statistical and Population Genetics 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (low confidence)
Attributed Vortex impact: -0.7%
Engines: DataFusion No clear signal (-1.4%, low confidence) · DuckDB No clear signal (+0.1%, low confidence)
Vortex (geomean): hot 1.000x ➖ · cold 1.001x ➖
Parquet (geomean): hot 1.007x ➖ · cold 1.008x ➖
Shifts: Parquet (control) +0.7% · Median polish +0.2%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 1.002x ➖ · 11 queries × 6 targets (0↑ 1↓)

Each cell is develop 967bcb67 → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v1] datafusion:vortex-compact [file, v1] datafusion:parquet duckdb:vortex [file, v1] duckdb:vortex-compact [file, v1] duckdb:parquet
geomean 118 → 119 ms (1.01) 135 → 135 ms (1.00) 236 → 241 ms (1.02) 299 → 297 ms (0.99) 328 → 327 ms (1.00) 633 → 630 ms (0.99)
statpopgen_q00 1.80 → 1.78 ms (0.99) 1.79 → 1.77 ms (0.99) 1.92 → 1.96 ms (1.02) 13.2 → 13.0 ms (0.99) 13.2 → 13.1 ms (1.00) 253 → 251 ms (0.99)
statpopgen_q01 3.96 → 4.57 ms (1.15) 🔴 4.49 → 4.20 ms (0.93) 167 → 164 ms (0.99) 13.9 → 14.0 ms (1.00) 14.1 → 13.9 ms (0.98) 312 → 311 ms (1.00)
statpopgen_q02 39.9 → 40.2 ms (1.01) 45.5 → 46.6 ms (1.03) 291 → 295 ms (1.01) 171 → 162 ms (0.95) 160 → 159 ms (0.99) 578 → 584 ms (1.01)
statpopgen_q03 276 → 278 ms (1.01) 335 → 344 ms (1.03) 388 → 393 ms (1.01) 1.18 → 1.18 s (1.00) 1.25 → 1.27 s (1.01) 779 → 731 ms (0.94)
statpopgen_q04 276 → 272 ms (0.99) 330 → 342 ms (1.04) 400 → 396 ms (0.99) 1.18 → 1.19 s (1.00) 1.26 → 1.27 s (1.01) 806 → 757 ms (0.94)
statpopgen_q05 121 → 121 ms (1.00) 147 → 148 ms (1.01) 315 → 330 ms (1.05) 561 → 557 ms (0.99) 643 → 646 ms (1.01) 553 → 531 ms (0.96)
statpopgen_q06 520 → 515 ms (0.99) 654 → 660 ms (1.01) 460 → 479 ms (1.04) 1.47 → 1.47 s (1.00) 1.56 → 1.55 s (1.00) 683 → 641 ms (0.94)
statpopgen_q07 307 → 309 ms (1.01) 355 → 352 ms (0.99) 558 → 582 ms (1.04) 140 → 141 ms (1.01) 194 → 190 ms (0.98) 907 → 936 ms (1.03)
statpopgen_q08 314 → 315 ms (1.00) 365 → 367 ms (1.01) 558 → 585 ms (1.05) 161 → 160 ms (1.00) 217 → 217 ms (1.00) 922 → 946 ms (1.03)
statpopgen_q09 454 → 448 ms (0.99) 494 → 488 ms (0.99) 367 → 362 ms (0.99) 871 → 863 ms (0.99) 956 → 943 ms (0.99) 709 → 758 ms (1.07)
statpopgen_q10 1.00 → 1.00 s (1.00) 1.09 → 1.09 s (1.00) 535 → 556 ms (1.04) 2.44 → 2.41 s (0.99) 2.54 → 2.54 s (1.00) 1.03 → 1.07 s (1.04)
Cold runs (first iteration): geomean 1.003x ➖ · 11 queries × 6 targets (0↑ 1↓)
query datafusion:vortex [file, v1] datafusion:vortex-compact [file, v1] datafusion:parquet duckdb:vortex [file, v1] duckdb:vortex-compact [file, v1] duckdb:parquet
geomean 168 → 167 ms (0.99) 184 → 187 ms (1.02) 427 → 432 ms (1.01) 331 → 327 ms (0.99) 357 → 358 ms (1.00) 651 → 655 ms (1.01)
statpopgen_q00 6.70 → 6.50 ms (0.97) 6.48 → 6.55 ms (1.01) 91.3 → 89.2 ms (0.98) 14.8 → 14.9 ms (1.01) 14.9 → 15.0 ms (1.00) 251 → 249 ms (0.99)
statpopgen_q01 13.4 → 13.5 ms (1.01) 13.4 → 15.0 ms (1.12) 🔴 175 → 177 ms (1.01) 17.2 → 17.4 ms (1.01) 17.2 → 17.2 ms (1.00) 312 → 313 ms (1.00)
statpopgen_q02 65.1 → 66.9 ms (1.03) 66.0 → 65.9 ms (1.00) 413 → 429 ms (1.04) 206 → 199 ms (0.96) 198 → 192 ms (0.97) 581 → 583 ms (1.00)
statpopgen_q03 329 → 307 ms (0.93) 368 → 364 ms (0.99) 541 → 533 ms (0.98) 1.23 → 1.24 s (1.01) 1.31 → 1.31 s (1.01) 839 → 791 ms (0.94)
statpopgen_q04 308 → 304 ms (0.99) 367 → 381 ms (1.04) 525 → 528 ms (1.01) 1.26 → 1.26 s (1.00) 1.32 → 1.34 s (1.02) 829 → 860 ms (1.04)
statpopgen_q05 148 → 149 ms (1.00) 162 → 167 ms (1.03) 435 → 461 ms (1.06) 611 → 616 ms (1.01) 693 → 702 ms (1.01) 571 → 579 ms (1.01)
statpopgen_q06 560 → 562 ms (1.00) 692 → 713 ms (1.03) 643 → 636 ms (0.99) 1.52 → 1.52 s (1.00) 1.60 → 1.60 s (1.00) 680 → 673 ms (0.99)
statpopgen_q07 338 → 336 ms (0.99) 377 → 372 ms (0.99) 683 → 707 ms (1.04) 165 → 155 ms (0.94) 208 → 205 ms (0.99) 914 → 929 ms (1.02)
statpopgen_q08 345 → 343 ms (0.99) 397 → 396 ms (1.00) 696 → 710 ms (1.02) 180 → 173 ms (0.96) 234 → 238 ms (1.02) 910 → 939 ms (1.03)
statpopgen_q09 492 → 499 ms (1.01) 530 → 551 ms (1.04) 505 → 485 ms (0.96) 948 → 926 ms (0.98) 1.01 → 1.01 s (1.00) 759 → 805 ms (1.06)
statpopgen_q10 1.09 → 1.06 s (0.98) 1.17 → 1.15 s (0.98) 687 → 713 ms (1.04) 2.45 → 2.47 s (1.01) 2.55 → 2.58 s (1.01) 1.15 → 1.12 s (0.98)

No file size changes detected.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-DS SF=10 on NVME 📖

Commits: PR aff33cb1 vs base c84406eb
Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.6%
Engines: DataFusion No clear signal (-0.3%, low confidence) · DuckDB No clear signal (+1.5%, low confidence)
Vortex (geomean): hot 0.996x ➖ · cold 0.979x ➖
Parquet (geomean): hot 0.990x ➖ · cold 0.978x ➖
Shifts: Parquet (control) -1.0% · Median polish -0.3%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

Hot runs: geomean 0.993x ➖ · 99 queries × 4 targets (2↑ 2↓)

Each cell is develop c84406eb → this PR aff33cb1 (hot runs), with PR ÷ develop in parentheses: below 1.00 is faster. Changes past ±10% are bold. The geomean row compares each target's geomean over its queries.
Vortex columns name the integration and the scan engine: file reads through the engine's file format (DataFusion ListingTable, DuckDB MultiFileReader), datasource through DataFusion's VortexTable over a Vortex DataSource; v1 and v2 are the LayoutReader and plan scans. Each column compares with its own develop times.

query datafusion:vortex [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:parquet
geomean 54.1 → 54.0 ms (1.00) 85.1 → 85.2 ms (1.00) 40.5 → 40.3 ms (0.99) 57.2 → 56.0 ms (0.98)
tpcds_q01 35.2 → 35.6 ms (1.01) 59.4 → 59.7 ms (1.01) 32.8 → 32.5 ms (0.99) 41.9 → 42.5 ms (1.02)
tpcds_q02 50.8 → 50.8 ms (1.00) 68.7 → 69.5 ms (1.01) 30.2 → 29.8 ms (0.99) 40.1 → 38.6 ms (0.96)
tpcds_q03 21.1 → 21.2 ms (1.00) 29.2 → 28.8 ms (0.99) 15.3 → 15.4 ms (1.00) 25.2 → 23.3 ms (0.92)
tpcds_q04 296 → 296 ms (1.00) 645 → 645 ms (1.00) 130 → 128 ms (0.98) 203 → 202 ms (1.00)
tpcds_q05 73.7 → 74.2 ms (1.01) 81.5 → 80.5 ms (0.99) 56.7 → 56.3 ms (0.99) 70.2 → 66.5 ms (0.95)
tpcds_q06 44.6 → 45.0 ms (1.01) 90.1 → 92.2 ms (1.02) 59.4 → 59.7 ms (1.00) 75.8 → 75.4 ms (0.99)
tpcds_q07 42.5 → 43.4 ms (1.02) 102 → 100 ms (0.99) 31.8 → 32.4 ms (1.02) 59.8 → 59.3 ms (0.99)
tpcds_q08 30.5 → 30.9 ms (1.01) 48.4 → 48.6 ms (1.00) 33.0 → 32.6 ms (0.99) 40.7 → 37.0 ms (0.91)
tpcds_q09 43.5 → 42.1 ms (0.97) 75.7 → 75.3 ms (0.99) 56.5 → 56.2 ms (0.99) 85.1 → 85.5 ms (1.01)
tpcds_q10 40.7 → 39.2 ms (0.96) 58.5 → 60.8 ms (1.04) 41.1 → 40.5 ms (0.99) 53.2 → 52.5 ms (0.99)
tpcds_q11 177 → 178 ms (1.01) 417 → 412 ms (0.99) 97.4 → 96.9 ms (1.00) 158 → 159 ms (1.00)
tpcds_q12 17.7 → 18.1 ms (1.02) 40.5 → 40.3 ms (1.00) 17.8 → 17.9 ms (1.01) 39.7 → 40.2 ms (1.01)
tpcds_q13 39.4 → 40.0 ms (1.01) 86.0 → 86.7 ms (1.01) 40.5 → 40.4 ms (1.00) 72.2 → 67.9 ms (0.94)
tpcds_q14 238 → 240 ms (1.01) 383 → 387 ms (1.01) 128 → 122 ms (0.95) 188 → 186 ms (0.99)
tpcds_q15 41.2 → 40.6 ms (0.99) 50.4 → 47.0 ms (0.93) 61.0 → 61.0 ms (1.00) 61.4 → 60.1 ms (0.98)
tpcds_q16 23.9 → 23.9 ms (1.00) 45.5 → 45.4 ms (1.00) 34.6 → 33.7 ms (0.97) 38.1 → 37.6 ms (0.99)
tpcds_q17 68.9 → 65.7 ms (0.95) 138 → 134 ms (0.97) 56.9 → 56.3 ms (0.99) 83.0 → 82.5 ms (0.99)
tpcds_q18 68.1 → 68.8 ms (1.01) 110 → 111 ms (1.01) 83.6 → 84.5 ms (1.01) 88.7 → 87.2 ms (0.98)
tpcds_q19 25.3 → 26.2 ms (1.03) 47.8 → 47.3 ms (0.99) 46.2 → 45.6 ms (0.99) 47.6 → 46.1 ms (0.97)
tpcds_q20 23.7 → 23.5 ms (0.99) 41.5 → 40.8 ms (0.98) 22.6 → 21.8 ms (0.96) 42.5 → 42.6 ms (1.00)
tpcds_q21 47.5 → 45.4 ms (0.95) 61.3 → 63.5 ms (1.04) 35.8 → 35.6 ms (0.99) 24.0 → 23.5 ms (0.98)
tpcds_q22 450 → 444 ms (0.99) 462 → 465 ms (1.01) 158 → 158 ms (1.00) 163 → 173 ms (1.06)
tpcds_q23 296 → 300 ms (1.01) 358 → 367 ms (1.03) 184 → 185 ms (1.00) 182 → 186 ms (1.02)
tpcds_q24 122 → 119 ms (0.98) 241 → 243 ms (1.01) 52.2 → 52.3 ms (1.00) 71.2 → 69.6 ms (0.98)
tpcds_q25 64.2 → 63.4 ms (0.99) 147 → 143 ms (0.98) 52.4 → 52.2 ms (1.00) 51.9 → 50.0 ms (0.96)
tpcds_q26 36.9 → 37.4 ms (1.01) 54.8 → 54.2 ms (0.99) 27.3 → 25.9 ms (0.95) 43.6 → 42.3 ms (0.97)
tpcds_q27 73.5 → 77.8 ms (1.06) 206 → 214 ms (1.04) 34.1 → 34.5 ms (1.01) 57.9 → 55.9 ms (0.97)
tpcds_q28 42.9 → 43.4 ms (1.01) 89.4 → 85.5 ms (0.96) 43.4 → 42.4 ms (0.98) 93.0 → 90.2 ms (0.97)
tpcds_q29 68.4 → 65.8 ms (0.96) 141 → 142 ms (1.00) 51.6 → 50.5 ms (0.98) 84.6 → 82.1 ms (0.97)
tpcds_q30 51.9 → 51.3 ms (0.99) 97.1 → 98.3 ms (1.01) 38.5 → 37.8 ms (0.98) 71.9 → 70.8 ms (0.98)
tpcds_q31 209 → 206 ms (0.99) 320 → 316 ms (0.99) 39.5 → 39.6 ms (1.00) 98.8 → 97.3 ms (0.98)
tpcds_q32 15.3 → 15.2 ms (0.99) 29.2 → 29.2 ms (1.00) 14.5 → 14.1 ms (0.97) 18.3 → 18.1 ms (0.99)
tpcds_q33 35.0 → 35.8 ms (1.02) 53.7 → 52.3 ms (0.97) 35.8 → 35.2 ms (0.98) 61.6 → 61.6 ms (1.00)
tpcds_q34 45.4 → 44.6 ms (0.98) 63.5 → 64.1 ms (1.01) 25.4 → 24.7 ms (0.97) 33.1 → 33.1 ms (1.00)
tpcds_q35 94.8 → 94.2 ms (0.99) 125 → 126 ms (1.01) 69.6 → 67.6 ms (0.97) 77.6 → 76.5 ms (0.99)
tpcds_q36 69.5 → 71.5 ms (1.03) 97.3 → 99.1 ms (1.02) 30.8 → 30.7 ms (1.00) 97.0 → 93.3 ms (0.96)
tpcds_q37 25.5 → 23.3 ms (0.91) 50.8 → 51.0 ms (1.00) 27.8 → 27.2 ms (0.98) 41.4 → 41.3 ms (1.00)
tpcds_q38 80.5 → 81.6 ms (1.01) 121 → 120 ms (0.99) 64.4 → 64.3 ms (1.00) 77.6 → 74.9 ms (0.96)
tpcds_q39 827 → 828 ms (1.00) 847 → 952 ms (1.12) 🔴 58.1 → 62.0 ms (1.07) 61.2 → 61.1 ms (1.00)
tpcds_q40 28.7 → 28.6 ms (1.00) 35.3 → 35.8 ms (1.01) 18.9 → 19.2 ms (1.02) 27.9 → 27.4 ms (0.98)
tpcds_q41 19.3 → 19.4 ms (1.01) 32.4 → 32.6 ms (1.01) 19.5 → 19.6 ms (1.00) 20.8 → 20.3 ms (0.97)
tpcds_q42 17.1 → 16.8 ms (0.99) 25.8 → 25.9 ms (1.01) 15.9 → 15.9 ms (1.00) 25.9 → 23.6 ms (0.91)
tpcds_q43 18.7 → 18.6 ms (0.99) 28.3 → 27.6 ms (0.97) 21.7 → 21.4 ms (0.98) 28.2 → 26.3 ms (0.93)
tpcds_q44 43.0 → 43.3 ms (1.01) 64.8 → 65.2 ms (1.01) 28.1 → 27.6 ms (0.98) 59.9 → 57.2 ms (0.96)
tpcds_q45 43.4 → 43.2 ms (1.00) 65.8 → 65.6 ms (1.00) 46.6 → 46.2 ms (0.99) 60.7 → 58.7 ms (0.97)
tpcds_q46 76.2 → 76.7 ms (1.01) 108 → 108 ms (0.99) 51.9 → 51.4 ms (0.99) 71.2 → 70.5 ms (0.99)
tpcds_q47 182 → 182 ms (1.00) 236 → 241 ms (1.02) 82.6 → 84.4 ms (1.02) 90.7 → 90.5 ms (1.00)
tpcds_q48 37.4 → 37.0 ms (0.99) 78.0 → 78.5 ms (1.01) 55.4 → 56.1 ms (1.01) 67.6 → 63.3 ms (0.94)
tpcds_q49 81.4 → 83.6 ms (1.03) 112 → 117 ms (1.05) 40.5 → 39.9 ms (0.99) 68.6 → 65.4 ms (0.95)
tpcds_q50 57.4 → 56.6 ms (0.99) 89.8 → 92.2 ms (1.03) 34.5 → 33.7 ms (0.98) 44.6 → 43.3 ms (0.97)
tpcds_q51 246 → 246 ms (1.00) 256 → 256 ms (1.00) 208 → 207 ms (1.00) 216 → 216 ms (1.00)
tpcds_q52 17.7 → 17.2 ms (0.97) 26.5 → 26.6 ms (1.00) 15.9 → 16.1 ms (1.01) 27.8 → 24.0 ms (0.86) 🟢
tpcds_q53 24.1 → 24.9 ms (1.03) 36.2 → 36.2 ms (1.00) 24.0 → 23.6 ms (0.99) 34.7 → 34.5 ms (0.99)
tpcds_q54 59.5 → 59.0 ms (0.99) 86.7 → 84.4 ms (0.97) 46.2 → 45.8 ms (0.99) 55.3 → 53.2 ms (0.96)
tpcds_q55 16.9 → 16.8 ms (0.99) 26.2 → 25.1 ms (0.96) 16.4 → 15.8 ms (0.97) 27.2 → 23.7 ms (0.87) 🟢
tpcds_q56 34.8 → 34.9 ms (1.00) 52.3 → 52.2 ms (1.00) 35.5 → 34.0 ms (0.96) 70.5 → 65.6 ms (0.93)
tpcds_q57 132 → 131 ms (0.99) 172 → 169 ms (0.98) 64.7 → 64.6 ms (1.00) 68.5 → 69.0 ms (1.01)
tpcds_q58 50.6 → 51.5 ms (1.02) 98.0 → 98.7 ms (1.01) 38.8 → 40.0 ms (1.03) 55.2 → 53.4 ms (0.97)
tpcds_q59 57.3 → 57.8 ms (1.01) 104 → 105 ms (1.01) 56.4 → 56.5 ms (1.00) 60.3 → 61.2 ms (1.02)
tpcds_q60 36.1 → 35.8 ms (0.99) 54.6 → 53.5 ms (0.98) 36.6 → 37.0 ms (1.01) 81.4 → 78.7 ms (0.97)
tpcds_q61 51.9 → 50.0 ms (0.96) 76.3 → 78.0 ms (1.02) 36.5 → 35.7 ms (0.98) 80.4 → 76.4 ms (0.95)
tpcds_q62 19.0 → 19.1 ms (1.01) 27.7 → 27.6 ms (1.00) 15.3 → 16.9 ms (1.11) 🔴 20.9 → 20.8 ms (0.99)
tpcds_q63 24.8 → 24.6 ms (0.99) 37.2 → 36.6 ms (0.98) 23.4 → 22.9 ms (0.98) 33.7 → 32.5 ms (0.97)
tpcds_q64 335 → 335 ms (1.00) 665 → 663 ms (1.00) 119 → 119 ms (1.00) 136 → 136 ms (1.00)
tpcds_q65 77.4 → 77.8 ms (1.00) 101 → 102 ms (1.01) 53.7 → 54.1 ms (1.01) 93.1 → 98.0 ms (1.05)
tpcds_q66 52.4 → 52.4 ms (1.00) 76.0 → 76.2 ms (1.00) 42.4 → 42.7 ms (1.01) 47.6 → 46.1 ms (0.97)
tpcds_q67 424 → 426 ms (1.01) 434 → 430 ms (0.99) 275 → 277 ms (1.01) 293 → 291 ms (0.99)
tpcds_q68 67.4 → 69.0 ms (1.02) 109 → 110 ms (1.01) 45.9 → 46.0 ms (1.00) 60.3 → 62.4 ms (1.03)
tpcds_q69 39.8 → 38.5 ms (0.97) 59.8 → 58.1 ms (0.97) 39.9 → 39.7 ms (1.00) 65.8 → 65.3 ms (0.99)
tpcds_q70 195 → 197 ms (1.01) 211 → 209 ms (0.99) 27.8 → 27.4 ms (0.99) 41.5 → 44.0 ms (1.06)
tpcds_q71 30.0 → 29.0 ms (0.97) 39.2 → 37.6 ms (0.96) 29.7 → 29.3 ms (0.99) 40.7 → 37.4 ms (0.92)
tpcds_q72 1.58 → 1.61 s (1.02) 1.77 → 1.71 s (0.97) 140 → 138 ms (0.98) 174 → 174 ms (1.00)
tpcds_q73 41.2 → 41.1 ms (1.00) 62.2 → 64.6 ms (1.04) 23.0 → 22.2 ms (0.96) 27.4 → 27.9 ms (1.02)
tpcds_q74 137 → 138 ms (1.00) 295 → 291 ms (0.99) 87.7 → 82.1 ms (0.94) 214 → 214 ms (1.00)
tpcds_q75 166 → 160 ms (0.97) 194 → 197 ms (1.01) 114 → 113 ms (0.99) 133 → 131 ms (0.99)
tpcds_q76 35.1 → 35.0 ms (1.00) 50.9 → 50.4 ms (0.99) 29.1 → 28.8 ms (0.99) 50.1 → 50.0 ms (1.00)
tpcds_q77 57.5 → 56.3 ms (0.98) 72.5 → 70.0 ms (0.96) 38.0 → 38.7 ms (1.02) 55.4 → 52.4 ms (0.95)
tpcds_q78 142 → 143 ms (1.00) 211 → 212 ms (1.01) 137 → 137 ms (1.00) 150 → 149 ms (0.99)
tpcds_q79 57.0 → 56.9 ms (1.00) 77.8 → 79.0 ms (1.02) 39.3 → 38.3 ms (0.97) 51.2 → 52.6 ms (1.03)
tpcds_q80 82.5 → 84.0 ms (1.02) 144 → 144 ms (1.00) 59.5 → 59.2 ms (0.99) 83.2 → 79.5 ms (0.96)
tpcds_q81 56.0 → 56.6 ms (1.01) 107 → 108 ms (1.01) 57.6 → 57.7 ms (1.00) 97.1 → 95.8 ms (0.99)
tpcds_q82 31.7 → 31.0 ms (0.98) 54.4 → 56.3 ms (1.04) 28.6 → 28.6 ms (1.00) 45.8 → 44.7 ms (0.98)
tpcds_q83 42.4 → 42.2 ms (1.00) 65.3 → 66.3 ms (1.02) 22.2 → 21.9 ms (0.98) 28.2 → 27.9 ms (0.99)
tpcds_q84 13.1 → 13.3 ms (1.02) 35.2 → 35.8 ms (1.02) 13.6 → 13.7 ms (1.01) 22.7 → 22.0 ms (0.97)
tpcds_q85 60.7 → 61.0 ms (1.01) 106 → 101 ms (0.96) 37.3 → 37.6 ms (1.01) 51.8 → 49.7 ms (0.96)
tpcds_q86 18.5 → 18.7 ms (1.01) 21.8 → 22.1 ms (1.01) 17.7 → 17.4 ms (0.98) 25.0 → 25.2 ms (1.01)
tpcds_q87 80.8 → 81.4 ms (1.01) 121 → 122 ms (1.01) 81.1 → 76.2 ms (0.94) 85.4 → 88.0 ms (1.03)
tpcds_q88 65.8 → 65.9 ms (1.00) 85.5 → 87.3 ms (1.02) 40.4 → 39.7 ms (0.98) 107 → 104 ms (0.97)
tpcds_q89 28.5 → 27.8 ms (0.98) 39.1 → 39.5 ms (1.01) 26.2 → 26.4 ms (1.01) 37.3 → 36.4 ms (0.98)
tpcds_q90 13.0 → 13.1 ms (1.01) 17.6 → 17.4 ms (0.99) 10.5 → 10.3 ms (0.98) 15.3 → 14.8 ms (0.97)
tpcds_q91 19.6 → 19.2 ms (0.98) 42.4 → 42.0 ms (0.99) 19.5 → 19.8 ms (1.02) 25.8 → 24.8 ms (0.96)
tpcds_q92 15.4 → 15.3 ms (0.99) 28.8 → 28.6 ms (0.99) 13.5 → 13.6 ms (1.01) 21.9 → 20.4 ms (0.93)
tpcds_q93 55.1 → 55.0 ms (1.00) 61.7 → 62.9 ms (1.02) 24.9 → 24.8 ms (1.00) 36.3 → 34.3 ms (0.95)
tpcds_q94 21.6 → 21.5 ms (0.99) 38.9 → 38.7 ms (0.99) 32.0 → 32.5 ms (1.02) 39.5 → 37.5 ms (0.95)
tpcds_q95 95.5 → 96.7 ms (1.01) 128 → 129 ms (1.01) 95.3 → 93.0 ms (0.98) 81.7 → 81.7 ms (1.00)
tpcds_q96 13.3 → 13.5 ms (1.01) 21.0 → 21.0 ms (1.00) 13.9 → 13.8 ms (0.99) 17.2 → 17.9 ms (1.04)
tpcds_q97 39.8 → 40.2 ms (1.01) 54.5 → 55.4 ms (1.02) 64.2 → 64.6 ms (1.01) 73.3 → 73.5 ms (1.00)
tpcds_q98 23.1 → 23.5 ms (1.02) 61.6 → 61.6 ms (1.00) 28.8 → 28.4 ms (0.98) 54.8 → 52.8 ms (0.96)
tpcds_q99 30.0 → 29.9 ms (1.00) 42.8 → 43.0 ms (1.00) 23.0 → 23.5 ms (1.02) 24.9 → 25.4 ms (1.02)
Cold runs (first iteration): geomean 0.979x ➖ · 99 queries × 4 targets (11↑ 3↓)
query datafusion:vortex [file, v2] datafusion:parquet duckdb:vortex [file, v2] duckdb:parquet
geomean 87.8 → 86.0 ms (0.98) 116 → 114 ms (0.99) 60.8 → 59.4 ms (0.98) 68.9 → 66.8 ms (0.97)
tpcds_q01 60.2 → 59.9 ms (0.99) 81.3 → 79.5 ms (0.98) 44.4 → 43.4 ms (0.98) 50.4 → 49.9 ms (0.99)
tpcds_q02 76.3 → 76.3 ms (1.00) 90.8 → 90.3 ms (0.99) 43.2 → 43.6 ms (1.01) 52.3 → 45.5 ms (0.87) 🟢
tpcds_q03 48.8 → 45.9 ms (0.94) 48.4 → 49.2 ms (1.02) 31.1 → 31.1 ms (1.00) 36.1 → 32.0 ms (0.89) 🟢
tpcds_q04 358 → 348 ms (0.97) 708 → 710 ms (1.00) 152 → 149 ms (0.98) 218 → 227 ms (1.04)
tpcds_q05 103 → 112 ms (1.08) 114 → 112 ms (0.98) 82.8 → 76.6 ms (0.92) 87.6 → 83.4 ms (0.95)
tpcds_q06 70.7 → 70.1 ms (0.99) 117 → 113 ms (0.97) 80.3 → 80.2 ms (1.00) 86.9 → 91.2 ms (1.05)
tpcds_q07 79.7 → 76.9 ms (0.97) 135 → 139 ms (1.02) 55.1 → 53.4 ms (0.97) 76.7 → 71.9 ms (0.94)
tpcds_q08 55.6 → 51.9 ms (0.93) 75.2 → 72.7 ms (0.97) 43.2 → 42.7 ms (0.99) 47.8 → 40.7 ms (0.85) 🟢
tpcds_q09 77.1 → 75.7 ms (0.98) 105 → 107 ms (1.02) 80.5 → 77.0 ms (0.96) 97.1 → 95.9 ms (0.99)
tpcds_q10 67.6 → 65.2 ms (0.97) 76.4 → 86.5 ms (1.13) 🔴 59.2 → 56.4 ms (0.95) 63.2 → 62.7 ms (0.99)
tpcds_q11 216 → 213 ms (0.99) 453 → 456 ms (1.01) 110 → 112 ms (1.02) 167 → 171 ms (1.03)
tpcds_q12 38.5 → 37.3 ms (0.97) 57.5 → 57.2 ms (0.99) 30.1 → 31.1 ms (1.03) 47.1 → 46.8 ms (1.00)
tpcds_q13 84.0 → 81.0 ms (0.96) 131 → 130 ms (0.99) 68.3 → 70.9 ms (1.04) 89.0 → 85.1 ms (0.96)
tpcds_q14 284 → 276 ms (0.97) 419 → 416 ms (0.99) 146 → 146 ms (1.00) 199 → 192 ms (0.97)
tpcds_q15 60.0 → 59.4 ms (0.99) 72.7 → 68.6 ms (0.94) 74.1 → 73.1 ms (0.99) 68.6 → 67.3 ms (0.98)
tpcds_q16 42.6 → 44.8 ms (1.05) 63.1 → 62.7 ms (0.99) 52.8 → 50.7 ms (0.96) 47.0 → 45.3 ms (0.97)
tpcds_q17 89.1 → 97.7 ms (1.10) 177 → 173 ms (0.98) 80.3 → 77.5 ms (0.97) 89.9 → 88.5 ms (0.98)
tpcds_q18 93.1 → 95.4 ms (1.02) 145 → 142 ms (0.98) 105 → 102 ms (0.97) 98.6 → 94.5 ms (0.96)
tpcds_q19 54.3 → 52.4 ms (0.97) 80.4 → 75.7 ms (0.94) 65.3 → 63.3 ms (0.97) 62.0 → 58.3 ms (0.94)
tpcds_q20 42.0 → 38.7 ms (0.92) 55.3 → 55.7 ms (1.01) 37.0 → 36.6 ms (0.99) 48.1 → 53.3 ms (1.11) 🔴
tpcds_q21 63.6 → 61.3 ms (0.97) 74.5 → 76.3 ms (1.02) 55.2 → 52.9 ms (0.96) 30.2 → 29.4 ms (0.97)
tpcds_q22 478 → 474 ms (0.99) 477 → 479 ms (1.00) 169 → 171 ms (1.01) 164 → 168 ms (1.03)
tpcds_q23 351 → 347 ms (0.99) 405 → 413 ms (1.02) 202 → 204 ms (1.01) 194 → 191 ms (0.99)
tpcds_q24 160 → 163 ms (1.02) 275 → 279 ms (1.01) 73.8 → 71.3 ms (0.97) 83.5 → 82.3 ms (0.99)
tpcds_q25 95.6 → 93.6 ms (0.98) 183 → 183 ms (1.00) 75.1 → 73.8 ms (0.98) 61.2 → 56.7 ms (0.93)
tpcds_q26 61.7 → 59.7 ms (0.97) 77.4 → 77.7 ms (1.00) 49.3 → 40.8 ms (0.83) 🟢 52.3 → 49.7 ms (0.95)
tpcds_q27 107 → 113 ms (1.05) 237 → 236 ms (1.00) 54.2 → 54.7 ms (1.01) 75.7 → 68.5 ms (0.90)
tpcds_q28 91.9 → 92.3 ms (1.01) 120 → 113 ms (0.94) 66.5 → 63.4 ms (0.95) 111 → 105 ms (0.94)
tpcds_q29 106 → 96.8 ms (0.92) 179 → 169 ms (0.95) 75.1 → 69.7 ms (0.93) 98.0 → 100 ms (1.03)
tpcds_q30 79.0 → 76.4 ms (0.97) 123 → 119 ms (0.97) 51.0 → 50.6 ms (0.99) 80.2 → 78.2 ms (0.98)
tpcds_q31 239 → 235 ms (0.98) 351 → 341 ms (0.97) 55.1 → 55.4 ms (1.01) 116 → 109 ms (0.94)
tpcds_q32 33.7 → 31.6 ms (0.94) 44.6 → 43.0 ms (0.96) 30.5 → 29.4 ms (0.96) 26.5 → 25.7 ms (0.97)
tpcds_q33 70.6 → 67.2 ms (0.95) 76.6 → 79.0 ms (1.03) 57.6 → 57.9 ms (1.00) 73.5 → 71.7 ms (0.97)
tpcds_q34 70.9 → 67.9 ms (0.96) 85.4 → 83.4 ms (0.98) 40.5 → 40.6 ms (1.00) 42.6 → 40.2 ms (0.94)
tpcds_q35 128 → 121 ms (0.94) 152 → 149 ms (0.98) 89.7 → 88.9 ms (0.99) 86.4 → 88.4 ms (1.02)
tpcds_q36 103 → 108 ms (1.04) 129 → 130 ms (1.01) 50.3 → 49.9 ms (0.99) 113 → 114 ms (1.01)
tpcds_q37 44.6 → 41.2 ms (0.92) 68.1 → 67.0 ms (0.98) 69.0 → 63.3 ms (0.92) 46.1 → 45.9 ms (0.99)
tpcds_q38 108 → 108 ms (1.00) 144 → 150 ms (1.04) 72.9 → 79.7 ms (1.09) 83.9 → 80.6 ms (0.96)
tpcds_q39 848 → 842 ms (0.99) 867 → 979 ms (1.13) 🔴 83.3 → 76.3 ms (0.92) 67.3 → 67.1 ms (1.00)
tpcds_q40 49.0 → 48.2 ms (0.98) 57.8 → 53.3 ms (0.92) 35.2 → 34.6 ms (0.98) 36.2 → 34.4 ms (0.95)
tpcds_q41 33.7 → 31.8 ms (0.94) 42.9 → 42.6 ms (0.99) 26.4 → 26.5 ms (1.00) 26.9 → 26.7 ms (0.99)
tpcds_q42 39.8 → 41.7 ms (1.05) 47.5 → 44.5 ms (0.94) 31.7 → 32.0 ms (1.01) 39.1 → 35.6 ms (0.91)
tpcds_q43 41.4 → 39.8 ms (0.96) 49.9 → 46.9 ms (0.94) 37.4 → 34.6 ms (0.93) 36.2 → 38.3 ms (1.06)
tpcds_q44 72.5 → 72.1 ms (0.99) 93.2 → 88.8 ms (0.95) 48.9 → 47.4 ms (0.97) 77.1 → 75.3 ms (0.98)
tpcds_q45 65.7 → 60.9 ms (0.93) 94.7 → 87.1 ms (0.92) 61.9 → 60.3 ms (0.97) 67.1 → 67.6 ms (1.01)
tpcds_q46 108 → 100 ms (0.93) 153 → 145 ms (0.95) 76.6 → 74.0 ms (0.97) 92.0 → 90.1 ms (0.98)
tpcds_q47 229 → 228 ms (0.99) 275 → 280 ms (1.02) 102 → 96.1 ms (0.94) 103 → 102 ms (1.00)
tpcds_q48 69.8 → 68.5 ms (0.98) 107 → 101 ms (0.95) 81.3 → 77.0 ms (0.95) 80.8 → 79.5 ms (0.98)
tpcds_q49 120 → 122 ms (1.02) 172 → 176 ms (1.02) 77.7 → 75.1 ms (0.97) 92.0 → 88.4 ms (0.96)
tpcds_q50 84.5 → 84.3 ms (1.00) 123 → 119 ms (0.97) 57.2 → 51.9 ms (0.91) 56.7 → 52.2 ms (0.92)
tpcds_q51 277 → 276 ms (0.99) 281 → 279 ms (0.99) 220 → 220 ms (1.00) 228 → 224 ms (0.98)
tpcds_q52 40.4 → 39.8 ms (0.99) 47.8 → 45.4 ms (0.95) 33.1 → 31.9 ms (0.96) 37.9 → 33.5 ms (0.88) 🟢
tpcds_q53 49.9 → 50.6 ms (1.01) 59.0 → 57.2 ms (0.97) 40.0 → 40.0 ms (1.00) 47.1 → 45.6 ms (0.97)
tpcds_q54 90.7 → 87.6 ms (0.97) 112 → 107 ms (0.96) 66.5 → 65.2 ms (0.98) 64.7 → 59.4 ms (0.92)
tpcds_q55 39.7 → 39.8 ms (1.00) 46.4 → 44.1 ms (0.95) 33.7 → 31.8 ms (0.95) 37.9 → 32.7 ms (0.86) 🟢
tpcds_q56 63.1 → 61.2 ms (0.97) 72.8 → 75.4 ms (1.04) 57.5 → 55.1 ms (0.96) 77.7 → 80.3 ms (1.03)
tpcds_q57 162 → 159 ms (0.98) 195 → 197 ms (1.01) 76.2 → 80.0 ms (1.05) 74.2 → 73.9 ms (0.99)
tpcds_q58 86.0 → 84.2 ms (0.98) 128 → 126 ms (0.98) 61.3 → 59.2 ms (0.97) 69.0 → 65.9 ms (0.95)
tpcds_q59 83.2 → 84.6 ms (1.02) 136 → 129 ms (0.95) 68.5 → 70.8 ms (1.03) 68.8 → 67.3 ms (0.98)
tpcds_q60 67.5 → 62.9 ms (0.93) 75.5 → 76.2 ms (1.01) 58.1 → 57.3 ms (0.99) 95.6 → 89.9 ms (0.94)
tpcds_q61 69.4 → 68.8 ms (0.99) 104 → 89.4 ms (0.86) 🟢 55.2 → 54.9 ms (1.00) 94.5 → 94.4 ms (1.00)
tpcds_q62 40.3 → 39.9 ms (0.99) 44.9 → 44.2 ms (0.98) 26.7 → 26.0 ms (0.98) 26.6 → 24.1 ms (0.91)
tpcds_q63 49.9 → 47.7 ms (0.96) 57.9 → 57.0 ms (0.98) 40.4 → 38.7 ms (0.96) 42.6 → 44.5 ms (1.04)
tpcds_q64 394 → 399 ms (1.01) 736 → 739 ms (1.00) 149 → 155 ms (1.04) 150 → 152 ms (1.02)
tpcds_q65 113 → 108 ms (0.95) 137 → 135 ms (0.99) 69.2 → 62.8 ms (0.91) 102 → 93.6 ms (0.91)
tpcds_q66 79.3 → 75.5 ms (0.95) 102 → 101 ms (0.99) 59.9 → 57.6 ms (0.96) 54.9 → 55.7 ms (1.01)
tpcds_q67 522 → 518 ms (0.99) 499 → 499 ms (1.00) 280 → 290 ms (1.04) 304 → 296 ms (0.98)
tpcds_q68 97.8 → 101 ms (1.03) 157 → 153 ms (0.97) 70.4 → 68.0 ms (0.97) 78.1 → 75.6 ms (0.97)
tpcds_q69 66.7 → 63.4 ms (0.95) 77.1 → 80.3 ms (1.04) 61.2 → 59.4 ms (0.97) 75.3 → 76.6 ms (1.02)
tpcds_q70 232 → 230 ms (0.99) 248 → 243 ms (0.98) 43.2 → 43.0 ms (1.00) 54.8 → 54.5 ms (1.00)
tpcds_q71 58.4 → 58.9 ms (1.01) 65.0 → 70.9 ms (1.09) 51.4 → 50.8 ms (0.99) 55.2 → 52.6 ms (0.95)
tpcds_q72 1.62 → 1.69 s (1.04) 1.79 → 1.88 s (1.06) 159 → 157 ms (0.99) 187 → 183 ms (0.98)
tpcds_q73 66.1 → 65.4 ms (0.99) 85.9 → 86.8 ms (1.01) 40.1 → 39.1 ms (0.97) 35.7 → 35.3 ms (0.99)
tpcds_q74 170 → 173 ms (1.02) 325 → 328 ms (1.01) 103 → 96.9 ms (0.94) 222 → 228 ms (1.03)
tpcds_q75 193 → 200 ms (1.04) 239 → 228 ms (0.95) 138 → 136 ms (0.98) 148 → 147 ms (0.99)
tpcds_q76 75.6 → 73.6 ms (0.97) 80.8 → 81.7 ms (1.01) 56.9 → 54.6 ms (0.96) 67.7 → 63.6 ms (0.94)
tpcds_q77 90.5 → 89.9 ms (0.99) 102 → 96.2 ms (0.94) 62.6 → 62.1 ms (0.99) 69.9 → 66.7 ms (0.95)
tpcds_q78 193 → 188 ms (0.97) 269 → 267 ms (0.99) 161 → 158 ms (0.98) 160 → 171 ms (1.06)
tpcds_q79 84.3 → 86.4 ms (1.02) 115 → 108 ms (0.94) 60.9 → 59.2 ms (0.97) 69.8 → 66.7 ms (0.95)
tpcds_q80 130 → 125 ms (0.96) 197 → 191 ms (0.97) 93.0 → 93.9 ms (1.01) 104 → 95.9 ms (0.92)
tpcds_q81 83.1 → 81.3 ms (0.98) 130 → 129 ms (0.99) 70.7 → 69.3 ms (0.98) 106 → 105 ms (0.99)
tpcds_q82 62.9 → 49.3 ms (0.78) 🟢 71.4 → 72.9 ms (1.02) 70.3 → 62.9 ms (0.90) 🟢 53.4 → 51.6 ms (0.97)
tpcds_q83 68.6 → 66.3 ms (0.97) 85.1 → 82.1 ms (0.96) 33.5 → 33.0 ms (0.98) 33.4 → 33.3 ms (1.00)
tpcds_q84 32.5 → 30.3 ms (0.93) 49.6 → 49.5 ms (1.00) 25.3 → 23.3 ms (0.92) 29.9 → 28.2 ms (0.94)
tpcds_q85 91.3 → 90.8 ms (0.99) 133 → 126 ms (0.95) 52.9 → 53.5 ms (1.01) 61.1 → 60.6 ms (0.99)
tpcds_q86 39.6 → 36.6 ms (0.92) 39.6 → 38.9 ms (0.98) 29.6 → 28.7 ms (0.97) 32.0 → 30.0 ms (0.94)
tpcds_q87 109 → 108 ms (0.99) 146 → 143 ms (0.98) 91.0 → 85.9 ms (0.94) 93.5 → 93.6 ms (1.00)
tpcds_q88 105 → 99.6 ms (0.95) 116 → 109 ms (0.94) 51.5 → 52.5 ms (1.02) 110 → 114 ms (1.04)
tpcds_q89 54.3 → 55.1 ms (1.01) 64.6 → 60.7 ms (0.94) 43.5 → 43.6 ms (1.00) 46.8 → 48.3 ms (1.03)
tpcds_q90 33.3 → 33.9 ms (1.02) 34.3 → 33.4 ms (0.97) 21.6 → 20.4 ms (0.94) 21.8 → 19.4 ms (0.89) 🟢
tpcds_q91 41.6 → 39.1 ms (0.94) 58.9 → 57.0 ms (0.97) 30.2 → 29.0 ms (0.96) 33.5 → 31.8 ms (0.95)
tpcds_q92 37.6 → 35.4 ms (0.94) 47.1 → 47.4 ms (1.01) 25.9 → 27.3 ms (1.05) 29.6 → 29.2 ms (0.98)
tpcds_q93 90.3 → 90.2 ms (1.00) 88.6 → 84.5 ms (0.95) 51.8 → 49.1 ms (0.95) 51.1 → 46.2 ms (0.90)
tpcds_q94 46.6 → 45.7 ms (0.98) 57.4 → 57.4 ms (1.00) 45.5 → 46.3 ms (1.02) 46.7 → 45.3 ms (0.97)
tpcds_q95 126 → 127 ms (1.01) 158 → 158 ms (1.00) 105 → 104 ms (0.99) 86.4 → 89.2 ms (1.03)
tpcds_q96 34.9 → 31.5 ms (0.90) 35.1 → 36.5 ms (1.04) 27.2 → 28.4 ms (1.04) 24.0 → 23.3 ms (0.97)
tpcds_q97 68.9 → 68.9 ms (1.00) 81.4 → 81.7 ms (1.00) 83.4 → 77.8 ms (0.93) 82.4 → 78.8 ms (0.96)
tpcds_q98 48.7 → 48.6 ms (1.00) 83.3 → 86.9 ms (1.04) 47.4 → 45.8 ms (0.97) 67.1 → 62.5 ms (0.93)
tpcds_q99 48.0 → 46.8 ms (0.97) 59.1 → 58.5 ms (0.99) 36.8 → 36.9 ms (1.00) 39.9 → 32.4 ms (0.81) 🟢

No file size changes detected.

…nfig

StructListOfInts wrote its Parquet input with parquet-rs's defaults and
kept a cached file in any layout. It now uses parquet_writer_properties()
and writes a cached file in an older layout again, like the other
compression datasets.

Signed-off-by: Will Manning <will@willmanning.io>
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Benchmarks: Compression 📖

Commits: PR 3b15172c vs base 967bcb67


vortex / vortex-file-compressed / ns (1.000x ➖, 1↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
compress time/arade 452380764 / 453251942 / -0.2% 450099996 / 454379981 / -0.9% 1.01 / 1.00 / +0.8%
compress time/bimbo 2326652218 / 2358415508 / -1.3% 2321964458 / 2357273515 / -1.5% 1.00 / 1.00 / +0.2%
compress time/cmsprovider 1049818572 / 1049316704 / +0.0% 1034290625 / 1036546927 / -0.2% 1.02 / 1.01 / +0.3%
compress time/euro2016 178910094 / 175607957 / +1.9% 178224335 / 175728459 / +1.4% 1.00 / 1.00 / +0.5%
compress time/food 208214486 / 208327590 / -0.1% 209048986 / 208365166 / +0.3% 1.00 / 1.00 / -0.4%
compress time/hashtags 371184572 / 369430363 / +0.5% 365689344 / 365053366 / +0.2% 1.02 / 1.01 / +0.3%
compress time/sparse-wide-flat-runs 93517225 / 87058345 / +7.4% 92282310 / 91803907 / +0.5% 1.01 / 0.95 / +6.9%
compress time/sparse-wide-lists-runs 676557832 / 687884924 / -1.6% 652095669 / 682492716 / -4.5% 1.04 / 1.01 / +2.9%
compress time/sparse-wide-lists-shuffled 301164969 / 303138280 / -0.7% 291635544 / 289789052 / +0.6% 1.03 / 1.05 / -1.3%
compress time/sparse-wide-nested-child-nulls 620246558 / 618214899 / +0.3% 634346129 / 621239353 / +2.1% 0.98 / 1.00 / -1.7%
compress time/sparse-wide-nested-runs 609640772 / 630863454 / -3.4% 622710247 / 628834889 / -1.0% 0.98 / 1.00 / -2.4%
compress time/statpopgen 495601028 / 530405310 / -6.6% 505582692 / 492232034 / +2.7% 0.98 / 1.08 / -9.0%
compress time/taxi 205779942 / 207549552 / -0.9% 205775577 / 207604568 / -0.9% 1.00 / 1.00 / +0.0%
compress time/tpc-h l_comment canonical 103466178 / 108305624 / -4.5% 104627493 / 109379585 / -4.3% 0.99 / 0.99 / -0.1%
compress time/tpc-h l_comment chunked 103430035 / 108308638 / -4.5% 104646642 / 109514671 / -4.4% 0.99 / 0.99 / -0.1%
compress time/tpc-h sf1 lineitem 370326377 / 397578234 / -6.9% 369210888 / 396491398 / -6.9% 1.00 / 1.00 / +0.0%
compress time/tpc-h sf1 orders 81265201 / 87424904 / -7.0% 81581936 / 87314649 / -6.6% 1.00 / 1.00 / -0.5%
compress time/tpc-h sf1 partsupp 58317178 / 58742966 / -0.7% 59349732 / 59417043 / -0.1% 0.98 / 0.99 / -0.6%
compress time/wide table cols=100 chunks=1 rows=1000 5667470 / 5714562 / -0.8% 7407816 / 7738415 / -4.3% 0.77 / 0.74 / +3.6%
compress time/wide table cols=100 chunks=50 rows=1000 5784432 / 5795953 / -0.2% 7935937 / 7047374 / +12.6% 🔴 0.73 / 0.82 / -11.4%
compress time/wide table cols=1000 chunks=1 rows=1000 55309344 / 54772577 / +1.0% 56685121 / 58067698 / -2.4% 0.98 / 0.94 / +3.4%
compress time/wide table cols=1000 chunks=50 rows=1000 55128080 / 54644396 / +0.9% 59097803 / 57897361 / +2.1% 0.93 / 0.94 / -1.2%
compress time/wide table cols=10000 chunks=1 rows=1000 635827809 / 626342008 / +1.5% 638817122 / 623715733 / +2.4% 1.00 / 1.00 / -0.9%
compress time/wide table cols=10000 chunks=50 rows=1000 635381616 / 626159378 / +1.5% 639338194 / 625249616 / +2.3% 0.99 / 1.00 / -0.8%
decompress time/arade 15077924 / 15032994 / +0.3% 15964832 / 15968731 / -0.0% 0.94 / 0.94 / +0.3%
decompress time/bimbo 67420892 / 68334638 / -1.3% 70377023 / 70895436 / -0.7% 0.96 / 0.96 / -0.6%
decompress time/cmsprovider 39515568 / 39935386 / -1.1% 45776440 / 41502947 / +10.3% 🔴 0.86 / 0.96 / -10.3%
decompress time/euro2016 12615973 / 12559920 / +0.4% 14069525 / 13277428 / +6.0% 0.90 / 0.95 / -5.2%
decompress time/food 4164759 / 4043074 / +3.0% 6071894 / 5148267 / +17.9% 🔴 0.69 / 0.79 / -12.7%
decompress time/hashtags 57354650 / 56636758 / +1.3% 58067904 / 56577013 / +2.6% 0.99 / 1.00 / -1.3%
decompress time/sparse-wide-flat-runs 12231828 / 14160669 / -13.6% 🟢 13067350 / 14831954 / -11.9% 🟢 0.94 / 0.95 / -2.0%
decompress time/sparse-wide-lists-runs 16864232 / 16443014 / +2.6% 17467860 / 16846598 / +3.7% 0.97 / 0.98 / -1.1%
decompress time/sparse-wide-lists-shuffled 27261438 / 27309167 / -0.2% 28026986 / 27565151 / +1.7% 0.97 / 0.99 / -1.8%
decompress time/sparse-wide-nested-child-nulls 17663058 / 16201674 / +9.0% 18228615 / 17628624 / +3.4% 0.97 / 0.92 / +5.4%
decompress time/sparse-wide-nested-runs 17204694 / 14340063 / +20.0% 🔴 16574888 / 15214435 / +8.9% 1.04 / 0.94 / +10.1%
decompress time/statpopgen 713495262 / 704829202 / +1.2% 701215206 / 717690271 / -2.3% 1.02 / 0.98 / +3.6%
decompress time/taxi 10457596 / 10528617 / -0.7% 13712579 / 11884204 / +15.4% 🔴 0.76 / 0.89 / -13.9%
decompress time/tpc-h l_comment canonical 5208384 / 5183045 / +0.5% 6135328 / 5806063 / +5.7% 0.85 / 0.89 / -4.9%
decompress time/tpc-h l_comment chunked 5307396 / 5220697 / +1.7% 6320993 / 5499850 / +14.9% 🔴 0.84 / 0.95 / -11.5%
decompress time/tpc-h sf1 lineitem 15671694 / 15916160 / -1.5% 18459896 / 16141212 / +14.4% 🔴 0.85 / 0.99 / -13.9%
decompress time/tpc-h sf1 orders 4260130 / 4191924 / +1.6% 5628714 / 4981444 / +13.0% 🔴 0.76 / 0.84 / -10.1%
decompress time/tpc-h sf1 partsupp 3267690 / 3305642 / -1.1% 4676587 / 4009747 / +16.6% 🔴 0.70 / 0.82 / -15.2%
decompress time/wide table cols=100 chunks=1 rows=1000 1536400 / 1553400 / -1.1% 2241476 / 1682867 / +33.2% 🔴 0.69 / 0.92 / -25.7%
decompress time/wide table cols=100 chunks=50 rows=1000 1567928 / 1551086 / +1.1% 2190691 / 1666221 / +31.5% 🔴 0.72 / 0.93 / -23.1%
decompress time/wide table cols=1000 chunks=1 rows=1000 13934484 / 13734483 / +1.5% 14952481 / 14417107 / +3.7% 0.93 / 0.95 / -2.2%
decompress time/wide table cols=1000 chunks=50 rows=1000 13967796 / 13825734 / +1.0% 14891586 / 14421552 / +3.3% 0.94 / 0.96 / -2.2%
decompress time/wide table cols=10000 chunks=1 rows=1000 146625724 / 143272815 / +2.3% 148631803 / 144320359 / +3.0% 0.99 / 0.99 / -0.6%
decompress time/wide table cols=10000 chunks=50 rows=1000 146543295 / 142889612 / +2.6% 148629403 / 143550807 / +3.5% 0.99 / 1.00 / -0.9%
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
vortex-file-compressed size/arade 141323304 / 141323316 / -0.0%
vortex-file-compressed size/bimbo 410263716 / 410263708 / +0.0%
vortex-file-compressed size/cmsprovider 413116844 / 413116844 / +0.0%
vortex-file-compressed size/euro2016 152436976 / 152436620 / +0.0%
vortex-file-compressed size/food 37247736 / 37247744 / -0.0%
vortex-file-compressed size/hashtags 169636480 / 169636564 / -0.0%
vortex-file-compressed size/sparse-wide-flat-runs 1552504 / 1552504 / +0.0%
vortex-file-compressed size/sparse-wide-lists-runs 3948028 / 3948028 / +0.0%
vortex-file-compressed size/sparse-wide-lists-shuffled 6436588 / 6436588 / +0.0%
vortex-file-compressed size/sparse-wide-nested-child-nulls 1687476 / 1687476 / +0.0%
vortex-file-compressed size/sparse-wide-nested-runs 1298532 / 1298532 / +0.0%
vortex-file-compressed size/statpopgen 232298372 / 232298372 / +0.0%
vortex-file-compressed size/taxi 48083012 / 48083012 / +0.0%
vortex-file-compressed size/tpc-h l_comment canonical 64987292 / 64987292 / +0.0%
vortex-file-compressed size/tpc-h l_comment chunked 64987292 / 64987292 / +0.0%
vortex-file-compressed size/tpc-h sf1 lineitem 154809600 / 154809600 / +0.0%
vortex-file-compressed size/tpc-h sf1 orders 33372900 / 33372900 / +0.0%
vortex-file-compressed size/tpc-h sf1 partsupp 24526904 / 24526904 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=1 rows=1000 934968 / 934968 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=50 rows=1000 934968 / 934968 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=1 rows=1000 9333768 / 9333768 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=50 rows=1000 9333768 / 9333768 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=1 rows=1000 93357768 / 93357768 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=50 rows=1000 93357768 / 93357768 / +0.0%
vortex / vortex-compact / bytes (no group data, 0↑ 0↓)
name bytes (PR / base / %diff)
vortex-compact size/arade 118010412 / — / no baseline
vortex-compact size/bimbo 286506120 / — / no baseline
vortex-compact size/cmsprovider 291372428 / — / no baseline
vortex-compact size/euro2016 98681008 / — / no baseline
vortex-compact size/food 23380392 / — / no baseline
vortex-compact size/hashtags 117079900 / — / no baseline
vortex-compact size/sparse-wide-flat-runs 1471256 / — / no baseline
vortex-compact size/sparse-wide-lists-runs 3360580 / — / no baseline
vortex-compact size/sparse-wide-lists-shuffled 4991844 / — / no baseline
vortex-compact size/sparse-wide-nested-child-nulls 1178640 / — / no baseline
vortex-compact size/sparse-wide-nested-runs 1246036 / — / no baseline
vortex-compact size/statpopgen 115591892 / — / no baseline
vortex-compact size/taxi 34884244 / — / no baseline
vortex-compact size/tpc-h l_comment canonical 52939140 / — / no baseline
vortex-compact size/tpc-h l_comment chunked 52939140 / — / no baseline
vortex-compact size/tpc-h sf1 lineitem 132004072 / — / no baseline
vortex-compact size/tpc-h sf1 orders 33242120 / — / no baseline
vortex-compact size/tpc-h sf1 partsupp 21283884 / — / no baseline
vortex-compact size/wide table cols=100 chunks=1 rows=1000 934968 / — / no baseline
vortex-compact size/wide table cols=100 chunks=50 rows=1000 934968 / — / no baseline
vortex-compact size/wide table cols=1000 chunks=1 rows=1000 9333768 / — / no baseline
vortex-compact size/wide table cols=1000 chunks=50 rows=1000 9333768 / — / no baseline
vortex-compact size/wide table cols=10000 chunks=1 rows=1000 93357768 / — / no baseline
vortex-compact size/wide table cols=10000 chunks=50 rows=1000 93357768 / — / no baseline
vortex / parquet / ns (1.017x ➖, 2↑ 6↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
compress time/arade 830639202 / 1160328536 / -28.4% 🟢 831561292 / 1159376320 / -28.3% 🟢 1.00 / 1.00 / -0.2%
compress time/bimbo 2983144388 / 2147024276 / +38.9% 🔴 2993703024 / 2145385744 / +39.5% 🔴 1.00 / 1.00 / -0.4%
compress time/cmsprovider 1710421958 / 1670448660 / +2.4% 1702156807 / 1670631678 / +1.9% 1.00 / 1.00 / +0.5%
compress time/euro2016 979826119 / 995880478 / -1.6% 1010167650 / 997634888 / +1.3% 0.97 / 1.00 / -2.8%
compress time/food 437323437 / 407477375 / +7.3% 439460003 / 408283810 / +7.6% 1.00 / 1.00 / -0.3%
compress time/hashtags 465163381 / 539002606 / -13.7% 🟢 475482645 / 548600928 / -13.3% 🟢 0.98 / 0.98 / -0.4%
compress time/sparse-wide-flat-runs 41583537 / 41741338 / -0.4% 42176484 / 42538826 / -0.9% 0.99 / 0.98 / +0.5%
compress time/sparse-wide-lists-runs 81343052 / 80975154 / +0.5% 81777193 / 81975984 / -0.2% 0.99 / 0.99 / +0.7%
compress time/sparse-wide-lists-shuffled 155678656 / 155494140 / +0.1% 156197044 / 156248842 / -0.0% 1.00 / 1.00 / +0.2%
compress time/sparse-wide-nested-child-nulls 55010700 / 54481398 / +1.0% 55822940 / 55762523 / +0.1% 0.99 / 0.98 / +0.9%
compress time/sparse-wide-nested-runs 54934181 / 55069196 / -0.2% 55945986 / 55811953 / +0.2% 0.98 / 0.99 / -0.5%
compress time/statpopgen 1024562630 / 1024853265 / -0.0% 1031191959 / 1029789093 / +0.1% 0.99 / 1.00 / -0.2%
compress time/taxi 279145700 / 240792032 / +15.9% 🔴 282342601 / 240923168 / +17.2% 🔴 0.99 / 1.00 / -1.1%
compress time/tpc-h l_comment canonical 714660532 / 721412272 / -0.9% 715785772 / 723672587 / -1.1% 1.00 / 1.00 / +0.2%
compress time/tpc-h l_comment chunked 713926475 / 721343260 / -1.0% 715185262 / 724146319 / -1.2% 1.00 / 1.00 / +0.2%
compress time/tpc-h sf1 lineitem 1185962822 / 1198400120 / -1.0% 1200367802 / 1204954920 / -0.4% 0.99 / 0.99 / -0.7%
compress time/tpc-h sf1 orders 331770534 / 342622394 / -3.2% 333893638 / 343944160 / -2.9% 0.99 / 1.00 / -0.3%
compress time/tpc-h sf1 partsupp 310351736 / 305387464 / +1.6% 311215063 / 309964216 / +0.4% 1.00 / 0.99 / +1.2%
compress time/wide table cols=100 chunks=1 rows=1000 3758522 / 3799342 / -1.1% 4398838 / 4451901 / -1.2% 0.85 / 0.85 / +0.1%
compress time/wide table cols=100 chunks=50 rows=1000 3567287 / 3767418 / -5.3% 4312320 / 4589847 / -6.0% 0.83 / 0.82 / +0.8%
compress time/wide table cols=1000 chunks=1 rows=1000 37856210 / 37976598 / -0.3% 38882738 / 39535334 / -1.7% 0.97 / 0.96 / +1.4%
compress time/wide table cols=1000 chunks=50 rows=1000 36751219 / 37482002 / -1.9% 37931212 / 38822964 / -2.3% 0.97 / 0.97 / +0.4%
compress time/wide table cols=10000 chunks=1 rows=1000 377398733 / 379325772 / -0.5% 383294611 / 390309002 / -1.8% 0.98 / 0.97 / +1.3%
compress time/wide table cols=10000 chunks=50 rows=1000 374736620 / 377099912 / -0.6% 380890327 / 386282337 / -1.4% 0.98 / 0.98 / +0.8%
decompress time/arade 54167767 / 54637864 / -0.9% 54267287 / 54961667 / -1.3% 1.00 / 0.99 / +0.4%
decompress time/bimbo 109584734 / 92663999 / +18.3% 🔴 109905694 / 92615800 / +18.7% 🔴 1.00 / 1.00 / -0.3%
decompress time/cmsprovider 135819976 / 103547482 / +31.2% 🔴 135507165 / 103091553 / +31.4% 🔴 1.00 / 1.00 / -0.2%
decompress time/euro2016 158303222 / 159439000 / -0.7% 158735334 / 159864177 / -0.7% 1.00 / 1.00 / -0.0%
decompress time/food 28250830 / 26583921 / +6.3% 28373272 / 26764613 / +6.0% 1.00 / 0.99 / +0.2%
decompress time/hashtags 467384051 / 446066394 / +4.8% 468113137 / 446165976 / +4.9% 1.00 / 1.00 / -0.1%
decompress time/sparse-wide-flat-runs 14923746 / 15049036 / -0.8% 15313891 / 15659114 / -2.2% 0.97 / 0.96 / +1.4%
decompress time/sparse-wide-lists-runs 276790248 / 277301176 / -0.2% 277041519 / 277297382 / -0.1% 1.00 / 1.00 / -0.1%
decompress time/sparse-wide-lists-shuffled 318161084 / 319423308 / -0.4% 318299811 / 320060912 / -0.6% 1.00 / 1.00 / +0.2%
decompress time/sparse-wide-nested-child-nulls 52853827 / 53097772 / -0.5% 53366040 / 53387326 / -0.0% 0.99 / 0.99 / -0.4%
decompress time/sparse-wide-nested-runs 52623282 / 52978630 / -0.7% 52951815 / 53370394 / -0.8% 0.99 / 0.99 / +0.1%
decompress time/statpopgen 2917296510 / 2903683058 / +0.5% 2959442865 / 2937816936 / +0.7% 0.99 / 0.99 / -0.3%
decompress time/taxi 68447654 / 60835765 / +12.5% 🔴 68666942 / 61026247 / +12.5% 🔴 1.00 / 1.00 / -0.0%
decompress time/tpc-h l_comment canonical 35885768 / 36022972 / -0.4% 36025289 / 36112463 / -0.2% 1.00 / 1.00 / -0.1%
decompress time/tpc-h l_comment chunked 35878954 / 36010860 / -0.4% 36021399 / 36154631 / -0.4% 1.00 / 1.00 / +0.0%
decompress time/tpc-h sf1 lineitem 92451533 / 82141227 / +12.6% 🔴 92646892 / 82351428 / +12.5% 🔴 1.00 / 1.00 / +0.0%
decompress time/tpc-h sf1 orders 95652211 / 88084118 / +8.6% 95972173 / 88526357 / +8.4% 1.00 / 1.00 / +0.2%
decompress time/tpc-h sf1 partsupp 122353276 / 111293550 / +9.9% 122675431 / 111646997 / +9.9% 1.00 / 1.00 / +0.1%
decompress time/wide table cols=100 chunks=1 rows=1000 1943370 / 1973928 / -1.5% 2209182 / 2260692 / -2.3% 0.88 / 0.87 / +0.7%
decompress time/wide table cols=100 chunks=50 rows=1000 1946924 / 1948178 / -0.1% 2134769 / 2088772 / +2.2% 0.91 / 0.93 / -2.2%
decompress time/wide table cols=1000 chunks=1 rows=1000 22449910 / 22665608 / -1.0% 22982942 / 23179230 / -0.8% 0.98 / 0.98 / -0.1%
decompress time/wide table cols=1000 chunks=50 rows=1000 22242647 / 22611714 / -1.6% 22533136 / 23179996 / -2.8% 0.99 / 0.98 / +1.2%
decompress time/wide table cols=10000 chunks=1 rows=1000 231901916 / 233432913 / -0.7% 235221037 / 236723852 / -0.6% 0.99 / 0.99 / -0.0%
decompress time/wide table cols=10000 chunks=50 rows=1000 229901484 / 233081674 / -1.4% 231680772 / 234902915 / -1.4% 0.99 / 0.99 / +0.0%
vortex / parquet / bytes (1.018x ➖, 0↑ 2↓)
name bytes (PR / base / %diff)
parquet size/arade 201351339 / 210020851 / -4.1%
parquet size/bimbo 357921328 / 376187078 / -4.9%
parquet size/cmsprovider 372629831 / 364268159 / +2.3%
parquet size/euro2016 108307574 / 111035158 / -2.5%
parquet size/food 34549350 / 34903783 / -1.0%
parquet size/hashtags 126827169 / 126189203 / +0.5%
parquet size/sparse-wide-flat-runs 1496302 / 1388944 / +7.7%
parquet size/sparse-wide-lists-runs 4521942 / 4327813 / +4.5%
parquet size/sparse-wide-lists-shuffled 5927637 / 5710569 / +3.8%
parquet size/sparse-wide-nested-child-nulls 1417612 / 1266042 / +12.0% 🔴
parquet size/sparse-wide-nested-runs 1540074 / 1388944 / +10.9% 🔴
parquet size/statpopgen 81651718 / 81284006 / +0.5%
parquet size/taxi 52652659 / 52142534 / +1.0%
parquet size/tpc-h l_comment canonical 51890033 / 52204116 / -0.6%
parquet size/tpc-h l_comment chunked 51890033 / 52204116 / -0.6%
parquet size/tpc-h sf1 lineitem 150053395 / 153403836 / -2.2%
parquet size/tpc-h sf1 orders 36640529 / 37029542 / -1.1%
parquet size/tpc-h sf1 partsupp 28091851 / 29200612 / -3.8%
parquet size/wide table cols=100 chunks=1 rows=1000 968004 / 932404 / +3.8%
parquet size/wide table cols=100 chunks=50 rows=1000 968004 / 932404 / +3.8%
parquet size/wide table cols=1000 chunks=1 rows=1000 9683472 / 9324004 / +3.9%
parquet size/wide table cols=1000 chunks=50 rows=1000 9683472 / 9324004 / +3.9%
parquet size/wide table cols=10000 chunks=1 rows=1000 96902474 / 93240004 / +3.9%
parquet size/wide table cols=10000 chunks=50 rows=1000 96902474 / 93240004 / +3.9%
vortex / parquet-snappy / bytes (no group data, 0↑ 0↓)
name bytes (PR / base / %diff)
parquet-snappy size/arade 306107561 / — / no baseline
parquet-snappy size/bimbo 503944317 / — / no baseline
parquet-snappy size/cmsprovider 493684196 / — / no baseline
parquet-snappy size/euro2016 166575246 / — / no baseline
parquet-snappy size/food 48417835 / — / no baseline
parquet-snappy size/hashtags 201091744 / — / no baseline
parquet-snappy size/sparse-wide-flat-runs 2029564 / — / no baseline
parquet-snappy size/sparse-wide-lists-runs 6065078 / — / no baseline
parquet-snappy size/sparse-wide-lists-shuffled 8052734 / — / no baseline
parquet-snappy size/sparse-wide-nested-child-nulls 1899339 / — / no baseline
parquet-snappy size/sparse-wide-nested-runs 2073336 / — / no baseline
parquet-snappy size/statpopgen 123668955 / — / no baseline
parquet-snappy size/taxi 72778246 / — / no baseline
parquet-snappy size/tpc-h l_comment canonical 77179282 / — / no baseline
parquet-snappy size/tpc-h l_comment chunked 77179282 / — / no baseline
parquet-snappy size/tpc-h sf1 lineitem 214289966 / — / no baseline
parquet-snappy size/tpc-h sf1 orders 56277393 / — / no baseline
parquet-snappy size/tpc-h sf1 partsupp 43780871 / — / no baseline
parquet-snappy size/wide table cols=100 chunks=1 rows=1000 966704 / — / no baseline
parquet-snappy size/wide table cols=100 chunks=50 rows=1000 966704 / — / no baseline
parquet-snappy size/wide table cols=1000 chunks=1 rows=1000 9670472 / — / no baseline
parquet-snappy size/wide table cols=1000 chunks=50 rows=1000 9670472 / — / no baseline
parquet-snappy size/wide table cols=10000 chunks=1 rows=1000 96772474 / — / no baseline
parquet-snappy size/wide table cols=10000 chunks=50 rows=1000 96772474 / — / no baseline
vortex / arrow-ipc / ns (1.001x ➖, 0↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
compress time/arade 91950570 / 95210774 / -3.4% 115150227 / 121731645 / -5.4% 0.80 / 0.78 / +2.1%
compress time/bimbo 582131922 / 601821128 / -3.3% 644047667 / 667182634 / -3.5% 0.90 / 0.90 / +0.2%
compress time/cmsprovider 350449826 / 380177630 / -7.8% 402876520 / 444836149 / -9.4% 0.87 / 0.85 / +1.8%
compress time/euro2016 41535950 / 38405654 / +8.2% 56863337 / 52998471 / +7.3% 0.73 / 0.72 / +0.8%
compress time/food 39227991 / 40278518 / -2.6% 52510947 / 54152349 / -3.0% 0.75 / 0.74 / +0.4%
compress time/hashtags 76772204 / 78173552 / -1.8% 104825757 / 106684090 / -1.7% 0.73 / 0.73 / -0.1%
compress time/sparse-wide-flat-runs 27689445 / 27988243 / -1.1% 36561617 / 37260798 / -1.9% 0.76 / 0.75 / +0.8%
compress time/sparse-wide-lists-runs 44101928 / 45105042 / -2.2% 50631747 / 52065428 / -2.8% 0.87 / 0.87 / +0.5%
compress time/sparse-wide-lists-shuffled 44377079 / 47462588 / -6.5% 49184737 / 50654831 / -2.9% 0.90 / 0.94 / -3.7%
compress time/sparse-wide-nested-child-nulls 40335629 / 40746262 / -1.0% 49795220 / 50758920 / -1.9% 0.81 / 0.80 / +0.9%
compress time/sparse-wide-nested-runs 40376526 / 40999672 / -1.5% 49908924 / 50876584 / -1.9% 0.81 / 0.81 / +0.4%
compress time/statpopgen 216299273 / 216582546 / -0.1% 262342853 / 262940122 / -0.2% 0.82 / 0.82 / +0.1%
compress time/taxi 58120122 / 59107656 / -1.7% 74681049 / 75987698 / -1.7% 0.78 / 0.78 / +0.0%
compress time/tpc-h l_comment canonical 22804530 / 22816388 / -0.1% 30496916 / 31167500 / -2.2% 0.75 / 0.73 / +2.1%
compress time/tpc-h l_comment chunked 22870516 / 22805304 / +0.3% 30354356 / 31328360 / -3.1% 0.75 / 0.73 / +3.5%
compress time/tpc-h sf1 lineitem 128284726 / 130564884 / -1.7% 163691044 / 171953237 / -4.8% 0.78 / 0.76 / +3.2%
compress time/tpc-h sf1 orders 24534019 / 24810600 / -1.1% 33985729 / 35032639 / -3.0% 0.72 / 0.71 / +1.9%
compress time/tpc-h sf1 partsupp 13150638 / 13183793 / -0.3% 18632549 / 19427148 / -4.1% 0.71 / 0.68 / +4.0%
compress time/wide table cols=100 chunks=1 rows=1000 197235.5 / 196866 / +0.2% 680117 / 676810 / +0.5% 0.29 / 0.29 / -0.3%
compress time/wide table cols=100 chunks=50 rows=1000 371058 / 198477.5 / +87.0% 🔴 495563 / 638780 / -22.4% 🟢 0.75 / 0.31 / +141.0%
compress time/wide table cols=1000 chunks=1 rows=1000 2309588 / 2356397 / -2.0% 2835742 / 3278816 / -13.5% 🟢 0.81 / 0.72 / +13.3%
compress time/wide table cols=1000 chunks=50 rows=1000 2297908 / 2327486 / -1.3% 2829258 / 3114569 / -9.2% 0.81 / 0.75 / +8.7%
compress time/wide table cols=10000 chunks=1 rows=1000 24841834 / 25095065 / -1.0% 31206271 / 31458101 / -0.8% 0.80 / 0.80 / -0.2%
compress time/wide table cols=10000 chunks=50 rows=1000 24964722 / 25103357 / -0.6% 31048596 / 31601679 / -1.8% 0.80 / 0.79 / +1.2%
decompress time/arade 209706720 / 217108714 / -3.4% 209893937 / 217456352 / -3.5% 1.00 / 1.00 / +0.1%
decompress time/bimbo 399746057 / 418796642 / -4.5% 406082812 / 418801511 / -3.0% 0.98 / 1.00 / -1.6%
decompress time/cmsprovider 672668552 / 673985294 / -0.2% 673881340 / 674170641 / -0.0% 1.00 / 1.00 / -0.2%
decompress time/euro2016 83422120 / 85661520 / -2.6% 83696652 / 85870731 / -2.5% 1.00 / 1.00 / -0.1%
decompress time/food 65424442 / 67207162 / -2.7% 65605190 / 67267179 / -2.5% 1.00 / 1.00 / -0.2%
decompress time/hashtags 159750573 / 164376028 / -2.8% 160295453 / 167721444 / -4.4% 1.00 / 0.98 / +1.7%
decompress time/sparse-wide-flat-runs 18715718 / 19120208 / -2.1% 19389090 / 19950226 / -2.8% 0.97 / 0.96 / +0.7%
decompress time/sparse-wide-lists-runs 81409182 / 81796192 / -0.5% 81611680 / 82018448 / -0.5% 1.00 / 1.00 / +0.0%
decompress time/sparse-wide-lists-shuffled 81663912 / 82053088 / -0.5% 81802313 / 82172828 / -0.5% 1.00 / 1.00 / -0.0%
decompress time/sparse-wide-nested-child-nulls 27052912 / 26262170 / +3.0% 27600311 / 26878646 / +2.7% 0.98 / 0.98 / +0.3%
decompress time/sparse-wide-nested-runs 27560792 / 26368156 / +4.5% 30521232 / 26872720 / +13.6% 🔴 0.90 / 0.98 / -8.0%
decompress time/statpopgen 469292308 / 462785120 / +1.4% 469737923 / 462988814 / +1.5% 1.00 / 1.00 / -0.1%
decompress time/taxi 50727206 / 50927488 / -0.4% 50887019 / 51189527 / -0.6% 1.00 / 0.99 / +0.2%
decompress time/tpc-h l_comment canonical 39865003 / 40902628 / -2.5% 40180256 / 41242546 / -2.6% 0.99 / 0.99 / +0.0%
decompress time/tpc-h l_comment chunked 39847120 / 39856580 / -0.0% 40206653 / 40158120 / +0.1% 0.99 / 0.99 / -0.1%
decompress time/tpc-h sf1 lineitem 188615032 / 189407384 / -0.4% 189244595 / 189971404 / -0.4% 1.00 / 1.00 / -0.0%
decompress time/tpc-h sf1 orders 38525324 / 38566956 / -0.1% 38610605 / 38680269 / -0.2% 1.00 / 1.00 / +0.1%
decompress time/tpc-h sf1 partsupp 13741779 / 13720300 / +0.2% 13865474 / 13816491 / +0.4% 0.99 / 0.99 / -0.2%
decompress time/wide table cols=100 chunks=1 rows=1000 314039 / 320268 / -1.9% 337204 / 340269 / -0.9% 0.93 / 0.94 / -1.1%
decompress time/wide table cols=100 chunks=50 rows=1000 317904.5 / 316562.5 / +0.4% 356818 / 343031 / +4.0% 0.89 / 0.92 / -3.5%
decompress time/wide table cols=1000 chunks=1 rows=1000 3749047 / 3764028 / -0.4% 3707552 / 3816620 / -2.9% 1.01 / 0.99 / +2.5%
decompress time/wide table cols=1000 chunks=50 rows=1000 3549703 / 3715723 / -4.5% 3668318 / 3764429 / -2.6% 0.97 / 0.99 / -2.0%
decompress time/wide table cols=10000 chunks=1 rows=1000 35739584 / 36360814 / -1.7% 36505658 / 37147990 / -1.7% 0.98 / 0.98 / +0.0%
decompress time/wide table cols=10000 chunks=50 rows=1000 35711831 / 35701624 / +0.0% 36479977 / 36261608 / +0.6% 0.98 / 0.98 / -0.6%
vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
arrow-ipc size/arade 680639738 / 680639722 / +0.0%
arrow-ipc size/bimbo 3168319202 / 3168319202 / +0.0%
arrow-ipc size/cmsprovider 2394412338 / 2394412322 / +0.0%
arrow-ipc size/euro2016 359300058 / 359300042 / +0.0%
arrow-ipc size/food 252069522 / 252069442 / +0.0%
arrow-ipc size/hashtags 700364578 / 700364562 / +0.0%
arrow-ipc size/sparse-wide-flat-runs 245220562 / 245220370 / +0.0%
arrow-ipc size/sparse-wide-lists-runs 142046434 / 142046306 / +0.0%
arrow-ipc size/sparse-wide-lists-shuffled 142871010 / 142870882 / +0.0%
arrow-ipc size/sparse-wide-nested-child-nulls 251527634 / 251527506 / +0.0%
arrow-ipc size/sparse-wide-nested-runs 251527250 / 251527122 / +0.0%
arrow-ipc size/statpopgen 1890451954 / 1890451826 / +0.0%
arrow-ipc size/taxi 474481242 / 474481242 / +0.0%
arrow-ipc size/tpc-h l_comment canonical 184829202 / 184829074 / +0.0%
arrow-ipc size/tpc-h l_comment chunked 184829202 / 184829074 / +0.0%
arrow-ipc size/tpc-h sf1 lineitem 1021072410 / 1021072290 / +0.0%
arrow-ipc size/tpc-h sf1 orders 194812082 / 194812026 / +0.0%
arrow-ipc size/tpc-h sf1 partsupp 131286682 / 131286562 / +0.0%
arrow-ipc size/wide table cols=100 chunks=1 rows=1000 1231818 / 1231690 / +0.0%
arrow-ipc size/wide table cols=100 chunks=50 rows=1000 1231818 / 1231690 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=1 rows=1000 12312634 / 12312506 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=50 rows=1000 12312634 / 12312506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=1 rows=1000 123192634 / 123192506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=50 rows=1000 123192634 / 123192506 / +0.0%

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Random Access (S3) 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +3.3%
Engines: random-access No clear signal (+3.3%, environment too noisy confidence)
Vortex (geomean): hot 0.818x ➖ · cold 0.832x ➖
Parquet (geomean): hot 0.792x ➖ · cold 0.794x ➖
Shifts: Parquet (control) -20.8% · Median polish -9.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

random-access / vortex-file-compressed / ns (0.818x ➖, 2↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors-s3/correlated/vortex-cached 109339.5 / 109256 / +0.1% 109935 / 110431 / -0.4% 0.99 / 0.99 / +0.5%
random-access/feature-vectors-s3/correlated/vortex-reopen 106660758 / 145397152 / -26.6% 226489731 / 241927217 / -6.4% 0.47 / 0.60 / -21.6%
random-access/feature-vectors-s3/uniform/vortex-cached 51339792 / 66307056 / -22.6% 47878547 / 53719331 / -10.9% 1.07 / 1.23 / -13.1%
random-access/feature-vectors-s3/uniform/vortex-reopen 145757592 / 189289773 / -23.0% 267351977 / 304528007 / -12.2% 0.55 / 0.62 / -12.3%
random-access/nested-lists-s3/correlated/vortex-cached 99618 / 103962 / -4.2% 105277 / 108973 / -3.4% 0.95 / 0.95 / -0.8%
random-access/nested-lists-s3/correlated/vortex-reopen 79526643 / 83649855 / -4.9% 179086285 / 240175603 / -25.4% 0.44 / 0.35 / +27.5%
random-access/nested-lists-s3/uniform/vortex-cached 34045687 / 37826565 / -10.0% 34801686 / 41031936 / -15.2% 0.98 / 0.92 / +6.1%
random-access/nested-lists-s3/uniform/vortex-reopen 201963245 / 115453540 / +74.9% 🔴 140891531 / 167209492 / -15.7% 1.43 / 0.69 / +107.6%
random-access/nested-structs-s3/correlated/vortex-cached 122252 / 126066 / -3.0% 126918 / 124388 / +2.0% 0.96 / 1.01 / -5.0%
random-access/nested-structs-s3/correlated/vortex-reopen 76823823 / 87869200 / -12.6% 102102372 / 162531081 / -37.2% 🟢 0.75 / 0.54 / +39.2%
random-access/nested-structs-s3/uniform/vortex-cached 243549.5 / 249553 / -2.4% 250907 / 246743 / +1.7% 0.97 / 1.01 / -4.0%
random-access/nested-structs-s3/uniform/vortex-reopen 89357763 / 94054901 / -5.0% 135449182 / 142906572 / -5.2% 0.66 / 0.66 / +0.2%
random-access/taxi-s3/correlated/vortex-cached 320798 / 326389 / -1.7% 316139 / 329771 / -4.1% 1.01 / 0.99 / +2.5%
random-access/taxi-s3/correlated/vortex-reopen 115596996 / 136188283 / -15.1% 148230451 / 140328674 / +5.6% 0.78 / 0.97 / -19.6%
random-access/taxi-s3/uniform/vortex-cached 852605 / 870556.5 / -2.1% 880762 / 868834 / +1.4% 0.97 / 1.00 / -3.4%
random-access/taxi-s3/uniform/vortex-reopen 126189258 / 379907357 / -66.8% 🟢 171654809 / 441809648 / -61.1% 🟢 0.74 / 0.86 / -14.5%
random-access/taxi-s3/vortex-cached 264637 / 271736.5 / -2.6% 263245 / 265640 / -0.9% 1.01 / 1.02 / -1.7%
random-access/taxi-s3/vortex-reopen 109210845 / 533593788 / -79.5% 🟢 127584948 / 319460459 / -60.1% 🟢 0.86 / 1.67 / -48.8%
random-access / parquet / ns (0.792x ➖, 4↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors-s3/correlated/parquet-cached 42530373 / 52297963 / -18.7% 38545603 / 49036209 / -21.4% 1.10 / 1.07 / +3.5%
random-access/feature-vectors-s3/correlated/parquet-reopen 109895309 / 125037001 / -12.1% 139948851 / 158495443 / -11.7% 0.79 / 0.79 / -0.5%
random-access/feature-vectors-s3/uniform/parquet-cached 489215496 / 516533665 / -5.3% 600070220 / 681955199 / -12.0% 0.82 / 0.76 / +7.6%
random-access/feature-vectors-s3/uniform/parquet-reopen 548953234 / 631273441 / -13.0% 615485373 / 769095785 / -20.0% 0.89 / 0.82 / +8.7%
random-access/nested-lists-s3/correlated/parquet-cached 38556822 / 43837904 / -12.0% 33491481 / 39812027 / -15.9% 1.15 / 1.10 / +4.6%
random-access/nested-lists-s3/correlated/parquet-reopen 109841448 / 118138770 / -7.0% 138092548 / 167689305 / -17.6% 0.80 / 0.70 / +12.9%
random-access/nested-lists-s3/uniform/parquet-cached 164666030 / 175587856 / -6.2% 164532124 / 183258349 / -10.2% 1.00 / 0.96 / +4.5%
random-access/nested-lists-s3/uniform/parquet-reopen 237915334 / 249946515 / -4.8% 294560581 / 247315316 / +19.1% 0.81 / 1.01 / -20.1%
random-access/nested-structs-s3/correlated/parquet-cached 45777556 / 52352534 / -12.6% 46551948 / 51950813 / -10.4% 0.98 / 1.01 / -2.4%
random-access/nested-structs-s3/correlated/parquet-reopen 122211313 / 129983576 / -6.0% 167592521 / 180885988 / -7.3% 0.73 / 0.72 / +1.5%
random-access/nested-structs-s3/uniform/parquet-cached 425803272 / 444109588 / -4.1% 425353156 / 447577345 / -5.0% 1.00 / 0.99 / +0.9%
random-access/nested-structs-s3/uniform/parquet-reopen 498654604 / 509706559 / -2.2% 512460314 / 522666193 / -2.0% 0.97 / 0.98 / -0.2%
random-access/taxi-s3/correlated/parquet-cached 107161788 / 198976736 / -46.1% 🟢 107104053 / 198943502 / -46.2% 🟢 1.00 / 1.00 / +0.0%
random-access/taxi-s3/correlated/parquet-reopen 179825396 / 258167635 / -30.3% 🟢 175545836 / 266696032 / -34.2% 🟢 1.02 / 0.97 / +5.8%
random-access/taxi-s3/parquet-cached 55158279 / 196194480 / -71.9% 🟢 54617633 / 196795959 / -72.2% 🟢 1.01 / 1.00 / +1.3%
random-access/taxi-s3/parquet-reopen 126126151 / 264565972 / -52.3% 🟢 206086628 / 275436910 / -25.2% 0.61 / 0.96 / -36.3%
random-access/taxi-s3/uniform/parquet-cached 194032234 / 217304517 / -10.7% 193412872 / 226730103 / -14.7% 1.00 / 0.96 / +4.7%
random-access/taxi-s3/uniform/parquet-reopen 267968772 / 266227110 / +0.7% 252265373 / 284041462 / -11.2% 1.06 / 0.94 / +13.3%
random-access / lance / ns (0.893x ➖, 1↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors-s3/correlated/lance-cached 33142196 / 35568476 / -6.8% 32898928 / 34730977 / -5.3% 1.01 / 1.02 / -1.6%
random-access/feature-vectors-s3/correlated/lance-reopen 191134517 / 194506280 / -1.7% 196960406 / 232942118 / -15.4% 0.97 / 0.83 / +16.2%
random-access/feature-vectors-s3/uniform/lance-cached 66780636 / 71970787 / -7.2% 62509589 / 94604659 / -33.9% 🟢 1.07 / 0.76 / +40.4%
random-access/feature-vectors-s3/uniform/lance-reopen 223426233 / 258038664 / -13.4% 234291772 / 273747878 / -14.4% 0.95 / 0.94 / +1.2%
random-access/nested-lists-s3/correlated/lance-cached 30424831 / 31667626 / -3.9% 28833826 / 32548295 / -11.4% 1.06 / 0.97 / +8.5%
random-access/nested-lists-s3/correlated/lance-reopen 209515490 / 218399033 / -4.1% 235962517 / 297513217 / -20.7% 0.89 / 0.73 / +21.0%
random-access/nested-lists-s3/uniform/lance-cached 67059144 / 109082207 / -38.5% 🟢 76597764 / 155923133 / -50.9% 🟢 0.88 / 0.70 / +25.1%
random-access/nested-lists-s3/uniform/lance-reopen 253358255 / 286944695 / -11.7% 251912554 / 283256135 / -11.1% 1.01 / 1.01 / -0.7%
random-access/nested-structs-s3/correlated/lance-cached 33764776 / 41512838 / -18.7% 31254051 / 32826741 / -4.8% 1.08 / 1.26 / -14.6%
random-access/nested-structs-s3/correlated/lance-reopen 229438051 / 228380315 / +0.5% 226483935 / 350209474 / -35.3% 🟢 1.01 / 0.65 / +55.3%
random-access/nested-structs-s3/uniform/lance-cached 92056904 / 108455510 / -15.1% 86690703 / 92746551 / -6.5% 1.06 / 1.17 / -9.2%
random-access/nested-structs-s3/uniform/lance-reopen 288465801 / 300530280 / -4.0% 301641884 / 324206427 / -7.0% 0.96 / 0.93 / +3.2%
random-access/taxi-s3/correlated/lance-cached 55519825 / 59681612 / -7.0% 239090429 / 66954868 / +257.1% 🔴 0.23 / 0.89 / -73.9%
random-access/taxi-s3/correlated/lance-reopen 262285018 / 262062440 / +0.1% 246009461 / 297650901 / -17.3% 1.07 / 0.88 / +21.1%
random-access/taxi-s3/lance-cached 40622264 / 52393930 / -22.5% 39653074 / 45533044 / -12.9% 1.02 / 1.15 / -11.0%
random-access/taxi-s3/lance-reopen 231573834 / 263068649 / -12.0% 298288116 / 292944703 / +1.8% 0.78 / 0.90 / -13.5%
random-access/taxi-s3/uniform/lance-cached 213762164 / 219448609 / -2.6% 234541539 / 216387464 / +8.4% 0.91 / 1.01 / -10.1%
random-access/taxi-s3/uniform/lance-reopen 422103765 / 491948180 / -14.2% 398861809 / 500514816 / -20.3% 1.06 / 0.98 / +7.7%

The TPC-H and TPC-DS generators write Utf8View columns. Since the
concatenated TPC-H files keep their Arrow schema, compress-bench read
them as Utf8View. The Arrow IPC writer writes every buffer that a view
array refers to, and batches from one Parquet reader share buffers, so
TPC-H's Arrow IPC files grew by up to 20 times. TPC-DS's new Arrow IPC
sizes had the same problem.

compress-bench's Arrow input and the size-only Arrow IPC copies now
read Utf8View as Utf8 and BinaryView as Binary, as develop's input was.

Signed-off-by: Will Manning <will@willmanning.io>
@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Random Access 📖

Commits: PR aff33cb1 vs base 967bcb67
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +88.4%
Engines: random-access No clear signal (+88.4%, environment too noisy confidence)
Vortex (geomean): hot 0.996x ➖ · cold 0.951x ➖
Parquet (geomean): hot 0.529x ✅ · cold 0.617x ✅
Shifts: Parquet (control) -47.1% · Median polish -14.4%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Query tables: One row per query and one column per target. Each cell reads develop → PR (PR ÷ develop), so (0.80) is 20% faster than develop. A change that clears this suite's threshold is bold and marked 🔴 slower or 🟢 faster; anything smaller is noise here and is left plain. The geomean row compares each target's geomean over its queries with develop's. Hot runs are shown first and cold runs are folded below them.
  • Other tables: Each cell reads PR / base / %diff, marked the same way.

vortex / arrow-ipc / ns (0.961x ➖, 5↑ 2↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors/correlated/arrow-ipc-cached 2003449 / 3273515 / -38.8% 🟢 1993217 / 3220912 / -38.1% 🟢 1.01 / 1.02 / -1.1%
random-access/feature-vectors/correlated/arrow-ipc-reopen 3363768 / 2161094 / +55.7% 🔴 11902648 / 11953507 / -0.4% 0.28 / 0.18 / +56.3%
random-access/feature-vectors/uniform/arrow-ipc-cached 53317120 / 35926477 / +48.4% 🔴 52575567 / 35810339 / +46.8% 🔴 1.01 / 1.00 / +1.1%
random-access/feature-vectors/uniform/arrow-ipc-reopen 54242519 / 53606194 / +1.2% 169212810 / 171362182 / -1.3% 0.32 / 0.31 / +2.5%
random-access/nested-lists/correlated/arrow-ipc-cached 61283 / 71779 / -14.6% 🟢 59391 / 71926 / -17.4% 🟢 1.03 / 1.00 / +3.4%
random-access/nested-lists/correlated/arrow-ipc-reopen 118451 / 116492 / +1.7% 3543501 / 3275437 / +8.2% 0.03 / 0.04 / -6.0%
random-access/nested-lists/uniform/arrow-ipc-cached 1193624 / 1332870 / -10.4% 🟢 1189423 / 1355154 / -12.2% 🟢 1.00 / 0.98 / +2.0%
random-access/nested-lists/uniform/arrow-ipc-reopen 1256361 / 1334482 / -5.9% 25910360 / 26472846 / -2.1% 0.05 / 0.05 / -3.8%
random-access/nested-structs/correlated/arrow-ipc-cached 49439 / 54508 / -9.3% 52211 / 55019 / -5.1% 0.95 / 0.99 / -4.4%
random-access/nested-structs/correlated/arrow-ipc-reopen 113094 / 107492 / +5.2% 3402484 / 2941524 / +15.7% 🔴 0.03 / 0.04 / -9.0%
random-access/nested-structs/uniform/arrow-ipc-cached 855218 / 992187 / -13.8% 🟢 855667 / 977772 / -12.5% 🟢 1.00 / 1.01 / -1.5%
random-access/nested-structs/uniform/arrow-ipc-reopen 950604 / 1070827 / -11.2% 🟢 20867106 / 20458605 / +2.0% 0.05 / 0.05 / -13.0%
random-access/taxi/arrow-ipc-cached 68322 / 73689 / -7.3% 68056 / 79807 / -14.7% 🟢 1.00 / 0.92 / +8.7%
random-access/taxi/arrow-ipc-reopen 161646.5 / 174516.5 / -7.4% 3331954 / 3093071 / +7.7% 0.05 / 0.06 / -14.0%
random-access/taxi/correlated/arrow-ipc-cached 120734 / 127495 / -5.3% 116478 / 123898 / -6.0% 1.04 / 1.03 / +0.7%
random-access/taxi/correlated/arrow-ipc-reopen 215925 / 233416 / -7.5% 1642857 / 1613325 / +1.8% 0.13 / 0.14 / -9.2%
random-access/taxi/uniform/arrow-ipc-cached 2807334 / 3058148 / -8.2% 2853425 / 3111117 / -8.3% 0.98 / 0.98 / +0.1%
random-access/taxi/uniform/arrow-ipc-reopen 2906363 / 3131582 / -7.2% 34721927 / 34487624 / +0.7% 0.08 / 0.09 / -7.8%
random-access / vortex-file-compressed / ns (0.996x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors/correlated/vortex-cached 109842.5 / 115821 / -5.2% 106372 / 118618 / -10.3% 🟢 1.03 / 0.98 / +5.8%
random-access/feature-vectors/correlated/vortex-reopen 1947284 / 1943215 / +0.2% 4362995 / 4352056 / +0.3% 0.45 / 0.45 / -0.0%
random-access/feature-vectors/uniform/vortex-cached 970479.5 / 985149 / -1.5% 957554 / 947367 / +1.1% 1.01 / 1.04 / -2.5%
random-access/feature-vectors/uniform/vortex-reopen 3041648 / 2936944 / +3.6% 11390941 / 12460115 / -8.6% 0.27 / 0.24 / +13.3%
random-access/nested-lists/correlated/vortex-cached 102080 / 106006 / -3.7% 97823 / 102613 / -4.7% 1.04 / 1.03 / +1.0%
random-access/nested-lists/correlated/vortex-reopen 396980 / 372172 / +6.7% 2331196 / 2586444 / -9.9% 0.17 / 0.14 / +18.3%
random-access/nested-lists/uniform/vortex-cached 418435 / 419502 / -0.3% 281325 / 427937 / -34.3% 🟢 1.49 / 0.98 / +51.7%
random-access/nested-lists/uniform/vortex-reopen 1075980 / 1029342 / +4.5% 8101392 / 8939559 / -9.4% 0.13 / 0.12 / +15.3%
random-access/nested-structs/correlated/vortex-cached 124504 / 129317 / -3.7% 125942 / 128770 / -2.2% 0.99 / 1.00 / -1.6%
random-access/nested-structs/correlated/vortex-reopen 461523 / 458757 / +0.6% 3901995 / 3909811 / -0.2% 0.12 / 0.12 / +0.8%
random-access/nested-structs/uniform/vortex-cached 245106 / 253436 / -3.3% 251670 / 259814 / -3.1% 0.97 / 0.98 / -0.2%
random-access/nested-structs/uniform/vortex-reopen 684547 / 662495 / +3.3% 5048237 / 4943829 / +2.1% 0.14 / 0.13 / +1.2%
random-access/taxi/correlated/vortex-cached 319950.5 / 329905 / -3.0% 329927 / 331372 / -0.4% 0.97 / 1.00 / -2.6%
random-access/taxi/correlated/vortex-reopen 1237097 / 1264977 / -2.2% 2737317 / 2575716 / +6.3% 0.45 / 0.49 / -8.0%
random-access/taxi/uniform/vortex-cached 849231 / 851733 / -0.3% 876452 / 870590 / +0.7% 0.97 / 0.98 / -1.0%
random-access/taxi/uniform/vortex-reopen 3042886 / 3025162 / +0.6% 5256138 / 6120589 / -14.1% 🟢 0.58 / 0.49 / +17.1%
random-access/taxi/vortex-cached 263576 / 268515 / -1.8% 265901 / 264017 / +0.7% 0.99 / 1.02 / -2.5%
random-access/taxi/vortex-reopen 1051578 / 1060266 / -0.8% 3797110 / 3541576 / +7.2% 0.28 / 0.30 / -7.5%
random-access / parquet / ns (0.529x ✅, 12↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors/correlated/parquet-cached 7146686 / 10234158 / -30.2% 🟢 7136745 / 10248345 / -30.4% 🟢 1.00 / 1.00 / +0.3%
random-access/feature-vectors/correlated/parquet-reopen 7429002 / 10462616 / -29.0% 🟢 13411073 / 17660684 / -24.1% 🟢 0.55 / 0.59 / -6.5%
random-access/feature-vectors/uniform/parquet-cached 130543560 / 132199860 / -1.3% 130530928 / 132113084 / -1.2% 1.00 / 1.00 / -0.1%
random-access/feature-vectors/uniform/parquet-reopen 135584992 / 136993269 / -1.0% 199424661 / 198615662 / +0.4% 0.68 / 0.69 / -1.4%
random-access/nested-lists/correlated/parquet-cached 3411690 / 4632754 / -26.4% 🟢 3405321 / 4667436 / -27.0% 🟢 1.00 / 0.99 / +0.9%
random-access/nested-lists/correlated/parquet-reopen 3530452 / 4766642 / -25.9% 🟢 9491584 / 11375907 / -16.6% 🟢 0.37 / 0.42 / -11.2%
random-access/nested-lists/uniform/parquet-cached 39463378 / 40364146 / -2.2% 39691720 / 40874723 / -2.9% 0.99 / 0.99 / +0.7%
random-access/nested-lists/uniform/parquet-reopen 40844287 / 39211014 / +4.2% 64382980 / 65256019 / -1.3% 0.63 / 0.60 / +5.6%
random-access/nested-structs/correlated/parquet-cached 4445597 / 9066642 / -51.0% 🟢 4448027 / 9053517 / -50.9% 🟢 1.00 / 1.00 / -0.2%
random-access/nested-structs/correlated/parquet-reopen 4560262 / 9310630 / -51.0% 🟢 15698730 / 22390287 / -29.9% 🟢 0.29 / 0.42 / -30.1%
random-access/nested-structs/uniform/parquet-cached 34260612 / 36874241 / -7.1% 34333178 / 36861345 / -6.9% 1.00 / 1.00 / -0.2%
random-access/nested-structs/uniform/parquet-reopen 34355863 / 36839373 / -6.7% 44188115 / 44996346 / -1.8% 0.78 / 0.82 / -5.0%
random-access/taxi/correlated/parquet-cached 3751632 / 23770218 / -84.2% 🟢 3739860 / 23685600 / -84.2% 🟢 1.00 / 1.00 / -0.0%
random-access/taxi/correlated/parquet-reopen 4054297 / 23854513 / -83.0% 🟢 11584905 / 27563541 / -58.0% 🟢 0.35 / 0.87 / -59.6%
random-access/taxi/parquet-cached 1692559 / 16870886 / -90.0% 🟢 1682357 / 16911972 / -90.1% 🟢 1.01 / 1.00 / +0.9%
random-access/taxi/parquet-reopen 1963326 / 16601707 / -88.2% 🟢 9432558 / 23067626 / -59.1% 🟢 0.21 / 0.72 / -71.1%
random-access/taxi/uniform/parquet-cached 28597909 / 37711200 / -24.2% 🟢 28622740 / 37596016 / -23.9% 🟢 1.00 / 1.00 / -0.4%
random-access/taxi/uniform/parquet-reopen 28978886 / 37667462 / -23.1% 🟢 34563154 / 40565987 / -14.8% 🟢 0.84 / 0.93 / -9.7%
random-access / lance / ns (1.005x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors/correlated/lance-cached 156003 / 157111 / -0.7% 160564 / 161887 / -0.8% 0.97 / 0.97 / +0.1%
random-access/feature-vectors/correlated/lance-reopen 988968.5 / 974548 / +1.5% 6263107 / 6716665 / -6.8% 0.16 / 0.15 / +8.8%
random-access/feature-vectors/uniform/lance-cached 988658 / 1003396 / -1.5% 968261 / 1014761 / -4.6% 1.02 / 0.99 / +3.3%
random-access/feature-vectors/uniform/lance-reopen 1826197 / 1823313 / +0.2% 9286411 / 9608430 / -3.4% 0.20 / 0.19 / +3.6%
random-access/nested-lists/correlated/lance-cached 113459 / 119378 / -5.0% 110122 / 116578 / -5.5% 1.03 / 1.02 / +0.6%
random-access/nested-lists/correlated/lance-reopen 451318 / 440501 / +2.5% 5589759 / 5899576 / -5.3% 0.08 / 0.07 / +8.1%
random-access/nested-lists/uniform/lance-cached 838655 / 815089 / +2.9% 851658 / 837001 / +1.8% 0.98 / 0.97 / +1.1%
random-access/nested-lists/uniform/lance-reopen 1355349 / 1349515 / +0.4% 9075220 / 8979853 / +1.1% 0.15 / 0.15 / -0.6%
random-access/nested-structs/correlated/lance-cached 216051 / 216413 / -0.2% 219018 / 213329 / +2.7% 0.99 / 1.01 / -2.8%
random-access/nested-structs/correlated/lance-reopen 511298 / 491117.5 / +4.1% 5313554 / 5615788 / -5.4% 0.10 / 0.09 / +10.0%
random-access/nested-structs/uniform/lance-cached 2184722 / 2127252 / +2.7% 2182416 / 2129431 / +2.5% 1.00 / 1.00 / +0.2%
random-access/nested-structs/uniform/lance-reopen 2458902 / 2508544 / -2.0% 14621481 / 15040855 / -2.8% 0.17 / 0.17 / +0.8%
random-access/taxi/correlated/lance-cached 633889 / 629578 / +0.7% 622511 / 674008 / -7.6% 1.02 / 0.93 / +9.0%
random-access/taxi/correlated/lance-reopen 1646668 / 1635503 / +0.7% 2763580 / 2810847 / -1.7% 0.60 / 0.58 / +2.4%
random-access/taxi/lance-cached 434765 / 428563 / +1.4% 438483 / 426169 / +2.9% 0.99 / 1.01 / -1.4%
random-access/taxi/lance-reopen 1255007 / 1255248 / -0.0% 7167848 / 7469230 / -4.0% 0.18 / 0.17 / +4.2%
random-access/taxi/uniform/lance-cached 6818492 / 6686984 / +2.0% 6797312 / 6747654 / +0.7% 1.00 / 0.99 / +1.2%
random-access/taxi/uniform/lance-reopen 7888784 / 7959598 / -0.9% 25361424 / 25587285 / -0.9% 0.31 / 0.31 / -0.0%
random-access / litdata / ns (1.002x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
random-access/feature-vectors/correlated/litdata-cached 739069.5 / 735822 / +0.4% 765615 / 729576 / +4.9% 0.97 / 1.01 / -4.3%
random-access/feature-vectors/correlated/litdata-reopen 3046724 / 3083236 / -1.2% 3763749 / 3758527 / +0.1% 0.81 / 0.82 / -1.3%
random-access/feature-vectors/uniform/litdata-cached 1078637 / 1117548 / -3.5% 1104288 / 1123091 / -1.7% 0.98 / 1.00 / -1.8%
random-access/feature-vectors/uniform/litdata-reopen 16044748 / 16184094 / -0.9% 31703993 / 31764750 / -0.2% 0.51 / 0.51 / -0.7%
random-access/nested-lists/correlated/litdata-cached 572114 / 575023 / -0.5% 579969 / 580494 / -0.1% 0.99 / 0.99 / -0.4%
random-access/nested-lists/correlated/litdata-reopen 17312452 / 17141494 / +1.0% 24084518 / 24037747 / +0.2% 0.72 / 0.71 / +0.8%
random-access/nested-lists/uniform/litdata-cached 614433 / 607214 / +1.2% 616428 / 613062 / +0.5% 1.00 / 0.99 / +0.6%
random-access/nested-lists/uniform/litdata-reopen 17065153 / 16783131 / +1.7% 24251898 / 24219852 / +0.1% 0.70 / 0.69 / +1.5%
random-access/nested-structs/correlated/litdata-cached 661744 / 667857 / -0.9% 665879 / 669279 / -0.5% 0.99 / 1.00 / -0.4%
random-access/nested-structs/correlated/litdata-reopen 18968560 / 18450878 / +2.8% 21817585 / 21963366 / -0.7% 0.87 / 0.84 / +3.5%
random-access/nested-structs/uniform/litdata-cached 725699 / 728315 / -0.4% 727045 / 726380 / +0.1% 1.00 / 1.00 / -0.5%
random-access/nested-structs/uniform/litdata-reopen 16396262 / 16217754 / +1.1% 25009763 / 25210337 / -0.8% 0.66 / 0.64 / +1.9%
random-access/taxi/correlated/litdata-cached 1182418 / 1171658 / +0.9% 1167221 / 1175215 / -0.7% 1.01 / 1.00 / +1.6%
random-access/taxi/correlated/litdata-reopen 20064084 / 20051447 / +0.1% 28982223 / 29622110 / -2.2% 0.69 / 0.68 / +2.3%
random-access/taxi/litdata-cached 78762 / 81523 / -3.4% 79024 / 81923 / -3.5% 1.00 / 1.00 / +0.2%
random-access/taxi/litdata-reopen 11342155 / 10883071 / +4.2% 14503453 / 14436619 / +0.5% 0.78 / 0.75 / +3.7%
random-access/taxi/uniform/litdata-cached 1310433 / 1313506 / -0.2% 1310681 / 1303373 / +0.6% 1.00 / 1.01 / -0.8%
random-access/taxi/uniform/litdata-reopen 67660115 / 66440388 / +1.8% 82533354 / 82107545 / +0.5% 0.82 / 0.81 / +1.3%

@lwwmanning lwwmanning added the action/bench-compress Run only the compression benchmark on this PR label Nov 21, 2026
@github-actions github-actions Bot removed the action/bench-compress Run only the compression benchmark on this PR label Nov 21, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9997597ad1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread vortex-bench/src/compress/mod.rs Resolved
arrow_uncompressed_size still read view types, and Arrow's memory size
counts a shared buffer once in every batch that refers to it. It now
reads through plain_arrow_reader, like the compression input, so each
ratio divides by the size of the data the formats compressed.

Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning lwwmanning added the action/bench-compress Run only the compression benchmark on this PR label Nov 21, 2026
@github-actions github-actions Bot removed the action/bench-compress Run only the compression benchmark on this PR label Nov 21, 2026
@lwwmanning lwwmanning changed the title Benchmarks write all Parquet with one tuned config, and record Snappy Parquet sizes Optimize shared Parquet config, and record Snappy Parquet sizes Nov 21, 2026
@lwwmanning
lwwmanning enabled auto-merge (squash) November 21, 2026 21:49
@lwwmanning

Copy link
Copy Markdown
Contributor Author

@codex please review
@claude please review

@github-actions

Copy link
Copy Markdown
Contributor

Claude review automation is disabled for pull requests that modify .github/ files.

Why:

  • workflow and action files are part of the automation policy
  • this review workflow refuses to evaluate automation changes from the same PR

Ask a human reviewer to inspect workflow changes directly.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 3b15172cfe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@lwwmanning
lwwmanning merged commit 81ed6fa into develop Nov 21, 2026
120 checks passed
@lwwmanning
lwwmanning deleted the wm/file-size-parquet-codecs branch November 21, 2026 22:09
lwwmanning added a commit to vortex-data/vortex.dev that referenced this pull request Nov 21, 2026
* Merge the size groups into one File Sizes group on bench

The Compression Size and SQL File Size groups showed the same metric,
file size per format, from two suites. One File Sizes group now charts
the compress-bench datasets and three SQL scopes: ClickBench, TPC-H at
SF10, and TPC-DS at SF1. Old Compression Size links redirect to it, and
the homepage card reads the group by its new or old name.

The summary compares each format with zstd Parquet on the compress-bench
datasets and on TPC-DS. Each format uses its newest commit that has zstd
Parquet. The wide-table and sparse-wide families count as one dataset
each. A format measured on only some datasets says how many, has no
total, and ranks after the complete formats. Parquet codec variants,
such as parquet-snappy, draw in Parquet's color with a dashed line.

Signed-off-by: Will Manning <will@willmanning.io>

* Keep the SQL File Sizes group beside File Sizes on bench

File Sizes moves to the Compression Size group key, so old Compression
Size links open it. SQL File Sizes comes back on its own key: it charts
every SQL benchmark and scale factor and ranks formats against Vortex,
as before. Each group's info tooltip says how the two differ.

Signed-off-by: Will Manning <will@willmanning.io>

* Note the TPC-DS size-only formats in the bench contract

Signed-off-by: Will Manning <will@willmanning.io>

* Rank a query series that skips a query below complete ones on bench

A series with no time for a query now ranks below every series that ran
all of them. The missing-series penalty alone scores a skipped query at
2x the best time, so a series could rank first by skipping the slowest
query. Lance skips TPC-DS q72, which it cannot run.

Snappy Parquet joins TPC-DS's queries, so a codec variant is now dotted,
and dash-dotted on DuckDB, to keep it apart from DuckDB's dashed Parquet.

Signed-off-by: Will Manning <will@willmanning.io>

* Block Vercel's preview Toolbar in Lighthouse runs

Vercel adds its Toolbar to preview deployments, and the proxy's
x-vercel-skip-toolbar header does not stop it. The Toolbar loads from
vercel.live and Pusher, and its third-party cookies and issues cut Best
Practices to 0.78 on every audited page. The site loads neither host,
so Lighthouse now blocks both.

Signed-off-by: Will Manning <will@willmanning.io>

* Explain bench's two Parquet variants on the methodology page

Parquet is zstd level 3, the size and throughput baseline. Parquet
(Snappy) is Parquet as DuckDB writes it by default, which TPC-DS SF1
queries and the compression benchmark sizes. ClickBench and FineWeb query
their published files as Parquet, and ClickBench's are Snappy. The page
also says that a query summary ranks a series that missed a query below
the complete ones, as Lance on TPC-DS q72.

The contract now names file sizes by format directory, not by codec.

Signed-off-by: Will Manning <will@willmanning.io>

* Name DuckDB 2.0 as the writer of Parquet (Snappy) on bench

vortex-data/vortex#11720 now installs the DuckDB 2.0.0 CLI, the version
the queries run on.

Signed-off-by: Will Manning <will@willmanning.io>

* Explain how bench's Parquet is tuned on the methodology page

A new section gives each Parquet writer setting and the test behind it:
zstd level, dictionary size, page size and page index, Bloom filters, and
row groups. Parquet (Snappy) is now the same Parquet with Snappy in place
of zstd, rather than Parquet as DuckDB writes it, and TPC-DS at scale factor 1
no longer queries it.

Signed-off-by: Will Manning <will@willmanning.io>

* Measure a size family only when a format has all of it, and bump the cache

A format with one wide table or sparse-wide dataset counted the whole family
as measured, so it could rank in the complete tier against formats measured
on every member. A family now counts as measured only when the format has
every dataset of it that any format has.

The File Sizes group reuses the Compression Size slug with another name,
other charts, and another summary, so the group payload cache moves to v16.

Signed-off-by: Will Manning <will@willmanning.io>

* Rename File Sizes to File Sizes & Compression Ratios, and leave DuckDB's file out of its ranking

TPC-DS records DuckDB's database file, which is a query engine's storage
rather than a file format, so the summary no longer ranks it. The group
still charts it. The contract and methodology also say that TPC-DS SF1
records Snappy Parquet's size.

Signed-off-by: Will Manning <will@willmanning.io>

* Rename the size group to Compression & File Size

The new name keeps more continuity with the Compression Size group it
replaces.

Signed-off-by: Will Manning <will@willmanning.io>

* Keep the renamed size groups' anchors, and describe the size ranking

Old #compression-size and #sql-file-size links land on the renamed
groups. The group tooltip says which scopes it charts and which it
ranks. The methodology page explains the Compression & File Size
summary's totals and how it counts the wide-table suites.

Signed-off-by: Will Manning <will@willmanning.io>

* Label zstd Parquet as parquet-zstd in the Compression & File Size group

The group charts Snappy Parquet beside zstd Parquet, so its legends,
tooltips, toolbar chips, and summary card name the zstd files
parquet-zstd, and the card's baseline header reads "vs parquet-zstd".
ClickBench's chart keeps parquet, because it charts the published
Snappy files. The series keys stay parquet, so filters, colors, and
URLs are unchanged. A compression chart's permalink page uses the
same label; a file-size permalink page does not, because the SQL File
Sizes group shares those charts. The group payload cache moves to v17.

Signed-off-by: Will Manning <will@willmanning.io>

* Order the size summary by its Parquet score, and document its Arrow ratio

The Compression & File Size card ordered formats by their ratio versus
Arrow once every format had one. TPC-DS has no Arrow size, so that order
left out the one SQL scope the summary ranks. A size summary now orders
by the Arrow ratio only when it covers every dataset each format is
ranked on, and by the Parquet ratio otherwise. The String Encoding Size
card, whose Arrow ratios cover every dataset, keeps its order.

The methodology now says what the vs Arrow column measures, that it
covers the compression datasets alone, and how the card is ordered. It
also says that the compression benchmark writes 1,048,576-row row
groups. The group payload cache moves to v18.

Signed-off-by: Will Manning <will@willmanning.io>

* Move the methodology link and build commit above the first group

On the landing page, the line with the methodology link and the build
commit was at the bottom, after every group. It now opens the page's
main content, left-aligned with the group cards and small, and the
methodology link is underlined so it reads as a link. The chart and
methodology pages keep the line at the bottom.

Signed-off-by: Will Manning <will@willmanning.io>

* Name ClickBench on the group's renamed parquet chip

In the Compression & File Size group, the toolbar's parquet-zstd chip
also toggles ClickBench's parquet line, which is the published Snappy
files. A renamed chip's title now says so: "parquet-zstd, and parquet
on clickbench".

Signed-off-by: Will Manning <will@willmanning.io>

* Rank SQL Benchmark File Sizes against parquet, one score per benchmark

Rename the SQL File Sizes group to SQL Benchmark File Sizes and move it
after the query groups, since it holds the files those queries read.

Its summary now compares each format with parquet, bench's standard
Parquet, at each format's newest commit that also has parquet. It ranks
only JOB, Polar Signals, statpopgen, TPC-DS, TPC-H, and nested TPC-H,
whose Parquet bench writes with its own settings; ClickBench, FineWeb,
Appian, sorted ClickBench, and the Vortex queries keep their charts.
Each benchmark counts once, as the geometric mean of its scale factors,
so TPC-H's three scale factors no longer outweigh the rest. DuckDB's
database file is left out, as in the Compression & File Size summary.

The card drops the Total and vs Arrow columns, which this summary never
filled, and says "of N benchmarks" in its coverage note. With no
baseline, the landing page's headline returns for this group.

Signed-off-by: Will Manning <will@willmanning.io>

* Show a partial file size score as not measured on the homepage

The homepage's file size panel dropped bench's coverage for that one
panel, so a format without TPC-DS sizes, such as Lance today, showed a
geomean over fewer datasets beside formats measured on all of them.
Lance is about 2x Parquet without TPC-DS and 4.3x on it, so the panel
flattered it. Pass the coverage through, as the other panels do;
results.js shows such a row as "Not measured".

Signed-off-by: Will Manning <will@willmanning.io>

* Count a SQL benchmark as measured when a format has any of its scale factors

The SQL Benchmark File Sizes card counted a benchmark only when a format
had every scale factor of it, so Lance, with TPC-DS at SF1 alone, showed
a 2.12x score beside "measured on 0 of 6 benchmarks". It now counts one
scale factor as enough, so Lance shows 1 of 6 and still ranks after the
formats measured on every benchmark. The Compression & File Size summary
keeps the stricter rule for its families.

Signed-off-by: Will Manning <will@willmanning.io>

* Rank DuckDB's file in the SQL size summary when it has every benchmark

DuckDB's database file is the file DuckDB's queries read, so the SQL
Benchmark File Sizes summary keeps it when it has a size on every
benchmark the summary covers. With fewer, it stays out, as in the
Compression & File Size summary.

Signed-off-by: Will Manning <will@willmanning.io>

* Mark a size headline partial when Vortex lacks some datasets

The collapsed group's size headline read the Vortex ratio alone, so a
mean over fewer datasets than the summary covers showed as a plain
"smaller than Parquet" claim while the card marked it partial. The
headline now adds "(partial)" when Vortex's measured count is below
the total.

Signed-off-by: Will Manning <will@willmanning.io>

* Show Parquet's codecs as parquet (zstd) and parquet (snappy)

The Compression & File Size group labeled its two Parquet series
parquet-zstd and parquet-snappy. They now read parquet (zstd) and
parquet (snappy), in the legends, tooltips, chips, and the summary's
"vs" header. ClickBench's chart still shows plain parquet for its
published Snappy files.

A renamed chip's title now lists, for each renamed format, the charts
that keep that format's real name, so the parquet (snappy) chip does
not claim ClickBench charts parquet-snappy.

Signed-off-by: Will Manning <will@willmanning.io>

* Name zstd Parquet in size headlines, and weigh llms.txt's SQL sizes by benchmark

In the Compression & File Size group, the collapsed headline and
llms.txt said "Parquet" although the summary divides by zstd Parquet
and the group also charts Snappy Parquet. They now say "Parquet (zstd)".

llms.txt described the SQL Benchmark File Sizes score as a geometric
mean over datasets, and its range as across datasets. The score now
says each benchmark counts once, as the geometric mean of its scale
factors, and the range is across benchmarks and scale factors.

Signed-off-by: Will Manning <will@willmanning.io>

---------

Signed-off-by: Will Manning <will@willmanning.io>
lwwmanning added a commit that referenced this pull request Nov 22, 2026
This changes how `vortex-bench` builds the SpatialBench `trip`,
`building`, and `customer` tables, and which SpatialBench crates it
depends on. The `zone` table still comes from the external
`spatialbench-cli`.

Before, SpatialBench used `spatialbench` and `spatialbench-arrow` 0.2.0,
the latest release. `spatialbench-arrow` builds on Arrow 56, and the
workspace's Parquet 59 writer can't take Arrow 56 batches. So `wkb.rs`
wrote them through a second Parquet 56 copy, aliased as
`spatialbench-parquet`. That kept 13 Arrow 56 and Parquet 56 crates in
`Cargo.lock`, and Renovate's PRs to bump the alias could never build
(#11320 is the latest).

After, the generators still come from `spatialbench` 0.2.0 on crates.io,
which does not depend on Arrow. The Arrow conversion for the three
tables is now in `datagen/batches.rs`, about 250 lines adapted from
`spatialbench-arrow` 0.2.0 and built on the workspace Arrow. A
SpatialBench release can no longer hold back an Arrow, Parquet, or
DataFusion upgrade. Upstream took ten months to move from Arrow 56 to
58. `wkb.rs` now writes with the workspace `parquet` and the shared
`parquet_writer_properties()` from #11720, and `Cargo.lock` has 20 fewer
packages.

The cost is a copy that must keep upstream's schemas and values. Compare
it with upstream when `spatialbench` changes version. Its header names
the source tag. The remaining Parquet 58.4.1 copy comes from
`lance-bench`, which needs it for lance 14.

At scale factors 0.1 and 1, the new code writes Parquet files that are
byte-identical to those from `spatialbench-arrow` on the same Arrow and
Parquet. Against the old Arrow 56 build, every table has the same
schema, row count, values, and GeoParquet `geo` metadata. File sizes
differ by less than 1%, which fits the writer changes between Parquet 56
and 59.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Will Manning <will@willmanning.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants