Repository navigation
Optimize shared Parquet config, and record Snappy Parquet sizes - #11720
Conversation
capture-file-sizes.py reads each Parquet file's footer and names its format by codec: zstd stays `parquet`, the codec the benchmarks write, and other codecs record as `parquet-<codec>`, as `vortex` and `vortex-compact` name Vortex's. ClickBench and FineWeb measure downloaded files, and the statpopgen and vortex suites write without zstd, so their sizes were recorded as zstd Parquet. Signed-off-by: Will Manning <will@willmanning.io>
compress-bench writes each dataset once more as Snappy Parquet, untimed, and records its size as `parquet-snappy` beside `parquet` (zstd level 3). Only sizes: Snappy adds no compression or decompression timings. Signed-off-by: Will Manning <will@willmanning.io>
compress-bench now writes each dataset once more with the compact Vortex strategy and records its size as vortex-compact, as the SQL benchmarks name it. The write is not timed and does not run in GPU mode. bench can then compare Vortex and compact Vortex on the compression datasets, as it does on the SQL benchmarks' files. Signed-off-by: Will Manning <will@willmanning.io>
Develop's TPC-DS SF1 run now writes three formats that no query reads, after its queries, so their file sizes are recorded with the others: - `data-gen --size-formats arrow-ipc,parquet-snappy` writes uncompressed Arrow IPC and Snappy Parquet from the base Parquet files. The Snappy files keep the base files' other writer settings and row groups. - `lance-bench --data-only` writes the Lance data and exits. - `vx-bench write-size-formats` runs both, from a new `size_formats` list on a matrix coverage. CI runs it before "Capture file sizes". `capture-file-sizes.py` records the `parquet-snappy` directory, and labels its files by codec like the `parquet` directory's. At SF1 the writes take about 8 seconds and 3.8 GB of disk. Signed-off-by: Will Manning <will@willmanning.io>
Develop's TPC-DS SF1 run now queries Snappy Parquet on DataFusion and DuckDB, and Lance on DataFusion. data-gen writes Snappy Parquet as the parquet-snappy format, from the base Parquet files with the same row groups. Arrow IPC stays a size-only format. Lance skips TPC-DS q72, which overflows its Utf8 offsets. bench ranks a series that misses a query below every series that ran them all. Signed-off-by: Will Manning <will@willmanning.io>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Signed-off-by: Will Manning <will@willmanning.io>
… vortex suite Every benchmark writes zstd level 3 Parquet, but three writers did not: statpopgen and the SpatialBench zone rewrite passed no writer settings, so parquet-rs wrote them uncompressed, and the vortex suite's DuckDB COPY named no codec, so DuckDB wrote Snappy. Their Parquet sizes and times measured a different codec from every other suite. All three now write zstd level 3. statpopgen's CI data cache also keys on the shared writer settings, so a change to them regenerates the cached files. Signed-off-by: Will Manning <will@willmanning.io>
`parquet-snappy` is now Parquet as DuckDB writes it: `COPY ... (FORMAT parquet, COMPRESSION snappy)` in the `duckdb` CLI, with DuckDB's row groups, dictionaries, statistics, and Bloom filters. data-gen writes the query files this way, and compress-bench records its Snappy size from the same writer. Every benchmark runner already installs the CLI. File sizes are named by directory again, not by codec. `parquet` is the suite's base Parquet: zstd level 3 where the benchmarks write it, and the dataset's own files where the suite prescribes them, as ClickBench's Snappy files are. This keeps ClickBench's `parquet` series continuous and keeps `parquet-snappy` for DuckDB's files. Signed-off-by: Will Manning <will@willmanning.io>
#11676 changed the integer cast's overflow error from "value exceeds target range" to "values exceed target range". The device batch test looked for the old wording, so CUDA tests fail on develop's code. Match the words both share. Signed-off-by: Will Manning <will@willmanning.io>
The queries run on DuckDB 2.0.0, which vortex-duckdb links, but the runners installed the 1.5.5 CLI, which writes parquet-snappy and the native duckdb database. Install the same version. DuckDB 2.0 ships the CLI as a tarball, so the action extracts it with tar. Signed-off-by: Will Manning <will@willmanning.io>
vortex-data/vortex#11720 now installs the DuckDB 2.0.0 CLI, the version the queries run on. Signed-off-by: Will Manning <will@willmanning.io>
2.0.1 is DuckDB 2.0's bugfix release. vortex-duckdb still links 2.0.0, so the CLI stays on the minor version the queries run on. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 147ab29be9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…zone files compress-bench wrote its size-only formats, DuckDB's Snappy Parquet and compact Vortex, between a format's timed compression and its timed decompression. Write them after every timed run of the dataset. vx-bench run dropped the OS caches before each query, but lance-bench wrote its Lance files on its first query, so that query read them from the page cache. The run now writes Lance's data before the first cache drop. SpatialBench's zone file was left as it was when it already had GeoParquet metadata, so a file tagged before the benchmarks wrote zstd stayed uncompressed. It is now rewritten unless it is also zstd. Signed-off-by: Will Manning <will@willmanning.io>
Writing them after the timed runs raised run_benchmark_for_dataset's cognitive complexity to 27, over clippy's limit of 25. Signed-off-by: Will Manning <will@willmanning.io>
Merging this PR will not alter performance2 warnings
Comparing Footnotes
|
Develop runs DataFusion and DuckDB on parquet-snappy beside zstd Parquet, so bench can compare the two Parquet variants on local disk and on S3. Signed-off-by: Will Manning <will@willmanning.io>
Lets this pull request's action/bench-sql run compare zstd and Snappy Parquet on NVMe and S3. Revert before merge. Signed-off-by: Will Manning <will@willmanning.io>
Polar Signals Profiling ResultsLatest Run
Previous Runs (56)
Powered by Polar Signals Cloud |
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e0e95b1de1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
compress-bench now records zstd Parquet's whole file, footer included, as it does for Vortex and DuckDB's Snappy Parquet. The Vortex query suite and statpopgen write their Parquet again when a cached file is not zstd. Signed-off-by: Will Manning <will@willmanning.io>
Benchmarks: TPC-DS SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 0.991x ➖ · 99 queries × 4 targets (0↑ 0↓)Each cell is develop
Cold runs (first iteration): geomean 0.990x ➖ · 99 queries × 4 targets (5↑ 3↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 0.892x ➖ · 22 queries × 6 targets (9↑ 0↓)Each cell is develop
Cold runs (first iteration): geomean 0.923x ➖ · 22 queries × 6 targets (7↑ 2↓)
|
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 0.899x 🟢 · 22 queries × 7 targets (39↑ 7↓)Each cell is develop
Cold runs (first iteration): geomean 0.949x ➖ · 22 queries × 7 targets (31↑ 32↓)
File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
Totals:
|
Benchmarks: PolarSignals Profiling 📖Commits: PR Hot runs: geomean 1.001x ➖ · 13 queries × 1 target (0↑ 0↓)Each cell is develop
Cold runs (first iteration): geomean 1.030x ➖ · 13 queries × 1 target (0↑ 1↓)
No file size changes detected. |
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 0.962x ➖ · 9 queries × 6 targets (2↑ 1↓)Each cell is develop
Cold runs (first iteration): geomean 0.910x ➖ · 9 queries × 6 targets (5↑ 2↓)
|
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 0.784x 🟢 · 22 queries × 7 targets (45↑ 2↓)Each cell is develop
Cold runs (first iteration): geomean 0.821x 🟢 · 22 queries × 7 targets (48↑ 1↓)
File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
Totals:
|
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 0.995x ➖ · 43 queries × 7 targets (4↑ 1↓)Each cell is develop
Cold runs (first iteration): geomean 1.014x ➖ · 43 queries × 7 targets (5↑ 17↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 1.001x ➖ · 99 queries × 7 targets (11↑ 10↓)Each cell is develop
Cold runs (first iteration): geomean 1.023x ➖ · 99 queries × 7 targets (6↑ 28↓)
No file size changes detected. |
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 1.002x ➖ · 11 queries × 6 targets (0↑ 1↓)Each cell is develop
Cold runs (first iteration): geomean 1.003x ➖ · 11 queries × 6 targets (0↑ 1↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
Hot runs: geomean 0.993x ➖ · 99 queries × 4 targets (2↑ 2↓)Each cell is develop
Cold runs (first iteration): geomean 0.979x ➖ · 99 queries × 4 targets (11↑ 3↓)
No file size changes detected. |
…nfig StructListOfInts wrote its Parquet input with parquet-rs's defaults and kept a cached file in any layout. It now uses parquet_writer_properties() and writes a cached file in an older layout again, like the other compression datasets. Signed-off-by: Will Manning <will@willmanning.io>
Benchmarks: Compression 📖Commits: PR vortex / vortex-file-compressed / ns (1.000x ➖, 1↑ 1↓)
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
vortex / vortex-compact / bytes (no group data, 0↑ 0↓)
vortex / parquet / ns (1.017x ➖, 2↑ 6↓)
vortex / parquet / bytes (1.018x ➖, 0↑ 2↓)
vortex / parquet-snappy / bytes (no group data, 0↑ 0↓)
vortex / arrow-ipc / ns (1.001x ➖, 0↑ 1↓)
vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)
|
Benchmarks: Random Access (S3) 📖Commits: PR How to read Verdict and Engines
random-access / vortex-file-compressed / ns (0.818x ➖, 2↑ 1↓)
random-access / parquet / ns (0.792x ➖, 4↑ 0↓)
random-access / lance / ns (0.893x ➖, 1↑ 0↓)
|
The TPC-H and TPC-DS generators write Utf8View columns. Since the concatenated TPC-H files keep their Arrow schema, compress-bench read them as Utf8View. The Arrow IPC writer writes every buffer that a view array refers to, and batches from one Parquet reader share buffers, so TPC-H's Arrow IPC files grew by up to 20 times. TPC-DS's new Arrow IPC sizes had the same problem. compress-bench's Arrow input and the size-only Arrow IPC copies now read Utf8View as Utf8 and BinaryView as Binary, as develop's input was. Signed-off-by: Will Manning <will@willmanning.io>
Benchmarks: Random Access 📖Commits: PR How to read Verdict and Engines
vortex / arrow-ipc / ns (0.961x ➖, 5↑ 2↓)
random-access / vortex-file-compressed / ns (0.996x ➖, 0↑ 0↓)
random-access / parquet / ns (0.529x ✅, 12↑ 0↓)
random-access / lance / ns (1.005x ➖, 0↑ 0↓)
random-access / litdata / ns (1.002x ➖, 0↑ 0↓)
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9997597ad1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
arrow_uncompressed_size still read view types, and Arrow's memory size counts a shared buffer once in every batch that refers to it. It now reads through plain_arrow_reader, like the compression input, so each ratio divides by the size of the data the formats compressed. Signed-off-by: Will Manning <will@willmanning.io>
|
Claude review automation is disabled for pull requests that modify Why:
Ask a human reviewer to inspect workflow changes directly. |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
* Merge the size groups into one File Sizes group on bench The Compression Size and SQL File Size groups showed the same metric, file size per format, from two suites. One File Sizes group now charts the compress-bench datasets and three SQL scopes: ClickBench, TPC-H at SF10, and TPC-DS at SF1. Old Compression Size links redirect to it, and the homepage card reads the group by its new or old name. The summary compares each format with zstd Parquet on the compress-bench datasets and on TPC-DS. Each format uses its newest commit that has zstd Parquet. The wide-table and sparse-wide families count as one dataset each. A format measured on only some datasets says how many, has no total, and ranks after the complete formats. Parquet codec variants, such as parquet-snappy, draw in Parquet's color with a dashed line. Signed-off-by: Will Manning <will@willmanning.io> * Keep the SQL File Sizes group beside File Sizes on bench File Sizes moves to the Compression Size group key, so old Compression Size links open it. SQL File Sizes comes back on its own key: it charts every SQL benchmark and scale factor and ranks formats against Vortex, as before. Each group's info tooltip says how the two differ. Signed-off-by: Will Manning <will@willmanning.io> * Note the TPC-DS size-only formats in the bench contract Signed-off-by: Will Manning <will@willmanning.io> * Rank a query series that skips a query below complete ones on bench A series with no time for a query now ranks below every series that ran all of them. The missing-series penalty alone scores a skipped query at 2x the best time, so a series could rank first by skipping the slowest query. Lance skips TPC-DS q72, which it cannot run. Snappy Parquet joins TPC-DS's queries, so a codec variant is now dotted, and dash-dotted on DuckDB, to keep it apart from DuckDB's dashed Parquet. Signed-off-by: Will Manning <will@willmanning.io> * Block Vercel's preview Toolbar in Lighthouse runs Vercel adds its Toolbar to preview deployments, and the proxy's x-vercel-skip-toolbar header does not stop it. The Toolbar loads from vercel.live and Pusher, and its third-party cookies and issues cut Best Practices to 0.78 on every audited page. The site loads neither host, so Lighthouse now blocks both. Signed-off-by: Will Manning <will@willmanning.io> * Explain bench's two Parquet variants on the methodology page Parquet is zstd level 3, the size and throughput baseline. Parquet (Snappy) is Parquet as DuckDB writes it by default, which TPC-DS SF1 queries and the compression benchmark sizes. ClickBench and FineWeb query their published files as Parquet, and ClickBench's are Snappy. The page also says that a query summary ranks a series that missed a query below the complete ones, as Lance on TPC-DS q72. The contract now names file sizes by format directory, not by codec. Signed-off-by: Will Manning <will@willmanning.io> * Name DuckDB 2.0 as the writer of Parquet (Snappy) on bench vortex-data/vortex#11720 now installs the DuckDB 2.0.0 CLI, the version the queries run on. Signed-off-by: Will Manning <will@willmanning.io> * Explain how bench's Parquet is tuned on the methodology page A new section gives each Parquet writer setting and the test behind it: zstd level, dictionary size, page size and page index, Bloom filters, and row groups. Parquet (Snappy) is now the same Parquet with Snappy in place of zstd, rather than Parquet as DuckDB writes it, and TPC-DS at scale factor 1 no longer queries it. Signed-off-by: Will Manning <will@willmanning.io> * Measure a size family only when a format has all of it, and bump the cache A format with one wide table or sparse-wide dataset counted the whole family as measured, so it could rank in the complete tier against formats measured on every member. A family now counts as measured only when the format has every dataset of it that any format has. The File Sizes group reuses the Compression Size slug with another name, other charts, and another summary, so the group payload cache moves to v16. Signed-off-by: Will Manning <will@willmanning.io> * Rename File Sizes to File Sizes & Compression Ratios, and leave DuckDB's file out of its ranking TPC-DS records DuckDB's database file, which is a query engine's storage rather than a file format, so the summary no longer ranks it. The group still charts it. The contract and methodology also say that TPC-DS SF1 records Snappy Parquet's size. Signed-off-by: Will Manning <will@willmanning.io> * Rename the size group to Compression & File Size The new name keeps more continuity with the Compression Size group it replaces. Signed-off-by: Will Manning <will@willmanning.io> * Keep the renamed size groups' anchors, and describe the size ranking Old #compression-size and #sql-file-size links land on the renamed groups. The group tooltip says which scopes it charts and which it ranks. The methodology page explains the Compression & File Size summary's totals and how it counts the wide-table suites. Signed-off-by: Will Manning <will@willmanning.io> * Label zstd Parquet as parquet-zstd in the Compression & File Size group The group charts Snappy Parquet beside zstd Parquet, so its legends, tooltips, toolbar chips, and summary card name the zstd files parquet-zstd, and the card's baseline header reads "vs parquet-zstd". ClickBench's chart keeps parquet, because it charts the published Snappy files. The series keys stay parquet, so filters, colors, and URLs are unchanged. A compression chart's permalink page uses the same label; a file-size permalink page does not, because the SQL File Sizes group shares those charts. The group payload cache moves to v17. Signed-off-by: Will Manning <will@willmanning.io> * Order the size summary by its Parquet score, and document its Arrow ratio The Compression & File Size card ordered formats by their ratio versus Arrow once every format had one. TPC-DS has no Arrow size, so that order left out the one SQL scope the summary ranks. A size summary now orders by the Arrow ratio only when it covers every dataset each format is ranked on, and by the Parquet ratio otherwise. The String Encoding Size card, whose Arrow ratios cover every dataset, keeps its order. The methodology now says what the vs Arrow column measures, that it covers the compression datasets alone, and how the card is ordered. It also says that the compression benchmark writes 1,048,576-row row groups. The group payload cache moves to v18. Signed-off-by: Will Manning <will@willmanning.io> * Move the methodology link and build commit above the first group On the landing page, the line with the methodology link and the build commit was at the bottom, after every group. It now opens the page's main content, left-aligned with the group cards and small, and the methodology link is underlined so it reads as a link. The chart and methodology pages keep the line at the bottom. Signed-off-by: Will Manning <will@willmanning.io> * Name ClickBench on the group's renamed parquet chip In the Compression & File Size group, the toolbar's parquet-zstd chip also toggles ClickBench's parquet line, which is the published Snappy files. A renamed chip's title now says so: "parquet-zstd, and parquet on clickbench". Signed-off-by: Will Manning <will@willmanning.io> * Rank SQL Benchmark File Sizes against parquet, one score per benchmark Rename the SQL File Sizes group to SQL Benchmark File Sizes and move it after the query groups, since it holds the files those queries read. Its summary now compares each format with parquet, bench's standard Parquet, at each format's newest commit that also has parquet. It ranks only JOB, Polar Signals, statpopgen, TPC-DS, TPC-H, and nested TPC-H, whose Parquet bench writes with its own settings; ClickBench, FineWeb, Appian, sorted ClickBench, and the Vortex queries keep their charts. Each benchmark counts once, as the geometric mean of its scale factors, so TPC-H's three scale factors no longer outweigh the rest. DuckDB's database file is left out, as in the Compression & File Size summary. The card drops the Total and vs Arrow columns, which this summary never filled, and says "of N benchmarks" in its coverage note. With no baseline, the landing page's headline returns for this group. Signed-off-by: Will Manning <will@willmanning.io> * Show a partial file size score as not measured on the homepage The homepage's file size panel dropped bench's coverage for that one panel, so a format without TPC-DS sizes, such as Lance today, showed a geomean over fewer datasets beside formats measured on all of them. Lance is about 2x Parquet without TPC-DS and 4.3x on it, so the panel flattered it. Pass the coverage through, as the other panels do; results.js shows such a row as "Not measured". Signed-off-by: Will Manning <will@willmanning.io> * Count a SQL benchmark as measured when a format has any of its scale factors The SQL Benchmark File Sizes card counted a benchmark only when a format had every scale factor of it, so Lance, with TPC-DS at SF1 alone, showed a 2.12x score beside "measured on 0 of 6 benchmarks". It now counts one scale factor as enough, so Lance shows 1 of 6 and still ranks after the formats measured on every benchmark. The Compression & File Size summary keeps the stricter rule for its families. Signed-off-by: Will Manning <will@willmanning.io> * Rank DuckDB's file in the SQL size summary when it has every benchmark DuckDB's database file is the file DuckDB's queries read, so the SQL Benchmark File Sizes summary keeps it when it has a size on every benchmark the summary covers. With fewer, it stays out, as in the Compression & File Size summary. Signed-off-by: Will Manning <will@willmanning.io> * Mark a size headline partial when Vortex lacks some datasets The collapsed group's size headline read the Vortex ratio alone, so a mean over fewer datasets than the summary covers showed as a plain "smaller than Parquet" claim while the card marked it partial. The headline now adds "(partial)" when Vortex's measured count is below the total. Signed-off-by: Will Manning <will@willmanning.io> * Show Parquet's codecs as parquet (zstd) and parquet (snappy) The Compression & File Size group labeled its two Parquet series parquet-zstd and parquet-snappy. They now read parquet (zstd) and parquet (snappy), in the legends, tooltips, chips, and the summary's "vs" header. ClickBench's chart still shows plain parquet for its published Snappy files. A renamed chip's title now lists, for each renamed format, the charts that keep that format's real name, so the parquet (snappy) chip does not claim ClickBench charts parquet-snappy. Signed-off-by: Will Manning <will@willmanning.io> * Name zstd Parquet in size headlines, and weigh llms.txt's SQL sizes by benchmark In the Compression & File Size group, the collapsed headline and llms.txt said "Parquet" although the summary divides by zstd Parquet and the group also charts Snappy Parquet. They now say "Parquet (zstd)". llms.txt described the SQL Benchmark File Sizes score as a geometric mean over datasets, and its range as across datasets. The score now says each benchmark counts once, as the geometric mean of its scale factors, and the range is across benchmarks and scale factors. Signed-off-by: Will Manning <will@willmanning.io> --------- Signed-off-by: Will Manning <will@willmanning.io>
This changes how `vortex-bench` builds the SpatialBench `trip`, `building`, and `customer` tables, and which SpatialBench crates it depends on. The `zone` table still comes from the external `spatialbench-cli`. Before, SpatialBench used `spatialbench` and `spatialbench-arrow` 0.2.0, the latest release. `spatialbench-arrow` builds on Arrow 56, and the workspace's Parquet 59 writer can't take Arrow 56 batches. So `wkb.rs` wrote them through a second Parquet 56 copy, aliased as `spatialbench-parquet`. That kept 13 Arrow 56 and Parquet 56 crates in `Cargo.lock`, and Renovate's PRs to bump the alias could never build (#11320 is the latest). After, the generators still come from `spatialbench` 0.2.0 on crates.io, which does not depend on Arrow. The Arrow conversion for the three tables is now in `datagen/batches.rs`, about 250 lines adapted from `spatialbench-arrow` 0.2.0 and built on the workspace Arrow. A SpatialBench release can no longer hold back an Arrow, Parquet, or DataFusion upgrade. Upstream took ten months to move from Arrow 56 to 58. `wkb.rs` now writes with the workspace `parquet` and the shared `parquet_writer_properties()` from #11720, and `Cargo.lock` has 20 fewer packages. The cost is a copy that must keep upstream's schemas and values. Compare it with upstream when `spatialbench` changes version. Its header names the source tag. The remaining Parquet 58.4.1 copy comes from `lance-bench`, which needs it for lance 14. At scale factors 0.1 and 1, the new code writes Parquet files that are byte-identical to those from `spatialbench-arrow` on the same Arrow and Parquet. Against the old Arrow 56 build, every table has the same schema, row count, values, and GeoParquet `geo` metadata. File sizes differ by less than 1%, which fits the writer changes between Parquet 56 and 59. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Will Manning <will@willmanning.io>
This changes how the benchmarks write Parquet, and which Parquet sizes they record. ClickBench and FineWeb still query their published files as
parquet.Before, each suite set its own Parquet writer settings. statpopgen and SpatialBench's zone table wrote uncompressed Parquet, and the
vortexsuite wrote Snappy. Now every generated Parquet file uses one config, invortex-bench/src/utils/parquet.rs. It sets zstd level 3, dictionaries of at most 64 KiB, a page index, and pages of at most 20,000 rows and 1 MiB. Each file records a layout version, and the generators write a cached file in an older layout again.Before, TPC-H and nested TPC-H wrote row groups of 1,048,576 rows on all storage, and TPC-DS wrote 122,880. Now every query suite writes 122,880 rows (DuckDB's default) for local NVMe and 1,048,576 rows (parquet-rs's default) for S3. In a TPC-H test at scale factor 10, the small row groups were 28% to 38% faster on NVMe. On S3, the large row groups were 14% to 21% faster on DataFusion, and 3% to 8% slower on DuckDB.
Each other setting was tested on its own, on TPC-H at scale factor 10 and on the random access datasets. 1 MiB dictionaries made files 22% larger, DataFusion 8% slower, and lookups up to 2.8 times slower. Bloom filters made files 14% larger and sped up no query. The 20,000-row page limit made lookups up to 4 times faster at the same size.
compress-bench and TPC-DS at scale factor 1 now also record the size of
parquet-snappy: the same Parquet asparquet, with Snappy in place of zstd. No query reads it. TPC-DS at scale factor 1 also records the size of Arrow IPC and queries Lance on DataFusion. Lance skips q72, which overflows its 32-bit string offsets.Parquet sizes and times change for every suite that generates Parquet. The bench side is vortex-data/vortex.dev#227.
🤖 Generated with Claude Code