Skip to content

Skip DataFusion datasource v1 in nightly benchmarks, and fix the nested TPC-H S3 data path - #11726

Merged
lwwmanning merged 1 commit into
developfrom
wm/nightly-bench-fixes
Nov 21, 2026
Merged

lwwmanning merged 1 commit into
developfrom
wm/nightly-bench-fixes

Conversation

@lwwmanning

Copy link
Copy Markdown
Contributor

This changes the nightly SQL benchmark matrix in bench-orchestrator/bench_orchestrator/ci_matrix/catalog.py. The develop and PR presets are unchanged.

Before, nightly ran DataFusion's datasource integration on the v1 scan for every benchmark, which #11685 added. On its first night, six of those jobs failed. TPC-DS SF=10 hung on q62 until the 120-minute job timeout, and the runner shut down during TPC-DS SF=100 q62 and TPC-H SF=100 q18. After, nightly skips datasource v1. The file v1 scans on DataFusion and DuckDB stay, because they pass. Develop and pr-scan still run datasource v1 on ClickBench and TPC-H SF=1.

Before, Nested TPC-H on S3 failed every night since #11191. prepare-data wrote the data under tpch-nested/100/list, but the upload read tpch-nested/100.0/list. After, the upload reads the directory that prepare-data writes. A new test checks that each S3 data path names the scale factor its job passes.

🤖 Generated with Claude Code

…ed TPC-H S3 data path

Nightly run 44843664966 lost six jobs to DataFusion's datasource
integration on the v1 scan: TPC-DS hung on q62 until the job timed
out, and the runner shut down during TPC-DS q62 and TPC-H q18. Nightly
now skips that combination (`NIGHTLY_SKIP_SCANS`). The file v1 scans
stay.

Nested TPC-H on S3 has failed every night since #11191: prepare-data
writes `tpch-nested/100/list` for `scale_factor=100`, but the upload
read `tpch-nested/100.0/list`. The catalog now names the directory
prepare-data writes, and a test checks that each S3 `local_dir` names
the scale factor its job passes.

Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning
lwwmanning marked this pull request as ready for review November 21, 2026 22:45
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-21T22:46:17.495836Z a13ba02 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@lwwmanning
lwwmanning enabled auto-merge (squash) November 21, 2026 22:49
@lwwmanning
lwwmanning merged commit 88a6d18 into develop Nov 21, 2026
157 of 158 checks passed
@lwwmanning
lwwmanning deleted the wm/nightly-bench-fixes branch November 21, 2026 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants