Skip to content

Write struct columns as Rust's vortex.struct layout - #739

Merged
dfa1 merged 1 commit into
mainfrom
fix/struct-layout-724
Nov 12, 2026
Merged

dfa1 merged 1 commit into
mainfrom
fix/struct-layout-724

Conversation

@dfa1

@dfa1 dfa1 commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Fixes #724.

A struct column was one flat array (under vortex.masked when nullable). It is now Rust's struct layout: validity child (nullable only) then one child per field, each chunked/dict-encoded on its own. Fields are zoned only while no struct above is nullable (checked against vortex-jni).

Tests: StructLayoutInteropIntegrationTest compares Java's layout tree with vortex-jni's (nullable and not) and has vortex-jni read Java's file; the two layout cases fail without the change. ./mvnw verify is green.

Not covered: a ColumnEncoding override on a struct column is now ignored (fields are leaves); the mnist size gain is not measured.

🤖 Generated with Claude Code

Each field of a struct column is its own chunked, repartitioned, dict-able
column under a validity child when the struct is nullable; fields are zoned
only while no struct above them is nullable, as in Rust.

Fixes #724

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dfa1
dfa1 merged commit d020649 into main Nov 12, 2026
6 checks passed
@dfa1
dfa1 deleted the fix/struct-layout-724 branch November 18, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nested struct columns are written as one flat array, not Rust's struct layout

1 participant