You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Nested struct columns are written as one flat array, not Rust's struct layout #724
Rust writes a struct column as a vortex.struct layout: each field is its own column, chunked and compressed on its own, with a validity child when the struct is nullable. vortex-java's LayoutTree uses STRUCT only for the file root, so a nested struct column is one flat struct array (under vortex.masked when nullable).
Found on Raincloud mnist: image (struct<bytes: binary, path: utf8>) is 2.2% bigger than Rust's (18.18 MB vs 17.80 MB from vortex-jni 0.91). Rust's layout also lets a reader project or prune one field without decoding the others.
Fix: recurse into struct fields at the layout level, with per-field repartition and zone maps and a validity child. The reader already reads these layouts in Rust-written files. Largest of the three Raincloud size gaps; depends on the nested repartition issue for per-field chunking.
Test first: a vortex-jni cross-check writing a nullable struct column, asserting Java's layout tree matches Rust's.
Activity