Repository navigation
fix(writer): no chunked layout around a single chunk, plus a metadata field-parity fitness test - #711
Merged
Merged
Conversation
Rust's ChunkedLayoutStrategy returns the only child itself when a column has one chunk (vortex-layout chunked/writer.rs at 0.90.0: `if child_layouts.len() == 1`). LayoutTree always wrapped the column's flats, and a dict's codes, in vortex.chunked: one extra layout node per single-chunk column, which every Rust and Java reader accepted, so no interop test noticed. Found by FooterFieldParityIntegrationTest (next commit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rtex-jni Every interop test compares what a file holds, never which optional FlatBuffer fields its metadata carries, so a field the Java writer leaves out fails nothing: the footer's file statistics went unwritten for releases (#706). This writes the same rows through both writers and asserts the set of fields present in the postscript, footer, dtype, layout and file statistics is equal, walking the generated readers by reflection (hasX() / non-null table accessors), so a field added to a schema is covered without editing the test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dfa1
force-pushed
the
footer-field-parity
branch
from
November 8, 2026 13:05
5664cdb to
9becd2d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #706 ("how did we never spot it?"): a fitness function that would have.
FooterFieldParityIntegrationTestwrites the same rows through vortex-jni and Java and asserts the set of present optional fields in the postscript, footer, dtype, layout and file statistics is equal. It walks the generated FlatBuffer readers by reflection, so new schema fields are covered automatically.vortex.chunked; Rust 0.90.0 writes the chunk itself (ChunkedLayoutStrategy,child_layouts.len() == 1). Two global-dict tests asserted the old shape and are updated.Separate commits per change. Not covered by the test: fields our
.fbscopies lack entirely (schema drift vs the pinned Rust tag, e.g. the postscript'smetadatasegments) — that would need a schema comparison.🤖 Generated with Claude Code