Repository navigation
Conversation
Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Martin Prammer <martin@spiraldb.com>
Contributor
|
I don't think we need to flatten everything |
Contributor
|
we have fixed it in #11204 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reading a
list<struct>column through the DuckDB extension fails withchunked DuckDB export range 0..N crosses chunk boundary at Mas soon as the list's elements span more than one chunk, which is what the file reader produces for any such column larger than one row group. The list exporter copies all of its elements into one DuckDB vector in a singleexport(0, n)call and asks for a flattened element exporter, but the struct exporter ignored that request and built its children withflatten = false, so a chunked child became the chunk-alignedChunkedExporter, which rejects a range crossing a chunk boundary.Changes
flattenis now threaded throughcanonical::new_exporterandstruct_::new_exporterto the struct's children, and through the two remaining paths that canonicalize and then build a child exporter (constant::new_exporter_with_validityand the flatten branch of the dictionary exporter). Top-level structs still useflatten = false, so their output and cost are unchanged; the only behaviour change is that struct children under a list, fixed-size list, or dictionary are canonicalized instead of erroring.test_export_list_of_structs_with_chunked_childreproduces the failure with a two-chunk child and a list that straddles the boundary.🤖 Generated with Claude Code