You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(writer): write vortex.union (declared in Rust's unstable2026.10.0 edition) #660
Write side of #442. #442 deferred the writer because vortex.union was in no Rust edition ("Revisit when an edition declares it"). That is no longer true: Rust's unstable2026.10.0 edition (vortex/editions/unstable/unstable2026.10.0.toml, vortex-edition/src/declarations/unstable/mod.rs) lists vortex.union. docs/compatibility.md (lines 39, 150) still says "in no edition" and needs correcting as part of this.
Scope
UnionEncodingEncoder in writer.encode, registered in WriteRegistry.Builder#registerDefaults(): type ids (U8, nullable iff the union is) + one row-aligned child per variant, no metadata, no buffers (Rust rejects either). Layout per Support DType::Union and the vortex.union array (read side) #442.
Cascading: compress type_ids and each child independently and rebuild the union, as vortex-compressor/src/compressor/cascade.rs does.
Edition gating: Rust now has an unstable family; EditionFamily is CORE/PREVIEW/ZSTD. Decide whether to add it (ADR 0023 says closed enum mirroring Rust's families, so probably yes) and make a default write never emit vortex.union.
Update docs/compatibility.md, AGENTS.md if the edition family changes, CHANGELOG.
Testing (vortex-jni cannot help: vortex-arrow bails on DataType::Union)
Ideas, cheapest first:
Rust-native cargo harness (as for the Pco encoder): a small Rust program linking vortex-array/vortex-file that (a) reads a Java-written file and dumps the array tree + values, and (b) builds a UnionArray and writes it with the unstable edition, so Java reads real Rust output. This also gives the Support DType::Union and the vortex.union array (read side) #442 decoder its first real-producer coverage.
Rust-side vtx/vortex-tui browse on a Java-written file, as a quick manual smoke check.
Byte-compare node trees: write the same logical union through the Rust harness and Java, diff the ArrayNode trees (the replicate-JNI-on-size-gaps approach).
Reuse Rust's own test vectors in vortex-array/src/arrays/union/tests as unit fixtures.
Activity