Skip to content

Edition catalog has drifted from Rust's vortex-edition declarations #441

Closed

Description

@dfa1

Our Editions catalog (core/src/main/java/io/github/dfa1/vortex/core/model/Editions.java) no longer matches Rust's vortex-edition/src/declarations (spiraldb/vortex develop). Editions decide what a write may emit (ADR 0023), so the drift means our default writes allow or refuse different encodings than Rust's. That violates the parity prime directive.

Diff (Rust → ours)

Rust edition (min lib) Rust members Ours
core 2025.05 23 arrays, layouts chunked/dict/flat/stats/struct, dtypes date/time/timestamp ✅ same 23 encodings (layouts/dtypes not modelled)
core 2025.06 vortex.pco, vortex.sequence, vortex.zstd ✅
core 2025.10 fastlanes.rle, vortex.fixed_size_list, vortex.listview, vortex.masked ✅
— (no 2026.07 edition) ❌ we have core2026.07.0 = vortex.variant
core 2026.08 (0.84.0) layout vortex.zoned; aggregates bounded_max/bounded_min/max/min/nan_count/null_count; array vortex.onpair ❌ ours core2026.08.0 = vortex.map; OnPair is unstable2026.06.0
core 2026.08.2 vortex.map ❌ missing
core 2026.08.3 (0.85.0) arrays vortex.parquet.variant, vortex.variant; dtype vortex.uuid ❌ missing (variant sits in our core2026.07.0, parquet.variant in unstable2026.04.0)
zstd 2026.02 (own family, origin vortex-zstd) vortex.zstd_buffers ❌ ours unstable2026.02.0
preview 2026.08 (new family, empty) — ❌ family missing
— Rust has no unstable family any more; fastlanes.delta is in no edition ("no edition includes fastlanes.delta yet") ❌ ours: unstable2025.05.0 (delta), unstable2026.04.0 (parquet.variant, patched, tensor.*), unstable2026.06.0 (onpair)

Consequences

  • OnPair: Rust's default (core 2026.08) writes may emit vortex.onpair, but ours refuse it without opting into an unstable edition. Our cascade never offers it by default.
  • Family names: withEdition(UNSTABLE_…) names an edition family Rust no longer has, and zstd/preview cannot be expressed. EditionFamily is a closed enum (CORE/UNSTABLE), so this needs an API change (see the ADR 0023 note in CLAUDE.md).
  • Scope: Rust editions also gate layouts, extension dtypes and zone-map aggregates (EditionMember::layout/dtype/aggregate). Ours model encodings only.

Fix

Re-sync Editions with Rust's declarations (versions, families, members, min_library_version), decide how to represent zstd/preview and the members of the retired unstable family, update docs/compatibility.md's edition table and ADR 0023 as needed, and add a guard that keeps the two catalogs from drifting silently again.

Activity

  1. added
    bugSomething isn't working
    on Oct 3, 2026
  2. dfa1 commented on Oct 3, 2026

    @dfa1
    OwnerAuthor

    Follow-up in ed12c2b: the first sync (05ae8ba) read only vortex-edition/src/declarations, but Rust 0.86.1 also declares edition families in plugin crates. vortex-zstd declares the zstd family (zstd2026.02.0 = vortex.zstd_buffers, buffer-level Zstd for GPU decode, never enabled by default). Added EditionFamily.ZSTD and Editions.ZSTD_2026_02_0; EditionCatalogParityIntegrationTest now also scans every encodings/*/src/editions.rs at the pinned tag, so a new plugin family fails the build too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions