Skip to content

Zone table of a column whose few bounds Rust's trained FSST table compresses #709

Open

Description

@dfa1

Split from #583. The zone-table segment of a string column has Rust's shape, but when the zone bounds are few and Rust's trained FSST table compresses them, the encoding of that segment can differ from ours, because the symbol tables are not byte-identical (tie order, see docs/compatibility.md).

To do: find which column shapes make Rust compress the bounds with FSST and what we write for them (vortex-jni cross-check of the zone table's encoding tree, as EncodingTrees does for data segments), then decide whether to match the shape or record it as a deliberate divergence.

Low priority: affects only the encoding of a small stats segment, not what is read.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions