Repository navigation
Description
Summary
Re-imported the Raincloud nyc-311 slug (18.49M rows, 42 mostly-Utf8 columns) through the current writer after the #287 FSST rewrite: vortex-java=2854.88MB vs the corpus reference (vortex-jni/Python-bindings-written) 1762.17MB — 1.62x larger. Diffing the two files' layout trees (vortex-cli inspect) surfaced two concrete, well-substantiated gaps, not a regression — FSST itself is working correctly (see #298, a separate inspector display issue that initially made this look like FSST wasn't being used at all).
Finding 1 — global-dict cardinality cap (GLOBAL_DICT_MAX_CARDINALITY = 2048) is too conservative
From ADR 0021, the cap was set deliberately to bound buffering memory during the OOM fix — not because 2048 is an inherent ceiling for when a global dictionary stops being worthwhile.
The reference file uses a genuinely global dictionary (one dict spanning all 18.49M rows) for columns whose distinct-value count is well past our cap:
| Column | Distinct values (reference) | Currently admitted to global dict? |
|---|---|---|
| Additional Details | 9,388 | No — falls back to per-chunk |
| Street Name | 20,082 | No |
| Cross Street 1 | 27,634 | No |
| Cross Street 2 | 23,674 | No |
| Intersection Street 1 | 16,764 | No |
| Intersection Street 2 | 17,691 | No |
| Landmark | 11,795 | No |
Java's admission gate rejects any column exceeding 2048 distinct values from global dict entirely, pushing these into costlier per-chunk fallbacks (FSST, or worse).
The underlying storage (short[] per-chunk code buffering) already supports cardinality up to ~32,767 without any structural change (see ADR 0021's "Risks to manage" section, which already flags this exact tension: "if the constant is ever raised past either bound, both the buffer element type and codePTypeForSize selection need revisiting together" — for a raise up to ~32767 the short[] buffer type itself does not need to change). Raising GLOBAL_DICT_MAX_CARDINALITY (with the accompanying codePTypeForSize/wire-code-width check ADR 0021 calls out) is a small, low-risk change with likely broad real-world impact, given how many text columns in this file land in the 1K–30K distinct-value range.
Finding 2 — DictEncodingEncoder's value pool is never compressed
writer/src/main/java/io/github/dfa1/vortex/writer/encode/DictEncodingEncoder.java#encodeUtf8 hardcodes the dictionary's distinct-values child as raw vortex.varbin:
EncodeNode valuesNode = new EncodeNode(EncodingId.VORTEX_VARBIN,
MemorySegment.ofArray(varBinMetaBytes),
new EncodeNode[]{offsetsNode},
new int[]{0});It is never cascaded through FSST (or any other candidate). The reference file's layout tags show vortex.dict and vortex.fsst together on the same columns — i.e. Rust dict-deduplicates rows and FSST-compresses the resulting distinct-value pool, capturing both row-level duplication and substring redundancy (shared "STREET"/"AVENUE"/borough names across otherwise-distinct dictionary entries). This is additive on top of Finding 1 — even a column that stays under a raised cardinality cap still pays full uncompressed-varbin cost for its values pool today.
Secondary, less-isolated factor
Rust's physical chunk granularity for the affected columns (32,768–65,536 rows) is noticeably finer than Java's (131,072 rows per flat layout segment for this same file/import path). Finer chunking likely also shifts some per-chunk cost-competition outcomes (dict cardinality gates, FSST training quality) independent of the global-dict question above, but is harder to isolate and is not the primary lever — noted here for completeness, not as a first action item.
Suggested next steps (not started — analysis only, filed per user request)
- Raise
GLOBAL_DICT_MAX_CARDINALITYinVortexWriter(with the wire-code-width follow-through ADR 0021 flags) and re-measurenyc-311's file size — expected to be the highest-leverage single change. - Consider cascading
DictEncodingEncoder.encodeUtf8's values child through the same cost-based competition (FSST/varbin) that non-dict Utf8 columns already get, instead of hardcodingvortex.varbin. - Optionally revisit default physical chunk size for text-heavy columns, once 1–2 are measured.
Where
writer/src/main/java/io/github/dfa1/vortex/writer/VortexWriter.java—GLOBAL_DICT_MAX_CARDINALITYconstant, admission gatewriter/src/main/java/io/github/dfa1/vortex/writer/encode/DictEncodingEncoder.java—encodeUtf8's hardcodedvortex.varbinvalues nodeadr/0021-cardinality-bounded-global-dict-buffering.md— prior design context and the risk this issue's Finding 1 was already flagged against
Activity
dfa1 commentedon Jul 25, 2026
Both findings implemented and merged to
main; the goal is met (and then exceeded).Fixes
3f0efa66)d83d1b93) + per-chunk (6b9d1013)fbedbf03)Result on the hydrated nyc-311 corpus (18.49M rows × 42 cols)
With #302, vortex-java is smaller than the Rust reference on nyc-311 — the gap is closed and reversed.
Remaining per-column upside is tracked as focused follow-ups (both pure upside now that java is ahead overall):
Closing as done.
feat(writer): cascade FSST offset/length children (#299)
feat(writer): cascade Utf8 dict codes so they bitpack (#303)
perf(writer): de-box ArrayStats' distinct/most-frequent counter