You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
RunEndEncodingDecoder drops nullable run-values validity — silent null loss #225
Found by Raincloud triage round 4 (#205) on uci-online-retail, customerid (u16?). Same silent-corruption class as #210 (null rows decode as plausible values), in the lazy RunEnd path.
RunEndEncodingDecoder unwraps the values child's MaskedArray and passes only the raw data to LazyRunEnd*Array, which has no slot for per-run validity:
Array tree (chunk 0): vortex.runend(u16?) → values: fastlanes.for(u16?) → bitpacked(u16?) validity_child bool (1843 null runs). Ground truth vs Java:
row 622 rust=None java=12346 (= FoR reference base)
row 499785 rust=None java=0 (= raw underlying value)
row 0 rust=17850 java=17850 OK
135,080 wrong cells. The differing fillers (12346 vs 0) confirm it's the run-values' validity being dropped (chunk1 uses vortex.primitive(u16?) with its own validity child — same drop).
Fix direction: LazyRunEnd*Array must carry the run-values' validity and mark expanded rows null (a null run → all its rows null), mirroring the #210 approach; wrap the result in MaskedArray when the values child carried validity.
Activity
fix: DateTimeParts propagates null component rows instead of throwing (…
test: gate-running interop test for null-fill Sparse and null-run Run…
refactor(reader): share MaskedArray unwrap/rewrap across 9 decoders