Skip to content

KLL: validate n on deserialize - #594

Open
leerho wants to merge 2 commits into
masterfrom
kll-deserialize-n
Open

leerho wants to merge 2 commits into
masterfrom
kll-deserialize-n

Conversation

@leerho

@leerho leerho commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Since #585, update() and merge() cap n at kll_constants::MAX_N (2^63 - 1, as in datasketches-java), but deserialize() still accepted any n from the image. A corrupt image could therefore give a sketch with n above the cap, a sketch with retained items and n = 0 (which reports itself as empty), or a sketch whose n is smaller than the number of items it retains.

Both deserializers (stream and bytes) now check n once the levels are validated, and throw std::invalid_argument if:

  • n is 0 or above 2^63 - 1, or
  • n is less than the number of retained items, since every retained item has a weight of at least 1.

Empty images don't store n, and single-item images have n = 1 and one item, so nothing changes for them.

Tests, added to "deserialize rejects inconsistent levels": n = 2^63, n = 0 and n = (number of retained items - 1) are rejected from bytes and from a stream, and n = 2^63 - 1 is accepted. The rejection sections fail on master. All KLL tests pass under ASan and UBSan.

The same checks are in apache/datasketches-java#810, which makes Java KLL reject n < 0 and n below the number of retained items.

🤖 Generated with Claude Code

update() and merge() cap n at kll_constants::MAX_N since #585, but
deserialize() accepted any n, so a corrupt image could produce a sketch
with n above the cap, or a non-empty sketch with n = 0, which reports
itself as empty. Both deserializers now check n for images that are
neither empty nor a single item, and throw std::invalid_argument
otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coveralls

coveralls commented Nov 20, 2026 •

Copy link
Copy Markdown

Coverage Report for CI Build 44804563962

Coverage decreased (-0.01%) to 97.04%

Details

  • Coverage decreased (-0.01%) from the base build.
  • Patch coverage: 9 of 9 lines across 1 file are fully covered (100%).
  • 2 coverage regressions across 1 file.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

2 previously-covered lines in 1 file lost coverage.

File Lines Losing Coverage Coverage
common/include/serde.hpp 2 98.02%

Coverage Stats

Coverage Status
Relevant Lines: 17469
Covered Lines: 16952
Line Coverage: 97.04%
Coverage Strength: 1883603.33 hits per line

💛 - Coveralls

@leerho
leerho requested a review from proost November 20, 2026 19:30
Every retained item has a weight of at least 1, so n can't be less than
the number of retained items. The check now runs after the levels are
validated, for every image that isn't empty. Matches the KLL check in
apache/datasketches-java#810.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leerho leerho changed the title KLL: reject n outside 1 to 2^63 - 1 on deserialize KLL: validate n on deserialize Nov 20, 2026

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current logic is correct. I leave minor comment .

// a sketch that is not empty has n from 1 to kll_constants::MAX_N, the cap that update() and merge() keep,
// and at least as large as the number of retained items, each of which has a weight of at least 1
template<typename T, typename C, typename A>
void kll_sketch<T, C, A>::check_n(uint64_t n, uint32_t num_retained) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about adding check "num_retained == 0"?

Because this function is called after empty sketch early returned. So num_retained should be not zero.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants