Skip to content

fix: ebpps serialize end ptr - #591

Merged
leerho merged 2 commits into
apache:masterfrom
proost:fix-ebpps-serialize-end-ptr
Nov 20, 2026
Merged

leerho merged 2 commits into
apache:masterfrom
proost:fix-ebpps-serialize-end-ptr

Conversation

@proost

@proost proost commented Nov 20, 2026

Copy link
Copy Markdown
Member

size variable in L331 includes header_size_bytes already.

@proost proost self-assigned this Nov 20, 2026
@proost proost changed the title Fix ebpps serialize end ptr fix: ebpps serialize end ptr Nov 20, 2026

@leerho leerho left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the fix is right: size already includes header_size_bytes. Since end_ptr was only used for bounds checks, the bytes were correct before too, so the new test passes on master as well; that's fine as a regression test for serializing with a header.

The same line appears in four other serialize() functions: var_opt_sketch_impl.hpp:330, tuple_sketch_impl.hpp:444, kll_sketch_impl.hpp:521 and density_sketch_impl.hpp:229. Would you fix those here too, so they land together?

@leerho

leerho commented Nov 20, 2026

Copy link
Copy Markdown
Member

A correction to my list above: there are three more, so seven in all besides ebpps: var_opt_sketch_impl.hpp:330, tuple_sketch_impl.hpp:444, kll_sketch_impl.hpp:521, density_sketch_impl.hpp:229, req_sketch_impl.hpp:425, frequent_items_sketch_impl.hpp:234 and quantiles_sketch_impl.hpp:325. The other serialize() functions that take a header size are fine.

@leerho

leerho commented Nov 20, 2026

Copy link
Copy Markdown
Member

No need to touch the other seven here: I am fixing them in a separate PR, so this one can merge as it is.

@leerho
leerho merged commit 7425057 into apache:master Nov 20, 2026
21 checks passed
leerho added a commit that referenced this pull request Nov 20, 2026
serialize(header_size_bytes) set end_ptr to ptr + size, where ptr is
already past the header and size includes it, so the end was
header_size_bytes past the buffer. The same as #591 fixed for EBPPS, in
var_opt, tuple, KLL, density, REQ, frequent items and classic quantiles:
- The capacity checks of the serde used that end, so a serde that wrote
  more than size_of_item() reported could write past the buffer.
- Density compared its output with that end and threw whenever a header
  size was given.

REQ: get_serialized_size_bytes() counted a compactor header for 2 to 4
items, which are written as raw items without one, so serialize() to
bytes returned 20 trailing zero bytes that the stream form doesn't have.

Tests: serialize with a header and deserialize the bytes after it, and
check that a serde that under-reports the size of its items is stopped
by its capacity check. A test helper, undersized_string_serde, is added
to common/test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@proost
proost deleted the fix-ebpps-serialize-end-ptr branch November 21, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants