Skip to content

Use family ID 23 for serialized Density sketches #589

Closed
apache/datasketches-java
#811

Description

@tisonkun

The C++ writers for density_sketch and ebpps_sketch both put 19 in the family byte. A reader inspecting that byte alone cannot identify which sketch it has. The cross-language discussion assigns 23 (0x17) to Density and keeps 19 for EBPPS. TCK discussion

Please update both Density serialization entry points (stream and byte buffer) to write 23. Existing C++ releases and Python wheels have written Density images with 19. A Density reader can continue to read those images by checking the version-1 Density preamble and empty flag before decoding the payload. Byte 0 is 3 for an empty Density sketch and 6 otherwise. EBPPS uses 1 and 5 respectively. Reading an old Density image and serializing it again would then produce an image with ID 23. Density constants · EBPPS constants

The existing float and double snapshot generator can exercise the new ID for cross-language readers. Retaining a released ID-19 Density fixture would also cover legacy reads. C++ snapshot generator

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions