OpenEarable Protocol Schemas

This repository defines OpenEarable binary protocols as YAML schemas and generates language bindings from those schemas.

Generate Code

Run the generator from the repository root:

.venv/bin/python -m protocol_generator

By default, schemas are read from schemas/*/protocol.yml and generated code is written to:

• generated/dart/lib/open_earable_protocols.dart
• generated/dart/lib/src/protocol_runtime.dart
• generated/dart/lib/src/*_protocol.dart
• generated/c/include/protocol_runtime.h
• generated/c/include/*_protocol.h
• generated/c/include/zephyr/*_ble.h
• generated/c/src/protocol_runtime.c
• generated/c/src/*_protocol.c
• generated/c/CMakeLists.txt

Custom directories can be provided when integrating with a build system:

.venv/bin/python -m protocol_generator --schema-dir schemas --output-dir generated

Generate only one language target:

.venv/bin/python -m protocol_generator --language dart

The generator is implemented as the protocol_generator package. See docs/architecture.md for module responsibilities and the process for adding another language.

Each language emits one shared binary codec runtime. Generated protocol files import or include that runtime instead of embedding duplicate reader, writer, status, and primitive codec helpers.

Dart Package

The generated Dart bindings form the publishable open_earable_protocols package under generated/dart. Flutter and Dart applications can depend on that directory and import all generated protocols from:

import 'package:open_earable_protocols/open_earable_protocols.dart';

Before publishing a release, update generated/dart/pubspec.yaml and generated/dart/CHANGELOG.md, regenerate the bindings, and run:

cd generated/dart
dart pub publish --dry-run

Schema Format

Each protocol schema defines metadata and ordered message fields:

protocol: audio-response
version: 1
description: Protocol description.

messages:
  message_name:
    description: Human-readable documentation for the generated message type.
    fields:
      - name: count
        type: uint8
      - name: values
        type: uint16[count]

Message descriptions are optional but recommended. The generator emits them as Dart documentation comments and C Doxygen comments. When omitted, it generates a stable description from the message and protocol names.

Protocol Compatibility

Protocol changes must remain backwards compatible. Existing devices, applications, and generated bindings must continue to understand payloads that were valid before the change. In practice, this means existing message names, field order, field types, BLE UUIDs, and tagged-union discriminator values are stable once released.

Prefer additive changes. New messages, new fields at explicitly compatible extension points, new tagged-union variants with unused discriminator values, and new BLE characteristics are safer than changing existing wire contracts. Never reuse a removed discriminator value or characteristic UUID for a different meaning.

Optional BLE transport metadata is retained in generated APIs:

transport:
  ble:
    service_uuid: 12345678-1234-5678-1234-56789abcdef0
    characteristics:
      - name: measurements
        uuid: 12345678-1234-5678-1234-56789abcdef1
        properties: [read, notify]

Supported characteristic properties are broadcast, read, write_without_response, write, notify, indicate, authenticated_signed_writes, and extended_properties. Hyphenated and UniversalBLE-style camelCase spellings such as writeWithoutResponse are also accepted.

Dart exposes framework-neutral service and characteristic definitions through a protocol-specific *BleUuids constants class. Its property enum names match UniversalBLE's CharacteristicProperty names, so an application can convert them without making the generated package depend on UniversalBLE:

import 'package:open_earable_protocols/open_earable_protocols.dart';
import 'package:universal_ble/universal_ble.dart';

final definition = AudioResponseBleUuids.transferControlCharacteristic;
final properties = definition.mapPropertiesByName(
  CharacteristicProperty.values,
  (property) => property.name,
);
final characteristic = BleCharacteristic(definition.uuid, properties, []);

isReadable and isWritable are available when deriving peripheral permissions.

C exposes UUID strings and standard GATT property bitmasks as protocol-prefixed macros. An additional opt-in Zephyr header maps those values to Zephyr UUID declarations, characteristic properties, and default permissions:

#include "zephyr/audio_response_ble.h"

BT_GATT_SERVICE_DEFINE(audio_response_service,
  BT_GATT_PRIMARY_SERVICE(AUDIO_RESPONSE_ZEPHYR_SERVICE_UUID),
  BT_GATT_CHARACTERISTIC(
    AUDIO_RESPONSE_ZEPHYR_TRANSFER_CONTROL_CHARACTERISTIC_UUID,
    AUDIO_RESPONSE_ZEPHYR_TRANSFER_CONTROL_CHARACTERISTIC_PROPERTIES,
    AUDIO_RESPONSE_ZEPHYR_TRANSFER_CONTROL_CHARACTERISTIC_PERMISSIONS,
    read_callback, write_callback, NULL),
  BT_GATT_CCC(configuration_changed, BT_GATT_PERM_READ | BT_GATT_PERM_WRITE));

Add a BT_GATT_CCC entry when a characteristic uses notify or indicate. The Zephyr adapter derives ordinary read/write permissions; applications remain responsible for selecting encrypted or authenticated permission variants when required.

Supported scalar field types are:

• Integers: uint8, uint16, uint32, uint64, int8, int16, int32, and int64.
• Floating point: float and double.

Multi-byte values are encoded little-endian. float uses the 4-byte IEEE-754 binary32 representation and double uses the 8-byte IEEE-754 binary64 representation.

In C, 64-bit fields use uint64_t and int64_t. In Dart, both use BigInt to preserve every value on native and JavaScript targets. For example, BigInt.parse('18446744073709551615') represents the maximum uint64. Both types occupy eight little-endian bytes; signed values use two's complement. Dart encoders reject values outside 0..2^64−1 (uint64) or −2^63..2^63−1 (int64) with RangeError. Arrays of these types use List<BigInt> in Dart. 64-bit fields cannot serve as array lengths or union discriminators.

Supported dynamic field types:

• bytes[length_field] for variable-length byte payloads.
• uint16[length_field] and other scalar arrays for variable-length arrays.
• float[length_field] and double[length_field] for floating-point arrays.
• message_name[length_field] for arrays of nested message records.

Length fields must be integer fields of at most 32 bits and must appear before the dynamic field that references them.

Tagged unions use integer discriminators of at most 32 bits and declare their discriminator width and stable numeric tags explicitly:

- name: command
  type: union
  tag_type: uint8
  variants:
    0: transfer_start
    1: transfer_commit
    2: transfer_abort

The generated C API exposes a named discriminator enum, typed constructors, and a typed visitor. It also generates typed setters for union fields. Construct single-union messages with functions such as audio_response_transfer_control_from_start, or set an existing message with audio_response_transfer_control_set_command_start; dispatch decoded values with audio_response_transfer_control_dispatch. The generated Dart API uses a sealed payload interface and named factories such as AudioResponseTransferControl.start. Both APIs infer the discriminator from the selected payload type.

C Decode Ownership

Generated C structs use pointers for variable-length byte and array fields. The caller owns that memory. Dynamic fields also expose a size_t <field>_capacity member in C (not part of the wire payload). For byte fields it counts bytes; for scalar and record arrays it counts elements. Initialize every destination pointer and its capacity before using *_decode_bounded. Each record slot must have its own nested buffers initialized. The decoder checks capacities before writing and returns PROTOCOL_ERROR_BUFFER_TOO_SMALL for insufficient storage, without allocating memory. Output can be partially modified on failure; discard it unless decoding succeeds.

Existing *_decode entry points retain caller-managed capacity behavior for legacy byte/scalar arrays. Record arrays always check their capacity and use bounded decoding for their elements, including through *_decode. Prefer *_decode_bounded for new code. Capacities are ignored when encoding. Rebuild C consumers after regeneration because the generated structs have new members. The decoder fills the caller-provided buffers and reports PROTOCOL_ERROR_INVALID_DATA if the input is truncated or has an unknown union tag. Compile protocol_runtime.c once and link it together with all generated protocol-specific C sources. All generated C encode/decode functions return the shared protocol_status_t type.

C Package

The generated C99 bindings under generated/c are a CMake library. CMake projects can add the directory and link OpenEarable::Protocols. The generated target compiles every protocol source and exposes generated/c/include to consumers.

See generated/c/README.md for general usage and docs/open-earable-2-integration.md for explicit firmware integration.

Arrays of records

Declare a record as a message, then reference it as an array element:

messages:
  mode:
    fields:
      - name: id
        type: uint8
      - name: name_length
        type: uint8
      - name: name
        type: bytes[name_length]
  supported_modes:
    fields:
      - name: count
        type: uint8
      - name: modes
        type: mode[count]

The wire format is the count followed by the records in order. No extra record size, discriminator, or padding is inserted. Dart emits a list of record objects and checks that its length matches count during encoding. C emits a pointer to record structs and modes_capacity for decoding. The schema loader orders dependencies before their consumers and rejects direct or indirect recursive references. Message declaration order does not change field order. See the power-saving protocol for a complete BLE example.