Stream Frame

The stream_frame component provides a minimal wire framing (v2) for carrying typed, length-delimited, CRC-verified messages over any raw byte stream. It is intentionally free of any ESP-IDF / FreeRTOS dependency, so the framing (CRC-32, frame building, incremental parsing with resynchronization) builds and unit-tests on a host with nothing more than a C++20 standard library.

Wire format (all multi-byte fields little-endian):

[magic u16 = 0x4F54 ("OT")][flags u8][module u8][type u8]
    {[correlation u16] iff flags bit1}[len u32][payload][crc32 u32]

magic — the u16 0x4F54 (“OT”), so the raw bytes are 0x54 (‘T’) then 0x4F (‘O’).
flags — bit0 = reply: the frame’s direction in a request/response exchange — 0 = request (initiator→responder), 1 = response/event (responder→initiator). Which side is the “initiator” depends on the protocol’s roles. bit1 = correlation present; bits 2–3 reserved for future optional header fields; bits 4–7 = format version (currently 1).
module — routing / protocol id (0..255). Dispatcher APIs routes on this, so up to 256 protocols can share one stream.
type — message / transaction type within the module (0..255). The Transaction enum gives recommended standard values (Write / Read / WriteRead / Custom); a protocol may otherwise define its own type values and carry a finer opcode in the payload.
correlation — optional u16 (present only when flags bit1 is set): a protocol-defined correlation / sequence id (opaque to the codec) for matching a response to its request when more than one may be outstanding. It lives in the header, covered by the CRC. Optional fields are flag-gated, so a frame without them is byte-identical to before and more can be added under bits 2–3 without another breaking change.
len — payload length, <= kMaxPayloadSize (4096). The parser rejects and resynchronizes past any oversized length, bounding memory usage.
crc32 — standard zlib CRC-32 over the whole header + payload (golden check value crc32("123456789") == 0xCBF43926).

build_frame() encodes a frame; StreamParser::feed() consumes arbitrary chunks (frames may be split or batched) and yields every complete, CRC-verified frame, resynchronizing at the next intact frame after any corruption. It does not filter by module or type — routing a multi-protocol stream (and ignoring unknown modules) is the job of Dispatcher APIs. The buffering is bounded because the length field is capped.

  namespace sf = espp::stream_frame;

  // Build a request (module 4, type 0x42, a u32 payload) and a reply.
  std::vector<uint8_t> payload;
  sf::put_u32(payload, 0xDEADBEEF);
  const auto request = sf::build_frame(/*reply=*/false, /*module=*/4, /*type=*/0x42, payload);
  const auto reply = sf::build_frame(/*reply=*/true, /*module=*/4, /*type=*/0xC2);

  // Concatenate and feed the parser one byte at a time to show it reassembles
  // frames split across reads.
  std::vector<uint8_t> stream = request;
  stream.insert(stream.end(), reply.begin(), reply.end());

  sf::StreamParser parser;
  for (const uint8_t byte : stream) {
    for (const auto &frame : parser.feed(std::span<const uint8_t>(&byte, 1))) {
      logger.info("frame: module={} type=0x{:02X} reply={} payload={} bytes", frame.module,
                  frame.type, frame.is_reply(), frame.payload.size());
    }
  }
  logger.info(
      "crc32(\"123456789\") = 0x{:08X} (expect 0xCBF43926); dropped {} bytes",
      sf::crc32(std::span<const uint8_t>(reinterpret_cast<const uint8_t *>("123456789"), 9)),
      parser.dropped_bytes());

API Reference

Header File

components/stream_frame/include/stream_frame.hpp

Classes

class StreamParser

Incremental frame parser for the stream-frame protocol.

Feed arbitrary chunks of received bytes (USB bulk transfers, socket reads, single bytes, …) and it yields the complete, CRC-verified frames they contain. On a bad magic, an oversized length field (> kMaxPayloadSize) or a CRC mismatch it resynchronizes by discarding bytes until the next plausible frame start, so a corrupted stream recovers at the next intact frame.

The parser yields EVERY CRC-verified frame; it does not filter by module or type. Routing a multi-protocol stream to the right handler (and ignoring unknown modules) is the job of espp::Dispatcher (or the caller).

Buffering is bounded: because the length field is capped, the parser never retains more than kMaxFrameSize bytes between feed() calls (plus at most the chunk currently being processed), so a remote-supplied length cannot cause memory exhaustion.

Public Functions

inline std::vector<Frame> feed(std::span<const uint8_t> data)

Feed received bytes to the parser.

Parameters:: data – Any number of bytes (frames may be split or batched arbitrarily).
Returns:: All complete, CRC-verified frames terminated by these bytes, in order.

inline void reset(): Discard all buffered bytes (e.g. on transport reconnect or RX overflow).

inline size_t buffered() const: Number of bytes currently buffered awaiting frame completion.

inline size_t dropped_bytes() const: Total bytes discarded so far while resynchronizing (diagnostics).