Recursive AI
onearchiveDocs

Brotli decoder

OneArchive 0.7.0-preview.1docs/brotli-decoder.md at 3fa11f2 Markdown

All onearchive docs

RecursiveAI.OneCompress.Brotli: decoding of brotli data as RFC 7932 defines it. This document records the choices the RFC leaves to the implementation. The format itself is the RFC; section numbers below are its.

Names

Brotli (one-shot), BrotliDecoder (span decoder with OperationStatus) and BrotliStream (read-only stream) keep this library's naming convention. They share simple names with the types in System.IO.Compression, but live in a different namespace; a file that uses both aliases one of them. The public tier-3 parts are BrotliPrefixCode (prefix-code tables), BrotliWords (the static dictionary) and BrotliTransforms (the 121 word transformations).

Static data

The dictionary (section 8 and Appendix A), the transformation list (Appendix B) and the context lookup tables (section 7.1) are generated by tools/brotli/generate-tables.py from the RFC text, identified by its SHA-256. The script verifies every CRC-32 the RFC states for them. The generated files carry the IETF Trust's licence notice (see THIRD-PARTY-NOTICES.md). They are reachable only from the brotli types, so an application that does not use brotli does not contain them after trimming.

Limits (BrotliDecompressionOptions)

  • MaxWindowSize, default 2^24 - 16 (the largest window the format allows). A stream whose window exceeds it throws CompressionLimitException after the first header byte(s). Memory for the output ring is bounded by the window and by the declared output of the meta-blocks seen so far; the ring grows by doubling until it first wraps.
  • MaxOutputSize, default 1 GiB, checked against each meta-block's declared length before any of it is produced.
  • AllowTrailingData, default false: bytes after the last meta-block are an error naming their offset.

Choices

  • Large-window brotli (window-bit pattern 0010001, written by the tool's --large_window) is not part of RFC 7932 and is rejected as an invalid window size.
  • Context maps: the RFC says every tree value must be used by the map; the decoder does not check this, since an unused tree changes nothing a decoder can observe.
  • Errors: malformed data throws CompressionFormatException (format brotli, byte offset, what was expected; IsTruncated when more input would have been needed). The span decoder never returns InvalidData. After a throw the decoder must be Reset.
  • Streaming: header fields are decoded atomically; when input ends inside one, the decoder keeps the unfinished bytes and resumes when more arrive, so input and output may be cut anywhere. Output is never withheld: a call writes as much as fits in the destination and returns DestinationTooSmall when more is waiting.
  • Metadata meta-blocks are skipped. The bytes after the last meta-block's final bit up to the byte boundary must be zero.