Model container, version 1

Status: current. Version 1 is version 0 (model-container-v0.md) with format version 1 and one added manifest member, calibration. Writers produce version 1; readers read both. A version 0 file loads and has no calibration: its calibrated word confidence is its raw confidence, and the reader reports a diagnostic saying so. Everything else in version 0 stands and is repeated here so this document is complete.

The container is a cross-language contract: one weights file serves every implementation. Everything below is little-endian.

File layout

A reader rejects the file — before allocating anything sized by it — when the magic or version is wrong, the manifest is not valid UTF-8 JSON, or any length, offset, shape or count exceeds the reader's limits or the file's size. Tensors may not overlap. All padding bytes (after the manifest and between tensors) must be zero. The file ends exactly where the last tensor ends: no trailing padding, and bytes after the last tensor are an error. Tensors need not be listed in offset order.

The manifest is strict JSON: no comments, no trailing commas, no duplicate member names. Members this version does not name are ignored (provenance among them). In a version 0 file calibration is such a member and is ignored, whatever it holds. Sizes, offsets and shapes are JSON integers.

Manifest

{
  "architecture": "crnn-v0",
  "name": "line-recognizer",
  "version": "0.1.0",
  "license": "MIT OR Apache-2.0",
  "input": { "height": 32, "channels": 1, "mean": 0.5, "std": 0.5, "invert": false },
  "charset": ["<blank>", " ", "!", "..."],
  "config": { },
  "tensors": [
    { "name": "conv.0.weight", "dtype": "f16", "shape": [64, 1, 3, 3], "offset": 0, "length": 1152 }
  ],
  "calibration": {
    "raw": "word-mean-max-run",
    "scripts": {
      "Latin": { "x": [0.0, 0.5, 1.0], "y": [0.0, 0.4, 0.95] },
      "Greek": { "x": [0.0, 1.0], "y": [0.0, 1.0] }
    }
  },
  "provenance": { }
}

• architecture names the layer graph (below); a reader rejects any it doesn't know.
• input: line crops are grayscale, scaled to height keeping aspect ratio; a pixel p in 0–255 becomes x = p / 255 (1 - p / 255 when invert), then (x - mean) / std, all in float32 with a correctly rounded division. channels is 1 for crnn-v0; std is positive.
• charset: the output classes in order. Index 0 is the CTC blank; its text is a placeholder (the exporter writes <blank>) and is never output. Each other entry is a non-empty NFC string, usually one code point; entries, the blank placeholder included, are unique. (Readers do not check NFC; they check emptiness, uniqueness and length.)
• tensors: dtype is f16 (IEEE 754 binary16) in versions 0 and 1; values are widened to float32 for computation. offset is relative to D; length equals the element count times 2. Tensors are stored row-major in PyTorch's own dimension order.
• provenance: free-form record of the training run (commit, config, seed, data manifest hashes). Readers ignore it.

Architecture crnn-v0

A convolutional stack, then a bidirectional LSTM, then a per-timestep classifier, decoded by greedy CTC. config:

{
  "conv": [
    { "out": 64, "kernel": [3, 3], "padding": [1, 1], "pool": [2, 2] },
    { "out": 128, "kernel": [3, 3], "padding": [1, 1], "pool": null }
  ],
  "lstm": { "hidden": 256, "layers": 2 },
  "classes": 700
}

• config.conv[i]: out channels, kernel, padding and pool (an array of two integers [h, w], or null, or absent for no pooling). Kernel and pool sizes are at least 1, padding at least 0. config.classes equals the charset length.
• Conv block i (conv.<i>): 2-D convolution, stride 1, the given kernel [h, w] and zero padding [h, w], with bias; batch normalization is folded into the weight and bias at export. Then ReLU. Then, when pool is [ph, pw], max pooling with kernel and stride [ph, pw] and no padding (a trailing partial window is dropped; the block's input has in channels, the previous block's out, or input.channels for the first). A line image too narrow to leave at least one column after every block reads as empty text. Tensors: conv.<i>.weight [out, in, kh, kw], conv.<i>.bias [out].
• To sequence: the stack's output height must be exactly 1 for the configured input height (a reader checks this). Shape [C, 1, W] becomes a sequence of W vectors of size C, left to right.
• BiLSTM (lstm): layers stacked bidirectional layers; each layer's input is the previous layer's forward and reverse outputs concatenated (forward first). Gate order and equations are PyTorch's nn.LSTM (input, forget, cell, output; c' = f*c + i*g, h' = o*tanh(c')), zero initial state. Tensors use PyTorch's names under lstm.: lstm.weight_ih_l<k>, lstm.weight_hh_l<k>, lstm.bias_ih_l<k>, lstm.bias_hh_l<k>, and the same with suffix _reverse.
• Classifier (fc): linear layer 2 * hidden → classes with bias: fc.weight [classes, 2 * hidden], fc.bias [classes]. classes equals the charset length. A reader ignores tensors it does not need.
• Decoding: greedy CTC: take the arg-max class per timestep (lowest index on ties), merge repeats, drop blanks, concatenate charset entries. Confidences are defined under Confidence below.

Architecture dbnet-lite-v0

The page detector: text lines, layout regions and page orientation from one page image. The layer graph, pre- and post-processing are in detector-v0.md; this section is what the container carries.

• architecture: dbnet-lite-v0; name: text-detector-layout.
• input: {"height": 0, "channels": 1, "mean": 0.5, "std": 0.5, "invert": false}. Height 0 means "any multiple of 32" and is allowed for this architecture only; every other architecture needs a positive height. channels is 1.
• charset: ["<blank>", ...layout class names], present only to satisfy the envelope's charset rules; it is not an output alphabet and a reader ignores it for this architecture.
• calibration: the usual object with an empty scripts (the detector has no word confidences).
• config:

{
  "arch": "dbnet-lite", "stem": 16, "widths": [32, 72, 144, 224], "depths": [2, 3, 4, 3], "fpn": 56,
  "text": 32, "layout": 112, "layout_blocks": 3, "classes": 8,
  "layout_classes": ["text", "title", "list", "table", "figure", "header_footer", "form_field", "checkbox"],
  "layout_trained": ["text", "title", "list", "table", "figure", "header_footer"],
  "postprocess": {"text_threshold": 0.3, "text_expand": 1.0, "layout_threshold": 0.6, "layout_expand": 0.85}
}

arch must be dbnet-lite; widths and depths have four entries each; every width is at least 1 and within the reader's layer-width limit, every depth at least 1; classes equals the number of layout_classes, whose names are non-empty and unique. layout_trained is optional: the names of the layout classes the model was trained on. A reader never reports a class that is not listed. When the member is absent, every class except form_field and checkbox counts as trained (those two had no training labels in the first model). A name that is not in layout_classes is an error. postprocess is optional, and so is each of its members: text_threshold and layout_threshold are probabilities strictly between 0 and 1, compared as float32 logits; text_expand and layout_expand are factors in (0, 4] on the growth of a text line or layout core (detector-v0.md, "Post-processing"). Absent members take 0.3, 1.0, 0.6 and 0.85. A value out of range is an error.

• Tensors (all f16, PyTorch layout, batch normalization folded in; names in detector-v0.md): stem, stage<i>.<j>.dw/.pw, lat<i>, smooth8.dw/.pw, smooth4.dw/.pw, layout.<j>.dw/.pw, layout_out, text4.dw/.pw, stem_proj, text2.dw/.pw, prob_out (each with .weight and .bias), and orient.weight [4, widths[3]], orient.bias [4]. A depthwise weight is [c, 1, 3, 3], a 1x1 weight [out, in, 1, 1]. Shapes are checked against config.

Architecture dbnet-lite-v1

The page detector with the optional stride-16 layout branch and two-scale orientation head (detector-v1.md). The envelope, input, charset, calibration, layout_trained, postprocess and every tensor of dbnet-lite-v0 are as in the section above, with architecture dbnet-lite-v1. config gains three members, all optional:

• layout16_blocks: integer, 0 (the default) or the number of stride-16 blocks;
• layout16: the width of those blocks, required and positive when layout16_blocks is above 0;
• orient16: boolean, false by default; when true orient.weight is [4, widths[2] + widths[3]].

With the branch the file also carries layout16.<j>.dw/.pw (j < layout16_blocks) and layout_mix (.weight [layout, layout16, 1, 1], .bias). The three members in a dbnet-lite-v0 file are an error. Height 0 in input is allowed for both detector architectures.

Architecture docextract-v0

The document-extraction model: a small layout-aware transformer over the words of a page. It reads words (text and box), not pixels. The graph, the featurization and the assembly of labels into fields are in docextract-v0.md; this section is what the container carries.

• architecture: docextract-v0; name: document-extraction.
• input has no image members. It is {"kind": "words", "max_words": 512, "max_chars": 32, "bins": 128, "features": 8}: the most words in one window (MaxWordsPerWindow bounds it), the characters kept per word (MaxCharsPerWord), the cells per axis of the position grid (MaxPositionBins) and the numeric features per word (always 8). Every other architecture needs the image members (height, channels, mean, std, invert) and no kind.
• charset: the alphabet of the character embedding, charset-v1 (entry 0 <blank> is padding and never a text character). The embedding table has one more row than the charset: the last row is the unknown character.
• calibration: {"raw": "word-mean-max-run", "scripts": {}}. The model has no calibrated confidences; field confidence is the mean softmax probability of the chosen tags.
• config, all integers unless stated: d, layers, heads, ffn, char_dim, char_conv, pair_hidden, rel_dim, max_words (equal to input.max_words), classes, tags (1 + 2 * classes), roles (5), doc_types, unk_char (equal to the charset length); and the string lists classes_list (classes unique, non-empty names), roles_list (O, B-KEY, I-KEY, B-VAL, I-VAL) and doc_types_list. d is a multiple of heads. Widths are bounded by MaxLayerWidth, layers by MaxTransformerLayers, the three list lengths by MaxExtractionClasses, and (layers * heads + 2) * max_words^2 by MaxActivationElements, all before any tensor is read.
• Tensors (names and shapes exactly; a reader checks each against the configuration before reading it): char_emb.weight [charset + 1, char_dim], char_conv.weight [char_conv, char_dim, 3], char_conv.bias, char_proj.{weight [d, char_conv], bias}, pos_x0 / pos_y0 / pos_x1 / pos_y1 .weight [bins, d], feat_proj.{weight [d, 8], bias}, emb_ln.{weight, bias} [d], per block blocks.<l>.{ln1, ln2}.{weight, bias} [d], blocks.<l>.qkv.{weight [3d, d], bias}, blocks.<l>.proj.{weight [d, d], bias}, blocks.<l>.fc1.{weight [ffn, d], bias}, blocks.<l>.fc2.{weight [d, ffn], bias}, pair1.{weight [pair_hidden, 13], bias}, pair2.{weight [layers * heads + 2, pair_hidden], bias}, final_ln.{weight, bias}, tag_head.{weight [tags, d], bias}, role_head.{weight [5, d], bias}, doc_head.{weight [doc_types, d], bias}, rel_q.{weight [2 * rel_dim, d], bias}, rel_k.{weight [2 * rel_dim, d], bias}.
• A model trained for another schema (a fine-tuned head) is the same architecture with its own classes_list and tag_head.

Calibration (version 1)

A recognizer says how sure it is per character; users want the probability that a word is exactly right. The model file carries the map from one to the other, fitted by the trainer on a held-out calibration split.

Character probability. The network's output at a step is softmax over the classes. A character is a run: a maximal sequence of consecutive steps whose arg-max is the same non-blank class (a blank step ends a run, so "ll" is two runs). Its probability is the largest softmax probability of that class over the run's steps.

Raw word confidence ("raw": "word-mean-max-run", the only value defined; any other is an error): the words of a line are the maximal sequences of runs whose characters are not whitespace. A character is whitespace when its charset entry is non-empty and consists only of whitespace code units (char.IsWhiteSpace-style: U+0020, U+00A0, U+2003 and the other Unicode White_Space characters). Spaces are not part of any word; a line with no non-space run has no words. The raw confidence of a word is the mean of its runs' probabilities, summed left to right in float32 and divided by the number of runs. A whole line's raw confidence (RecognizedLine.Confidence) uses the same rule over every run of the line, spaces included; it is 0 for an empty line.

Script of a word. Count the word's letters (Unicode categories L*, by code point) in three groups: Greek (U+0370–03FF, U+1F00–1FFF), Cyrillic (U+0400–052F, U+1C80–1C8F, U+2DE0–2DFF, U+A640–A69F), Latin (every other letter). The script with the most letters wins; ties go to the first of Latin, Greek, Cyrillic. A word with no letters (digits, punctuation, symbols) is Latin.

Calibrated word confidence. The manifest member:

"calibration": { "raw": "word-mean-max-run", "scripts": { "Latin": { "x": [...], "y": [...] }, "Greek": {...}, "Cyrillic": {...} } }

• scripts is an object whose names are exactly Latin, Greek or Cyrillic (case-sensitive, no duplicates). Any other name is an error. A script may be missing; its calibrated confidence is then its raw confidence. scripts may be empty.

• Each script has x and y, JSON arrays of numbers of equal length, between 1 and 64 knots (more is a limit error, MaxCalibrationKnots).

• x is strictly increasing and every value is in [0, 1]; y is non-decreasing and every value is in [0, 1]. Numbers are IEEE doubles.

• A version 1 file must have the calibration object; the reader rejects a file without it.

• The map is piecewise linear through the knots (x[i], y[i]) and constant outside them: for r <= x[0] the value is y[0], for r >= x[last] it is y[last]. For x[i] < r < x[i+1], in IEEE double precision with each operation rounded on its own (no fused multiply-add):

  t = (r - x[i]) / (x[i+1] - x[i]), p = (y[i+1] - y[i]) * t, value = y[i] + p

  and the result is converted to float32. A one-knot curve is the constant y[0]. r is the float32 raw confidence converted to double.

Reporting curve (optional). A version 1 recognizer file may also carry report_calibration, an object of exactly the same shape and rules as calibration (same raw definition, scripts, knot limits and evaluation). Where it is present, the confidence a finished page reports for a word is report_calibration applied to the word's raw confidence (script taken from the word's final text); the word's rawConfidence is unchanged. Every decision the reader makes while building the page (noise and speck filtering, word joining and splitting, the page-angle search, table tests, reading order) keeps using calibration, so the text, boxes and structure of the page are the same with or without the member. The map is applied after the last of those decisions and nothing reads its result. Line and block confidences are recomputed from the reported word confidences by the rule below. A file without the member reports calibration as before; a version 0 file ignores the member like any member that version does not name. A malformed member is an error exactly as a malformed calibration is, and JSON null means absent. The reason for a second curve: a curve fitted to make the reported number predict correctness on a kind of page (receipt photos, say) differs from the one the filters were tuned against, and swapping it would change the text.

Line confidence in the result type is the calibrated confidences of the line's words averaged with each word weighted by its number of runs (double precision, converted to float32); 0 for a line with no words. This is a reader convention for reporting, not part of the file.

Numerics

Weights are stored in fp16 and computed in fp32. Implementations agree with the PyTorch reference within a documented per-layer tolerance; bit-exact agreement is only required between the C# library's own code paths. The order of floating-point operations that makes those paths bit-identical is written down in numerics.md.