# oneocr docs, all pages (OneOcr 1.0.0) Every page of https://recursiveai.org/oneocr/docs/ and https://recursiveai.org/oneocr/cookbook/ as markdown, getting started first. Index: https://recursiveai.org/oneocr/llms.txt # Getting started with OneOcr Install OneOcr for .NET 10, read the text of an invoice image, walk the page model with its boxes and confidences, turn the page into Markdown, and set a default engine. No native binaries, no `tessdata`, no network. ## Install OneOcr targets `net10.0`, so you need the .NET 10 SDK. Every package shares one version; this guide uses 1.0.0. ```shell dotnet new console -o OneOcrStart cd OneOcrStart dotnet add package RecursiveAI.OneOcr --version 1.0.0 ``` `RecursiveAI.OneOcr` brings in `RecursiveAI.OneOcr.Models`, which embeds the recognizer, the page detector and the two word models and checks them against their SHA-256 pins on every load. There is nothing to download and no model path to set. Pixels come from [OneImage](https://recursiveai.org/oneimage/), so any `OneImage.Image` (PNG, JPEG, WebP, TIFF and so on) is an input. The same weights are also attached to the [v1.0.0 release on GitHub](https://github.com/recursive-ai-oss/oneocr-csharp/releases/tag/v1.0.0) for `OcrEngine.Load(path)` and `.WithDetector(path)`. ## The sample images The guide and the [cookbook](https://recursiveai.org/oneocr/cookbook/) read two images. `invoice.png` is a one-page A4 invoice rendered from HTML at 300 DPI (2481 × 3509 pixels), with a seller and buyer block, a table of items and totals. `receipt.png` is a till receipt rendered at twice screen resolution (920 × 940). Any PNG or JPEG of a flat document works the same way; the outputs below are what 1.0.0 printed for these two. ## Read the text `OcrEngine.LoadBundled()` loads the bundled models. An engine is immutable and thread-safe: load it once and share it. `ToText` returns the page in reading order, with lines separated by a newline and table cells by a tab: ```csharp using OneImage; using OneOcr; OcrEngine engine = OcrEngine.LoadBundled(); // once at startup; immutable and thread-safe Image image = Image.Load("invoice.png"); Console.WriteLine(image.ToText(engine)); ``` ```text Northwind Supply Ltd 14 Harbour Street BS1 4DJ Bristol United Kingdom VAT ID: GB284193752 INVOICE Bill to Blue Heron Bakery 88 Elm Road OX1 2JD Oxford United Kingdom Invoice no. INV-2041 Date Due date Order no. 14/10/2026 13/11/2026 PO-7731 Description Qty Unit price VAT Amount Flour, 25 kg bag 4 pcs 31.50 0% 126.00 Cane sugar, 10 kg 2 pcs 18.25 0% 36.50 Baking parchment roll 6 pcs 7.40 20% 44.40 Payment terms: Payable within 30 days Subtotal VAT total Total £206.90 £8.88 £215.78 IBAN: GB33 BUKB 2020 1555 5555 55 BIC: BUKBGB22 Tel. +44 117 496 0321 ``` The page detector found the table and the blocks; reading order goes block by block, so the labels of the meta grid and the totals come before their values. ## The page model `Ocr` runs the same pipeline and returns the whole `OcrPage`: blocks, lines, words and characters with boxes, tables, key–value pairs and diagnostics. Coordinates are pixels from the top left, y down. A word’s `Confidence` is the calibrated probability that it is exactly right: ```csharp using OneImage; using OneOcr; OcrEngine engine = OcrEngine.LoadBundled(); OcrPage page = Image.Load("invoice.png").Ocr(engine); Console.WriteLine($"{page.Width} x {page.Height}, {page.Blocks.Count} blocks, {page.Words.Count} words, {page.Tables.Count} table"); foreach (OcrWord word in page.Blocks[0].Lines[0].Words) Console.WriteLine($"{word.Text,-10} {word.Confidence:F3} ({word.Box.Left:F0}, {word.Box.Top:F0}) to ({word.Box.Right:F0}, {word.Box.Bottom:F0})"); foreach (OcrDiagnostic diagnostic in page.Diagnostics) Console.WriteLine($"{diagnostic.Code}: {diagnostic.Message}"); ``` ```text 2481 x 3509, 19 blocks, 98 words, 1 table Northwind 0.986 (231, 198) to (580, 247) Supply 1.000 (614, 198) to (845, 260) Ltd 0.974 (878, 200) to (963, 247) OCR101: UnitsPerInch is estimated at 200 from the size of the text; pass OcrOptions.Dpi to set it. OCR107: 1 word(s) read from specks, grain or stray marks were dropped as noise; pass OcrOptions.DropNoise = false to keep them. ``` Diagnostics say what the reader decided on its own. This PNG doesn’t declare a resolution, so `OCR101` says it was estimated; pass `new OcrOptions { Dpi = 300 }` when you know it. It only sets `UnitsPerInch`: recognition doesn’t depend on it. ## Markdown `ToMarkdown` builds Markdown from the page model with no model call: headings from text size, lists, and pipe tables from detected tables, with headers, footers and page numbers dropped. ```csharp using OneImage; using OneOcr; OcrEngine engine = OcrEngine.LoadBundled(); Console.WriteLine(Image.Load("invoice.png").ToMarkdown(engine)); ``` ```text Northwind Supply Ltd 14 Harbour Street BS1 4DJ Bristol United Kingdom VAT ID: GB284193752 # INVOICE ## Bill to Blue Heron Bakery 88 Elm Road OX1 2JD Oxford United Kingdom Invoice no. INV-2041 Date Due date Order no. 14/10/2026 13/11/2026 PO-7731 | Description | Qty | Unit price | VAT | Amount | | --- | --- | --- | --- | --- | | Flour, 25 kg bag | 4 pcs | 31.50 | 0% | 126.00 | | Cane sugar, 10 kg | 2 pcs | 18.25 | 0% | 36.50 | | Baking parchment roll | 6 pcs | 7.40 | 20% | 44.40 | Payment terms: Payable within 30 days Subtotal VAT total Total £206.90 £8.88 £215.78 IBAN: GB33 BUKB 2020 1555 5555 55 BIC: BUKBGB22 Tel. +44 117 496 0321 ``` ## A default engine If you’d rather not pass the engine around, set a process-wide default once. The extension methods without an engine argument then use it; until you call `ConfigureDefault`, `OcrEngine.Default` throws `InvalidOperationException`. ```csharp using OneImage; using OneOcr; OcrEngine.ConfigureDefault(); // the bundled models become OcrEngine.Default Console.WriteLine($"Kernels: {OcrEngine.Default.KernelPath}"); string text = Image.Load("receipt.png").ToText(); // no engine argument: the default is used Console.WriteLine(text); ``` ```text Kernels: Avx512 Corner Market 1402 Oak Street 78704 Austin +1 512 555 0142 Date: 24/10/2026 Time: 14:32 Coffee beans 12.99 2x Oat milk 8.98 Bagels 5.25 Subtotal 27.22 Tax 8.25 % 2.07 TOTAL $ 29.29 Paid by Card **** 4821 Thank you! ``` `KernelPath` says which kernels run: `Avx512`, `Avx2`, `Vector128` or `Scalar`. The text, boxes and confidences are the same on all four ([Same bytes on every machine](https://recursiveai.org/oneocr/cookbook/same-bytes-everywhere/)). ## Good to know - Every local call takes a `CancellationToken` and has an `Async` twin (`OcrAsync`, `ToTextAsync`, `ToMarkdownAsync`) that returns the same result. - `Ocr(Image)` reads one page and throws `ArgumentException` for an image with several frames; `OcrPages` reads every frame of a TIFF. - Errors are typed: `OcrFormatException` for bytes that aren’t what they claim to be, `OcrLimitException` when an `OcrLimits` bound is crossed (checked before anything large is allocated), `OcrModelException` for a bad model file. - Scripts: Latin (including the EU languages, Turkish and Vietnamese), Greek and Cyrillic in one recognizer. CJK, Arabic and Indic scripts, handwriting and scene text are not supported yet. ## Next - [Invoices as fields](https://recursiveai.org/oneocr/cookbook/invoices/): typed fields with EN 16931 ids, each with the words and box it was read from. - [Words to check](https://recursiveai.org/oneocr/cookbook/words-to-check/): use calibrated confidence to find what a person should look at. - [PDF pages](https://recursiveai.org/oneocr/cookbook/pdf-pages/): text-layer words and OCR of scanned regions in one page model. - [Moving from Tesseract](https://recursiveai.org/oneocr/docs/migrating-from-tesseract/): commands, modes and outputs side by side. - [Overview and benchmarks](https://recursiveai.org/oneocr/docs/overview/#benchmarks): the README, with results against Tesseract, PaddleOCR, docTR and Apple Vision. # OneOcr **Local OCR and document understanding for .NET: a pure-managed, deterministic reader with small models of our own, a page model with boxes and calibrated confidences, invoices and receipts as fields, and optional LLM vision that is checked against the page.** Pure C# with no dependency beyond .NET itself (and the OneImage / OnePDF libraries for pixels and PDFs). No native binaries, no `tessdata`, no network unless you configure a provider. [![.NET 10](https://img.shields.io/badge/.NET-10.0-512BD4)](#install) [![License](https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue)](#license) Results against Tesseract, PaddleOCR, docTR and Apple Vision, per category, are in [Benchmarks](#benchmarks). Every number there comes from one run of the release gate on one machine; the method is written next to the tables so you can repeat it. ## Why OneOcr - **Free, fast, local.** The recognizer (5 MB), the page detector (1.4 MB) and the two word models (1.4 MB and 1.3 MB) ship inside the `RecursiveAI.OneOcr.Models` package and run on the CPU through our own SIMD kernels. Scripts: Latin (including the EU languages, Turkish and Vietnamese), Greek and Cyrillic in one model. - **A page, not a string.** The result is an `OcrPage`: blocks, lines, words and characters with boxes (and quads for rotated text), tables with row and column spans, key–value pairs, reading order, and a `Source` on every element. It serializes to a documented, versioned JSON format, `oneocr.page`, that shares its member names with [OnePDF's `onepdf.extraction`](https://www.nuget.org/packages/RecursiveAI.OnePdf) ([spec](https://recursiveai.org/oneocr/docs/page-json/)). - **Confidence you can use.** A word's `Confidence` is the calibrated probability that the word is exactly right, calibrated per script on held-out data. The release gate requires an expected calibration error of at most 0.05 in every category. - **Same bytes everywhere.** Text, boxes and confidences are bit-identical on AVX-512, AVX2, 128-bit vectors (SSE and NEON) and the scalar path, and at every degree of parallelism. The rules are written down in [docs/spec/numerics.md](https://recursiveai.org/oneocr/docs/numerics/). - **Documents as fields.** `ExtractInvoice()` and `ExtractReceipt()` return typed fields, each with the words and boxes it was read from; a small local model labels words on the page, so it cannot invent a value. Invoice fields carry their EN 16931 business-term ids. - **LLM vision that is held to the page.** Opt-in providers (Anthropic, OpenAI, OpenAI-compatible servers, Gemini, Azure OpenAI) over plain `HttpClient`. Every value the model returns is checked against the words on the page and comes back `Grounded`, `GroundedNormalized`, `Derived` or `Ungrounded`, with boxes where it has them. - **Train it on your documents.** Fine-tune the recognizer or a new extraction head from your own scans, on the CPU, deterministically and resumably, with a review queue for the lines a model and the reader disagree on. - **Safe on files you did not write.** Model files, images, page JSON, PDFs, review files and LLM responses are read by bounds-checked managed code under limits checked before anything large is allocated, and each of those readers has a fuzz target. The only `unsafe` code is in the SIMD kernels, which run on buffers that safe callers validated. NativeAOT and trimming are supported. ## Install ```sh dotnet add package RecursiveAI.OneOcr ``` `RecursiveAI.OneOcr` pulls in `RecursiveAI.OneOcr.Models` (the weights), so one install gives you the one-liner. The rest is opt-in: | Package | What it adds | |---|---| | `RecursiveAI.OneOcr` | Local OCR, the page model and `oneocr.page` JSON, local Markdown, invoice / receipt / custom extraction (inference). No networking code. | | `RecursiveAI.OneOcr.Models` | The recognizer, page detector and word model, embedded and checked against their SHA-256 pins on every load. | | `RecursiveAI.OneOcr.Models.Documents` | The invoice and receipt extraction model. | | `RecursiveAI.OneOcr.Llm` | LLM vision: providers, modes, grounding, cost controls, routing. The only package that uses the network. | | `RecursiveAI.OneOcr.Training` | `OcrTrainer`, `ExtractionTrainer` and the review queue. | | `RecursiveAI.OneOcr.Pdf` | PDF pages: text layer plus OCR of image regions. | | `RecursiveAI.OneOcr.Tool` | The `oneocr` command-line tool: `ocr`, `train`, `eval`, `review`. | ```sh dotnet add package RecursiveAI.OneOcr.Llm dotnet add package RecursiveAI.OneOcr.Pdf dotnet add package RecursiveAI.OneOcr.Models.Documents dotnet add package RecursiveAI.OneOcr.Training dotnet tool install --global RecursiveAI.OneOcr.Tool ``` Target framework `net10.0`. Namespaces: `OneOcr` (core), `OneOcr.Documents`, `OneOcr.Decoding`, `OneOcr.Models.Documents`, `OneOcr.Llm`, `OneOcr.Pdf`, `OneOcr.Training`. Pixels come from [`RecursiveAI.OneImage`](https://www.nuget.org/packages/RecursiveAI.OneImage), so any `OneImage.Image` (PNG, JPEG, WebP, TIFF and so on) is an input. Moving from Tesseract? See the [migration guide](https://recursiveai.org/oneocr/docs/migrating-from-tesseract/). ## Quick start ```csharp // Once at startup: the recognizer, the page detector and the word model that ship in RecursiveAI.OneOcr.Models. OcrEngine engine = OcrEngine.LoadBundled(); Image image = Image.Load("invoice.png"); string text = image.ToText(engine); // plain text in reading order OcrPage page = image.Ocr(engine); // the page model: blocks, lines, words, characters, tables string markdown = image.ToMarkdown(engine); // headings, lists and pipe tables ``` An `OcrEngine` is immutable and thread-safe; load it once and share it. If you would rather not pass it around, set a process-wide default once (`OcrEngine.ConfigureDefault`) and use the extension methods without an engine. Until you do, `OcrEngine.Default` throws `InvalidOperationException`. ```csharp OcrEngine.ConfigureDefault(); // once; the bundled models become OcrEngine.Default string text = Image.Load("page.png").ToText(); // the extension methods without an engine use the default OcrPage page = Image.Load("page.png").Ocr(); ``` From the command line (`dotnet tool install --global RecursiveAI.OneOcr.Tool`): ```sh oneocr ocr --in page.png # text to stdout oneocr ocr --in scans/ --out text/ --json # one .txt and one .page.json per image ``` `ocr` reads PNG and JPEG with the bundled models and never touches the network; `--degree N` caps the threads (the text does not depend on it). The same tool trains models, see [Training](#training). ## Reading a page: `OcrEngine` and options `ToText` and `Ocr` run the same pipeline; `ToText` returns the page's text and `Ocr` returns the whole page model. The pipeline finds the text lines with the page detector, turns the page upright if it came in rotated by 90, 180 or 270 degrees, straightens tilted lines, reads every line, decides word breaks from the line's pixels as well as from the network, drops specks that were read as text, works out reading order across columns and sidebars, and finds tables. An engine carries its options (`engine.Options`; `engine.WithOptions(...)` returns a copy with other ones). Every call also takes an `OcrOptions?`; `null` means the engine's, and a value replaces the engine's options as a whole (nothing is merged). An option out of range throws `ArgumentOutOfRangeException` naming `options`. ```csharp OcrPage page = image.Ocr(engine, new OcrOptions { Dpi = 300, // when the file does not say; otherwise estimated, with a diagnostic MaxDegreeOfParallelism = 2, // 0 = min(4, cores); the result does not depend on it IncludeChars = false, // skip per-character boxes DetectOrientation = true, // 0/90/180/270 (the default) Deskew = true, // tilts up to about 15 degrees (the default) }); foreach (OcrBlock block in page.Blocks) { foreach (OcrLine line in block.Lines) { foreach (OcrWord word in line.Words) { Console.WriteLine($"{word.Text} {word.Confidence:F3} ({word.Box.Left:F0}, {word.Box.Top:F0})"); } } } ``` Other `OcrOptions` switches are `SearchAngle` (re-reads a badly tilted photo at other page angles when the first reading is weak), `DropNoise`, `FixWordSpacing`, `FixDigitLookalikes` (reads the letter `O` as `0` inside identifiers and dates) and `FixSeparators` (reads a middle dot, bullet or vertical bar between words from its pixels when the network gave it as a hyphen or another small mark). These two are the only steps after recognition that change a character. All are on by default and documented on the type. - **Errors.** Public methods throw only the typed exceptions (all derive from `OcrException`), argument exceptions, I/O exceptions and `OperationCanceledException`: `OcrFormatException` (bytes that are not what they claim to be), `OcrLimitException`, `OcrModelException`, `OcrProviderException` and `OcrBudgetException` (in the LLM package). Where something is recoverable the page says so in `page.Diagnostics` instead of throwing. - **Threading and async.** Every local call takes a `CancellationToken`, checked between pages and between lines, and throws `OperationCanceledException`. Every local call also has an `Async` twin (`OcrAsync`, `ToTextAsync`, `ToMarkdownAsync`, `ExtractInvoiceAsync`, `ExtractReceiptAsync`, `ExtractAsync`, and the same on PDF pages and documents). A twin runs the very same code on a thread-pool thread, so its result is byte-identical to the sync call, and a cancelled token ends the task as cancelled. Keep the `Image` alive until the task completes. `engine.Ocr(images)` and `engine.OcrAsync(images)` read several images in order, with consecutive `OcrPage.Index` values. The LLM package is async only. - **Multi-page files.** `Ocr(Image)` reads one page and throws `ArgumentException` for an image with more than one frame, because silently reading the first frame of a TIFF loses pages. `OcrPages` reads every frame (`engine.OcrPages(path | stream | bytes | image)`, also as an extension on `Image`), and `ToText` and `ToMarkdown` on an image read all frames and join them with a blank line. The number of frames counts against `OcrLimits.MaxPagesPerCall`. - **Untrusted images.** A file OneImage refuses surfaces as `OcrFormatException`, and a file over `OcrLimits.MaxImagePixels` (or too many frames) as `OcrLimitException`; the original exception is the inner one. The pixel and frame limits are applied before the pixels are decoded when you pass a path, a stream or bytes to `OcrPages`. - **Resolution.** `OcrPages(path | stream | bytes)` takes the resolution a PNG (`pHYs`), JPEG (JFIF) or TIFF (`XResolution`) declares as `UnitsPerInch`. On an `Image` you loaded yourself the resolution is estimated from the size of the text, and a diagnostic says so; set `OcrOptions.Dpi` when you know it. ```csharp using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30)); OcrPage page = await image.OcrAsync(engine, new OcrOptions { Deskew = false }, cts.Token); string text = await image.ToTextAsync(engine, null, cts.Token); IReadOnlyList frames = await engine.OcrAsync([image, image], cancellationToken: cts.Token); ``` - **Decoding.** Page reading (`Ocr`, `ToMarkdown`) is always greedy CTC, because the word boxes come from the greedy alignment. `engine.WithDecoding(...)` sets the decoding the plain-text call `engine.ToText(image, engine.Decoding)` uses: beam search with a character language model or a lexicon. ```csharp OcrEngine engine = OcrEngine.Load(modelPath).WithDecoding(new DecodeOptions { Mode = CtcDecoding.Beam, BeamWidth = 8, LanguageModel = CharLanguageModel.Load(lmPath), }); ``` Words that contain digits or code symbols are read exactly as greedy decoding reads them, so the language model never "corrects" an identifier. There is no word dictionary by default. - **Your own model.** `OcrEngine.Load(path)` reads a model container (for example one you fine-tuned) and `.WithDetector(path)` adds a detector; without one the classical line finder runs. ## The page model and `oneocr.page` JSON `OcrPage → OcrBlock → OcrLine → OcrWord → OcrChar`, plus `Regions` (roles such as title, list, table or figure, where the detector reports them), `Tables` (cells with row and column spans that point at words), `KeyValues` and `Diagnostics`. | Member | Meaning | |---|---| | Coordinates | Top-left origin, y down, in page units: pixels for images, points for PDF pages. `Width`, `Height`, `UnitsPerInch`; PDF pages also carry `ToPdf`, the matrix back to PDF user space. | | `Source` | `Ocr`, `TextLayer` or `Llm`, on every element. | | `Confidence` | Calibrated probability that the word is exactly right. `RawConfidence` is the uncalibrated mean of the characters' best softmax probabilities. Lines and blocks average their words'. | | Boxes and quads | Axis-aligned `OcrBox` everywhere; an `OcrQuad` where text was rotated or straightened. Character boxes from OCR are approximate (they come from the network's alignment) and say so. | | `Orientation`, `SkewDegrees` | How far the page content was turned clockwise from upright (0, 90, 180 or 270) and how far it was straightened before reading. All coordinates stay in the image as you gave it, also on a page that came in rotated: boxes cover the words where they are in your image, and rotated text carries a quad that follows the text. | | `Diagnostics` | What the reader decided on its own, as `OcrDiagnostic(Code, Severity, Message)`. Codes are stable (`OCR101` resolution estimated, `OCR102` resolution defaulted, `OCR103` page turned, `OCR104` page straightened, `OCR105` straightening skipped, `OCR106` angle searched, `OCR107` noise dropped, `OCR108` stray accents fixed, `OCR109` line skipped, `OCR110` no calibration, `OCR2xx` extraction, `OCR301` PDF, `OCR401` LLM). The JSON keeps the message only. | ```csharp string json = OcrPageJson.ToJson([page], new OcrPageJsonOptions { Indented = true }); IReadOnlyList again = OcrPageJson.Parse(json); // checked against OcrLimits; throws OcrFormatException on bad input ``` The JSON is deterministic (the same pages give the same bytes), versioned (`"format": "oneocr.page"`, `"version": 1`), and specified member by member in [docs/spec/oneocr-page-v1.md](https://recursiveai.org/oneocr/docs/page-json/), including how it differs from `onepdf.extraction` (y runs down here, boxes are `[left, top, right, bottom]`). Every element is a sealed record that compares by value. You can build a page yourself with object initializers (`new OcrWord { Text = ..., Box = ..., Confidence = ..., Source = ... }`); required members are enforced by the compiler. `page.ToMarkdown()` turns a page into Markdown without any model call: headings from text size, lists, pipe tables from detected tables, headers, footers and page numbers dropped (see `MarkdownOptions`). `page.ToText()` joins lines with a newline and table cells with a tab; it is what `image.ToText()` returns. `page.Text` (and the JSON `text` member) puts a blank line between blocks; `page.ToText(OcrTextLayout.Paragraphs)` is the same thing as a method. ## Invoices, receipts and your own document types A small layout-aware model tags the words of a page; deterministic code assembles the fields from the tagged words. Each `ExtractedField` has the exact text, a normalized value (an ISO date, a plain decimal amount, an ISO currency code), the word indices and the box. ```csharp DocumentModels.Configure(); // the extraction model from RecursiveAI.OneOcr.Models.Documents, once Invoice invoice = image.ExtractInvoice(engine); Console.WriteLine($"{invoice.Number?.Value} {invoice.IssueDate?.Value} {invoice.Total?.Value} {invoice.Currency?.Value}"); foreach (InvoiceLine line in invoice.Lines) { Console.WriteLine($"{line.Description?.Text} {line.Quantity?.Value} x {line.UnitPrice?.Value} = {line.Amount?.Value}"); } Receipt receipt = image.ExtractReceipt(engine); Console.WriteLine($"{receipt.Merchant?.Text} {receipt.Total?.Value}"); ``` - **Invoice** fields follow EN 16931 business terms (`BT-1` number, `BT-2` issue date, `BT-5` currency, `BT-27` seller name, `BT-44` buyer name, `BT-84` IBAN, `BT-106` subtotal, `BT-110` tax, `BT-112` total, `BT-115` amount due, and so on), plus line items (`BT-126` to `BT-153`) and the tax breakdown per rate. Look any field up with `invoice.Extraction.Get("BT-112")`. - **Receipt** fields: merchant, date, time, subtotal, tax, tip, total, payment method and items. - A field the model did not find is `null`, never a guess. `Confidence` on an extracted field is the mean probability of its words' tags; it is not a calibrated probability. - `page.FindKeyValues()` pairs keys with values on any page and `page.WithKeyValues()` returns the page with the pairs filled in. Extraction takes a `DocumentExtractionOptions` (`MaxDegreeOfParallelism`, `Strict`) and `DocumentExtractor.Default` is the process-wide extractor (`DocumentExtractor.ConfigureDefault()`). For your own type, give `Extract` a source-generated `JsonTypeInfo` (no reflection, so it works under NativeAOT) and a model you trained for it (see [Training](#training)). A property is filled from the model class with the same JSON name, so name properties after the classes (`[JsonPropertyName("BT-112")]` for the invoice total). A property with no class keeps its default and a diagnostic says so (`OcrDiagnosticCodes.PropertyUnmapped`); with `DocumentExtractionOptions.Strict` it throws, and it always throws when no property maps. `result.WordBoxes(name)` throws `KeyNotFoundException` for a name that is not a property: ```csharp public sealed record PurchaseOrder(string? Number, string? Supplier, decimal? Total); [JsonSerializable(typeof(PurchaseOrder))] internal sealed partial class OrderContext : JsonSerializerContext; ``` ```csharp DocumentExtractor extractor = DocumentExtractor.Load("purchase-orders.onm"); // a model you fine-tuned (see Training) ExtractionResult result = image.Extract(OrderContext.Default.PurchaseOrder, engine, extractor); Console.WriteLine($"{result.Value.Number} {result.Value.Total}"); IReadOnlyList where = result.WordBoxes("Total"); // where on the page the value was read ``` The model format and its inference are specified in [docs/spec/docextract-v0.md](https://recursiveai.org/oneocr/docs/docextract/). ## PDF `RecursiveAI.OneOcr.Pdf` reads each page as one `OcrPage`: text-layer words wherever the PDF has them (from [OnePDF](https://www.nuget.org/packages/RecursiveAI.OnePdf)), and OCR of image regions that the text does not cover, rendered at 300 DPI through [`RecursiveAI.OneImage.Pdf`](https://www.nuget.org/packages/RecursiveAI.OneImage.Pdf). Every word keeps its `Source`. ```csharp IReadOnlyList pages = PdfOcr.Read("contract.pdf", engine, new PdfOcrOptions { RenderDpi = 300 }); foreach (OcrPage page in pages) { Console.WriteLine($"page {page.Index + 1}: {page.Words.Count} words"); } string markdown = pages.ToMarkdown(); ``` Extension methods on OnePDF's `PdfPage` and `PdfDocument` (`Ocr`, `ToText`, `ToMarkdown`, `ExtractInvoice`, `ExtractReceipt`, each with an `Async` twin) do the same on a document you already hold. - A text layer is not trusted blindly. An invisible layer laid over a scan (a previous OCR pass) is re-read with our engine unless you set `TrustExistingOcrLayer`. Lines that look garbled (U+FFFD, private-use code points, control characters) are re-read from a render, and a cross-check of a few sampled lines drops the whole layer when it disagrees with what the page looks like. - Page coordinates are PDF points, y down; `ToPdf` maps them back to PDF user space. - Not yet: text drawn as outlined glyphs, annotations. ## LLM vision, held to the page `RecursiveAI.OneOcr.Llm` sends a page image and the local reader's word list (id, text, box) to a model and asks for text, Markdown, a page model or JSON for your schema in one call. Everything it returns is checked against the page. | Provider class | API | |---|---| | `AnthropicProvider` | Anthropic Messages | | `OpenAiProvider` | OpenAI | | `OpenAiCompatibleProvider` | Ollama, llama.cpp, vLLM, LM Studio and other OpenAI-style servers (a local model needs no key) | | `GeminiProvider` | Google Gemini | | `AzureOpenAiProvider` | Azure OpenAI (endpoint and deployment) | There is no default model: you name the model string. Providers take an injected `HttpClient` (a shared one that never follows redirects is used otherwise, so a key header cannot be sent to another host), write requests with `Utf8JsonWriter` and read responses with `System.Text.Json` under size and depth limits; no vendor SDKs, no reflection. Structured output uses each provider's own mechanism (OpenAI `json_schema`, a single forced tool call for Anthropic, `responseSchema` for Gemini), and the model is never offered tools to run. ```csharp OcrPage page = image.Ocr(engine); LlmProvider provider = AnthropicProvider.FromEnvironment("your-model-name"); // reads ANTHROPIC_API_KEY, because you called it using var llm = new LlmClient(provider, new LlmOptions { MaxCostPerCallUsd = 0.25m, SpendCapUsd = 5m }); LlmReadResult result = await llm.ReadAsync(image, page, new LlmReadOptions { Outputs = LlmOutputs.Text | LlmOutputs.Json, // one call produces every requested output JsonSchema = schemaJson, UngroundedPolicy = UngroundedPolicy.Flag, }); foreach (GroundedField field in result.Fields) { Console.WriteLine($"{field.Path} = {field.Value} [{field.State}, {field.Confidence:F2}] {field.Boxes.Count} box(es)"); } ``` **Outputs** (`LlmOutputs`, combine with `|`): `Text` (the model's transcription merged with the local reading word by word: the local word wins where it is confident, the model's where the reader is unsure, and each disagreement is listed), `Markdown` (the model's Markdown with the spans where it disagrees with the local reader marked), `Json` (values for your schema, grounded) and `PageModel` (the merged page, every word with its `Source`). `ExtractAsync` takes a source-generated `JsonTypeInfo` instead of a schema string, derives the schema from it and is grounded the same way: it returns `LlmReadResult` with the `Value` and the grounded `Fields`. Cost caps work with a local OpenAI-compatible server (host `localhost`, `127.x.x.x` or `[::1]`), which is priced at 0; a server on another host needs an entry in the price table (`LlmPriceTable.With`). **Grounding.** For every scalar the model returns it must cite the words it read. We check the cited words against the value after normalizing dates, amounts, currency, whitespace and case; a missing or wrong citation triggers a search of the page. Each value ends in one of five states: | `GroundingState` | Meaning | |---|---| | `Grounded` | The value is the text of words on the page. Boxes provided. | | `GroundedNormalized` | Equal to words on the page after normalization (for example "Oct 3" and `2026-10-03`). Boxes and `NormalizedValue` provided. | | `Derived` | The model says it computed or inferred the value. Never boxed; confidence is high only when a sum, difference or product of the cited amounts reproduces it. | | `Ungrounded` | Nothing on the page supports it: a possible hallucination. | | `RoleConflict` | The value is on the page, but a local extraction model reads those words as a different field. Counts like `Ungrounded`. | Ungrounded values are re-asked once in a second call that shows the mismatch (counted against your spend), then `UngroundedPolicy` applies: `Flag` (default: keep and mark), `Null` or `Throw`. The rules are in [docs/spec/grounding.md](https://recursiveai.org/oneocr/docs/grounding/). One risk grounding cannot remove, and the spec says so: a value that is printed on the page but sits in the wrong role can ground. The role check catches this when a local extraction model is available. **Cost controls.** An estimate before you call, per-call and per-client USD caps from a dated price table you can replace (a warning appears when it is older than 180 days), a concurrency limit, bounded retries with backoff, and an opt-in result cache keyed by image, prompt, model and schema. ```csharp LlmEstimate estimate = llm.Estimate([new LlmPageInput(image, page)], "Read the invoice.", schemaJson); Console.WriteLine($"about {estimate.InputTokens} input tokens, at most ${estimate.WorstCaseCostUsd}"); ``` **Routing.** `LlmRouting.LlmOnly` (default), `LocalOnly`, or `LocalFirst`: the local extraction model answers the fields it is confident about and the model is called, with a reduced schema, only for the rest. When every field is local, no call is made and nothing is sent. ```csharp LlmReadResult result = await llm.ReadAsync(image, page, new LlmReadOptions { Outputs = LlmOutputs.Json, JsonSchema = schemaJson, Routing = LlmRouting.LocalFirst, // the model is called only for fields the local model is unsure of LocalExtractor = new DocumentFieldExtractor(extractor), ConfidenceThreshold = 0.9f, }); Console.WriteLine($"{result.Calls} call(s), ${result.CostUsd}"); ``` **Privacy.** - Nothing leaves the machine unless you configure a provider; local methods never touch the network, even with a provider configured. There is no telemetry. - API keys are passed explicitly or through an explicit `FromEnvironment()` call; nothing reads environment variables behind your back. Keys are kept in a type that never prints them and are scrubbed from error messages. - `LlmOptions.Offline = true` makes every call throw. - A redaction hook masks regions before upload, and masked words are left out of grounding: ```csharp var options = new LlmOptions { Offline = false, // true makes every call throw OcrProviderException Redactor = (page, pageNumber) => // boxes to mask before upload; masked words are not grounded [.. page.Words.Where(w => w.Text.StartsWith("DE", StringComparison.Ordinal) && w.Text.Length > 15).Select(w => w.Box)], }; using var llm = new LlmClient(OpenAiProvider.FromEnvironment("your-model-name"), options); LlmReadResult result = await llm.ReadAsync(image, image.Ocr(engine), new LlmReadOptions { Outputs = LlmOutputs.Markdown }); ``` - `LlmProvider.NoRetention` is on by default and sends `store: false` to OpenAI. Anthropic, Gemini and Azure OpenAI have no per-request switch, so retention there is set by your account agreement with them. - Document text is sent as delimited, untrusted data and the model gets no tools. Grounding stops invented values; the remaining risk (the model picking a different printed number) is described in the grounding spec. ## Training Training in C# is fine-tuning from the shipped weights, on the CPU, with our own backward passes (convolution with folded batch norm, ReLU, max pooling, bidirectional LSTM, linear layers, CTC and the extraction transformer), AdamW and checkpoints. Given the same labels, settings and seed, the weights are the same on every machine and at every degree of parallelism, and a run resumed from a checkpoint ends with exactly the weights an uninterrupted run would have. Base models are trained in PyTorch by [`training/`](https://github.com/recursive-ai-oss/oneocr-csharp/tree/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/training); every layer's forward and backward pass is tested against PyTorch references. **Recognition.** The local reader reads each page and the model you choose transcribes it. A line becomes training data only when the two agree; disagreements go to a review queue you can export, correct and import. ```csharp var trainer = new OcrTrainer(new OcrTrainerOptions { ModelPath = "recognizer.onm", WorkDirectory = "run1" }, llm); TrainingReport report = await trainer.RunAsync("scans/"); Console.WriteLine(OcrTrainer.ToJson(report)); LineReview.Export("run1", "review/"); // lines the two readers disagreed on, with crops // ...set "text" on the lines you checked in review/review.json, then: LineReview.Import("run1", "review/review.json"); ``` **Extraction.** Give a schema; the model reads your pages for its fields; only values that are found on the page become labels (a page with a value that is not on the page is left out entirely); a new tag head is trained on the shipped backbone, which is tuned gently along with it. ```csharp ExtractionSchema schema = ExtractionSchema.From(OrderContext.Default.PurchaseOrder); // top-level scalar properties are the fields var trainer = new ExtractionTrainer( new ExtractionTrainerOptions { ModelPath = "docextract.onm", Schema = schema, WorkDirectory = "po-run" }, llm); ExtractionReport report = await trainer.RunAsync("purchase-orders/"); DocumentExtractor tuned = DocumentExtractor.Load(report.ModelPath); ``` Every run holds out a share of the documents (by content hash) and reports error rates or per-field accuracy before and after, agreement with the labelling model, and a suggested `LocalFirst` threshold. The held-out references are the labelling model's text or grounded values (or your corrections), so the numbers measure agreement with them; read a few pages yourself before you rely on them. The same runs are available from the command line: ```sh oneocr train --model recognizer.onm --work run1 --docs scans/ \ --provider openai --llm-model --key-from-env OPENAI_API_KEY --budget-usd 2 oneocr review export --work run1 --out review/ oneocr review import --work run1 --file review/review.json oneocr train --model recognizer.onm --work run1 --restart # trains on the labels already in run1; no network oneocr eval --model run1/model.onm --work run1 oneocr train --extract schema.json --model docextract.onm --work po-run --docs purchase-orders/ \ --provider openai-compatible --base-url http://localhost:8080/v1 --llm-model ``` `eval`, `review` and `train` without `--docs` never touch the network. `oneocr help` lists every option. **Provider terms.** Using a provider's output to train a model is governed by your own agreement with that provider. As read on 2026-10-23: Anthropic's Commercial Terms of Service (section D.4) bar using the services "to build a competing product or service, including to train competing AI models"; OpenAI's Services Agreement bars using output "to develop artificial intelligence models that compete with OpenAI's products and services", except a model "primarily intended to categorize, classify, or organize data (e.g., embeddings or classifiers), if these models are not distributed or made commercially available to third parties". A model you train for your own internal extraction may fall under that exception; check your own agreement. A local open-weight model behind the OpenAI-compatible provider avoids the question. ## Determinism Text, boxes and confidences hash the same on every kernel path, every operating system and every degree of parallelism. This is a design rule, not luck: - Dot products and reductions use fused multiply-add with a fixed 16-lane accumulation order: element *k* always lands in partial sum *k mod 16*, whatever the vector width, and the 16 partials reduce in one fixed tree. The scalar path uses `Math.FusedMultiplyAdd`; hardware without FMA takes the same scalar path (exact, slower). - `exp`, `tanh`, `sigmoid`, `log` and softmax are our own polynomial implementations with a fixed operation order, never the platform's math library. - Parallelism splits independent outputs; no sum crosses threads. Weights are stored as half precision and computed in single precision. - Preprocessing, decoding and the page model are written down in [docs/spec/](https://github.com/recursive-ai-oss/oneocr-csharp/tree/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/docs/spec) so another implementation can reproduce them. Tests check every vector path against the scalar reference by hash, over randomized shapes including odd sizes and tails, at degree 1 and at the default degree. ```csharp Console.WriteLine(engine.KernelPath); // Avx512, Avx2, Vector128 or Scalar: the text is the same on all four if (engine.PerformanceNotice is { } notice) { Console.WriteLine(notice); // the scalar path is running on a CPU that could do better } ``` ## Limits and security > Nothing leaves the machine unless you configure a provider; local methods never touch the network. - **Limits before allocation.** `OcrLimits` bounds image pixels (134 million by default), pages per call, lines, words and characters per page, model file and tensor sizes, page-JSON size, LLM response size, schema size, JSON depth and review-import size. Each bound is checked against the declared or running value before the work it guards, and a breach is an `OcrLimitException` that names the limit. Recursion never follows input depth. ```csharp var limits = new OcrLimits { MaxImagePixels = 50_000_000, MaxLinesPerPage = 1000 }; OcrEngine engine = OcrEngine.Load(modelPath, limits); // OcrLimitException before anything large is allocated ``` - **Untrusted bytes.** Model containers, images, page JSON, PDFs, LLM responses, JSON Schemas and review files are parsed by safe, bounds-checked code with checked arithmetic on every size and offset taken from the input, and each has a fuzz target under [`fuzz/`](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/fuzz/README.md). No crash or hang on any input is a release requirement. - **`unsafe` only in the kernels.** `unsafe`, `Unsafe.*` and `MemoryMarshal` appear only under `src/OneOcr/Kernels/`, over buffers that safe callers validated. - **No networking in the core package.** `HttpClient` appears only in `RecursiveAI.OneOcr.Llm`. There is no telemetry. - **No runtime dependencies** beyond the .NET base class library, OneImage (pixels) and, for the PDF package, OnePDF and OneImage.Pdf. ## NativeAOT The libraries and the tool are trim- and AOT-clean (no reflection, no runtime code generation). A NativeAOT binary is compiled for one instruction-set level, and .NET's default for x64 (`x86-64-v2`) has no AVX2 or FMA. At that level the kernels fall back to their exact scalar path: the text is the same, but a page takes about a hundred times longer. The `RecursiveAI.OneOcr` package therefore sets `IlcInstructionSet` to `x86-64-v3` (AVX2 and FMA, any x86-64 CPU since about 2015) for x64 NativeAOT publishes whose project has not chosen a level; a value in your project or on the command line always wins. Use `x86-64-v4` for the AVX-512 kernels. A binary built for v3 refuses to start on an older CPU. Arm needs nothing. To see what a process runs, read `OcrEngine.KernelPath`; `OcrEngine.PerformanceNotice` is non-null when the scalar path is running on a CPU that could run AVX2 and says how to fix it (an app published below v3, or `DOTNET_EnableAVX2=0` in the environment). With `DOTNET_EnableHWIntrinsic=0` the scalar path runs too, but the CPU cannot be queried, so the notice stays null. It never throws and is not part of the page, so page JSON is the same on every path. The `oneocr` tool can be published the same way for fast starts in scripts: `dotnet publish src/OneOcr.Tool -c Release -r linux-x64` (see [src/OneOcr.Tool](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/src/OneOcr.Tool/README.md)). ## Models and licences Code and model weights are licensed `MIT OR Apache-2.0`. Weights are stored as half precision in our own versioned container format ([docs/spec/model-container-v1.md](https://recursiveai.org/oneocr/docs/model-container/)), which is the contract between the PyTorch exporter and the C# reader; user-trained models use the same format. | Model | Role | Architecture | Size | Package | |---|---|---|---|---| | `crnn-v0` 0.7.3 | Line recognizer: 622 classes (Latin, Greek, Cyrillic, punctuation, currency), per-script calibrated confidence | convolutions, 2-layer BiLSTM, CTC; 2.5 M parameters | 5.0 MB | `.Models` | | `dbnet-lite-v1` 0.3.2 | Page detector: text lines (oriented), page orientation, layout classes | DBNet-style, 0.7 M parameters; [spec](https://recursiveai.org/oneocr/docs/detector/) | 1.4 MB | `.Models` | | `charlm-v1` | Character n-gram model for word breaks and optional beam search; [spec](https://recursiveai.org/oneocr/docs/char-lm/) | interpolated Kneser–Ney, Latin + Greek + Cyrillic | 1.4 MB | `.Models` | | `wordfreq-v2` | Word frequencies that take back a space the recognizer put inside a word; [spec](https://recursiveai.org/oneocr/docs/word-lm/) | counts of 219 k Latin-script words | 1.3 MB | `.Models` | | `docextract-v0` | Invoice and receipt word tagger; [spec](https://recursiveai.org/oneocr/docs/docextract/) | character-CNN word embedding, 2-D position embedding, small transformer | about 4 MB | `.Models.Documents` | Training-data attributions and font licences are in [THIRD-PARTY-NOTICES.md](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/THIRD-PARTY-NOTICES.md). ## How the models were trained Everything is reproducible from [`training/`](https://github.com/recursive-ai-oss/oneocr-csharp/tree/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/training) (a Python project; pinned fetch scripts, generator seeds and configs). Weights start from random initialization or from our own earlier checkpoints; no third-party pretrained weights are used, and no weights, images or training data are committed to the repository. - **Recognizer.** Text lines rendered from public-domain books (Project Gutenberg) in 77 open font families (SIL OFL, Apache-2.0 and Ubuntu Font Licence), in Latin, Greek and Cyrillic, then degraded (blur, noise, JPEG, low resolution, faint ink, perspective, uneven light) with exact ground truth; plus word crops of the CORD v2 receipts training split with its labels, synthetic receipt and interface text, text lines cut from screenshots of public-domain US federal web pages (labels from the page's own text), and photos of receipts generated with an image model and transcribed by a language model (see Labels). Held-out font families, held-out books and held-out web sites form the evaluation sets. - **Detector.** Page images and text-line boxes from DocLayNet and PubLayNet (only articles under CC0 and CC BY), CORD v2 word quads, synthetic pages (including light-on-dark ones) rendered in Chromium with exact word boxes, and windows of public-domain US federal web pages with the browser's own line boxes. - **Extraction.** Generated invoices and receipts with exact field truth, plus CORD v2 training labels. - **Labels.** Labels for shipped weights come from exact-truth synthetic data, text in the page source, dataset-provided labels (CORD, DocLayNet, PubLayNet) and permissively licensed open-weight models. The one exception is the generated receipt photos in the recognizer's receipt fine-tune, which a commercial language model transcribed; it is listed in THIRD-PARTY-NOTICES.md. Apart from that, commercial LLM APIs appear only as benchmark rows for the LLM layer. Nobody hand-labels images. - **Clean room.** Other OCR systems were studied only through their papers, public documentation and the output of their released binaries. Their source code and model files were never read or used. ## Benchmarks **Accuracy result for 1.0, stated as measured: OneOcr is the best engine we measured on five of the ten gated categories and behind the best on five.** It beats Tesseract on every category except clean real PDFs, where Tesseract is ahead by about 0.1 point. It is ahead on clean print, degraded scans, phone photos and real scan-degraded and layout PDFs; it trails the best engine (PaddleOCR or Apple Vision) on synthetic screenshots and complex layout (CER and WER; it wins their reading order), real web screenshots, clean real PDFs (narrowly) and generated photos of US receipts. One run of the release gate on a held-out (sealed) test set that no tuning step saw, scorer v3.1, engine at the 1.0 commit with the bundled models; AMD Ryzen 9 7950X3D, 32 threads, load 5–8 before and after; competitor outputs measured once on the same pages. The clean real-PDF row uses a fresh cut of 27 pages: the first cut was opened up as a development set during development, so it was replaced before this run. *Accuracy, error rate %, CER / WER / reading order (lower is better). For PaddleOCR, docTR and Apple Vision each cell is the best of their published configurations (PaddleOCR PP-OCRv6 and v5 mobile/server, docTR default/mobile, Vision accurate with and without language correction); Tesseract 5.3.4 uses its best page-segmentation mode per metric.* | Category | OneOcr | Tesseract (best) | Tesseract (fast) | PaddleOCR | docTR | Apple Vision | OneOcr vs best | |---|---|---|---|---|---|---|---| | Clean 300 DPI print | 0.05 / 0.10 / 0.10 | 0.16 / 0.62 / 0.62 | 0.24 / 0.83 / 0.83 | 0.07 / 0.11 / 0.11 | 0.94 / 2.57 / 2.61 | 0.37 / 0.76 / 0.78 | win / win / win | | Degraded scans | 0.16 / 0.45 / 0.48 | 0.93 / 2.67 / 2.77 | 1.22 / 3.30 / 3.44 | 0.39 / 1.17 / 1.18 | 1.64 / 4.51 / 13.23 | 0.21 / 0.65 / 0.66 | win / win / win | | Screenshots and UI | 0.35 / 0.67 / 0.73 | 6.33 / 9.90 / 25.14 | 5.86 / 9.29 / 24.50 | 0.12 / 0.40 / 17.14 | 1.88 / 6.51 / 22.69 | 0.33 / 1.50 / 25.35 | loss / loss / win | | Complex layout | 0.14 / 0.27 / 0.70 | 13.57 / 11.69 / 13.62 | 13.58 / 11.94 / 13.96 | 0.08 / 0.12 / 62.32 | 1.14 / 3.20 / 63.33 | 0.60 / 1.23 / 22.82 | loss / loss / win | | Phone photos | 0.21 / 0.67 / 0.81 | 9.34 / 16.48 / 17.87 | 9.40 / 16.21 / 17.48 | 0.39 / 0.83 / 0.83 | 3.41 / 10.54 / 38.07 | 1.16 / 1.90 / 2.45 | win / win / win | | Receipt photos, US (generated images; recall-CER / recall-WER) | 8.14 / 14.24 | 29.37 / 47.43 | 27.41 / 44.77 | 3.74 / 8.72 | 4.59 / 8.63 | 3.93 / 8.03 | loss / loss | | Real web screenshots (CER / WER) | 1.98 / 3.14 | 8.62 / 10.93 | 8.76 / 11.66 | 0.88 / 1.63 | 2.89 / 5.16 | 0.81 / 1.63 | loss / loss | | Real born-digital PDFs: clean | 0.57 / 0.97 / 1.63 | 0.47 / 0.94 / 1.48 | 0.45 / 0.94 / 1.49 | 0.47 / 0.91 / 1.42 | 0.99 / 2.00 / 2.64 | 0.95 / 1.59 / 2.23 | loss / loss / loss | | Real born-digital PDFs: layout | 0.52 / 1.63 / 3.63 | 1.42 / 4.68 / 6.60 | 1.34 / 4.79 / 6.72 | 0.99 / 4.20 / 84.23 | 1.34 / 5.20 / 84.37 | 2.25 / 5.27 / 8.11 | win / win / win | | Real PDF pages, scan-degraded | 1.64 / 3.76 / 10.15 | 4.31 / 9.26 / 11.97 | 4.73 / 9.92 / 12.65 | 2.22 / 6.39 / 64.78 | 2.86 / 8.84 / 70.44 | 6.98 / 11.28 / 19.53 | win / win / win | | Rendered receipts (recall-CER / recall-WER; information) | 0.14 / 0.44 | 5.62 / 10.24 | 5.93 / 10.97 | 1.13 / 1.80 | 3.11 / 10.22 | 0.38 / 1.03 | ahead / ahead | The four "Real" rows use real pages (rendered public web pages and PDFs with a verified text layer); they are reported apart from the synthetic rows because their truth and their difficulty differ. Receipt photos are judged on recall over the annotated words, and reading order is shown only where it is gated. *Calibration and detection.* | Category | ECE (target ≤ 0.05) | Word-box F1 at IoU 0.5: OneOcr | Tesseract (best) | |---|---|---|---| | Clean 300 DPI print | 0.009 | 0.999 | 0.999 | | Degraded scans | 0.011 | 0.998 | 0.989 | | Screenshots and UI | 0.012 | 0.991 | 0.797 | | Complex layout | 0.012 | 0.997 | 0.935 | | Phone photos | 0.014 | 0.968 | 0.816 | | Receipt photos, US (generated) | 0.041 (ECE-m; partial truth) | - | - | | Real web screenshots | 0.012 | 0.980 | 0.904 | | Real born-digital PDFs: clean | 0.007 | 0.994 | 0.996 | | Real born-digital PDFs: layout | 0.014 | 0.872 | 0.927 | | Real PDF pages, scan-degraded | 0.006 | 0.867 | 0.912 | | Rendered receipts | 0.012 | 0.984 | 0.945 | *Speed and memory, milliseconds per page for the whole process (start-up and model load included), NativeAOT build of the `oneocr` tool for x86-64-v4, median of 3 runs. Tesseract 5.3.4 runs one process per page with one thread and its best page-segmentation mode. Development pages of the same six kinds (30–100 per set); AMD Ryzen 9 7950X3D, 32 threads, nothing else running (load 3.8 before, at most 3.4 during, 1.5 after).* | Category | OneOcr, 1 thread | OneOcr, 4 threads | Tesseract fast | Tesseract best | OneOcr peak RSS, MB (target ≤ 300) | |---|---|---|---|---|---| | Clean 300 DPI print | 528 | 210 | 677 | 1384 | 118 | | Degraded scans | 536 | 209 | 540 | 1120 | 140 | | Complex layout | 698 | 259 | 898 | 1704 | 197 | | Phone photos | 577 | 232 | 661 | 1162 | 152 | | Screenshots and UI | 196 | 78 | 226 | 389 | 127 | | Receipts | 222 | 101 | 289 | 425 | 130 | On one thread OneOcr is faster than Tesseract with its fast models in every category (degraded scans only just), and about twice as fast as Tesseract with its best models. **Method.** - **Metrics.** CER and WER are order-free: each truth word is paired with its closest leftover output word. Reading-order error is the edit distance over the word sequence divided by the number of truth words (it includes recognition errors, so read it next to WER). Text is normalized by NFC, typographic quotes and dashes folded to ASCII, ligatures split, whitespace collapsed; case and punctuation are kept; two apostrophes in a row count as one double quote. The truth word sequence is built from the truth's word boxes where it has them (words a page wraps at a hyphen stay separate, a punctuation mark touching a word joins it), the same for every engine; the older text-based scoring is reported beside it. Receipts have partial truth (CORD leaves store and payment blocks unannotated), so they are judged on recall-CER and recall-WER, the error over truth words only. Word-box F1 uses boxes tight to the printed ink. ECE uses 15 equal-width bins over word confidence, a word being correct when it matches its truth exactly. - **Truth.** Never produced by an OCR engine: browser layout geometry, the geometry of our own transforms, a dataset's shipped annotation, or a PDF's text layer checked by two independent readers. Pages and truth are hashed in manifests; the generators and seeds are in [`bench/`](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/bench/README.md). - **Engines.** Tesseract is the distribution's 5.3.4 binary with its own `tessdata_best` and `tessdata_fast` models, run as a black box; for each metric the best of page-segmentation modes 3, 4, 6 and 11 is taken (the hardest baseline). PaddleOCR (PP-OCRv5 mobile and server, PP-OCRv6), docTR (default and mobile) and Apple Vision (accurate) run as released packages with their stock models. Their output is a comparison row, never training data. - **Rule.** An engine is judged against the best value any measured engine reaches per metric. On clean print, an error within 0.05 percentage points of the best is reported as a tie. - **Speed.** One process per run, the shipped `oneocr ocr` published NativeAOT for the host, three runs, mean; peak RSS from the operating system. Tesseract is timed the same way at one thread and at the same thread count as OneOcr. - **Reproduce.** `python -m oneocr_bench gate` in [`bench/`](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/bench/README.md) prints the quality and calibration table for an evaluation directory; `bench/scripts/full_run.sh` runs the whole comparison on one machine. ## Not yet Searchable-PDF output and int8 inference (planned after 1.0); CJK, Arabic and Indic scripts (the page model already has right-to-left and vertical flags, but the local reader does not produce them); handwriting; curved-page dewarping and scene text; text drawn as outlined glyphs in PDFs; AWS Bedrock and Google Vertex providers; a built-in PII redactor (the redaction hook is yours to fill); a word-level character whitelist. ## Where to go - **Formats and rules** ([docs/spec/](https://github.com/recursive-ai-oss/oneocr-csharp/tree/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/docs/spec)): the [model container](https://recursiveai.org/oneocr/docs/model-container/), [page JSON](https://recursiveai.org/oneocr/docs/page-json/), [numeric rules](https://recursiveai.org/oneocr/docs/numerics/), [decoding](https://recursiveai.org/oneocr/docs/decoding/), [text normalization](https://recursiveai.org/oneocr/docs/text-normalization/), [Markdown](https://recursiveai.org/oneocr/docs/markdown/), [tables](https://recursiveai.org/oneocr/docs/tables/), [orientation](https://recursiveai.org/oneocr/docs/orientation/), [grounding](https://recursiveai.org/oneocr/docs/grounding/), [PDF bridge](https://recursiveai.org/oneocr/docs/pdf-bridge/), [review queue](https://recursiveai.org/oneocr/docs/review-queue/). - **Migrating:** [from Tesseract](https://recursiveai.org/oneocr/docs/migrating-from-tesseract/). - **Evidence:** [benchmark tooling](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/bench/README.md), [fuzzing](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/fuzz/README.md). - **The API reference** is the XML documentation your IDE shows. ## License Code and model weights are licensed under either of [MIT](https://github.com/recursive-ai-oss/oneocr-csharp/tree/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/LICENSE-MIT) or [Apache-2.0](https://github.com/recursive-ai-oss/oneocr-csharp/tree/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/LICENSE-APACHE) at your option. Copyright (c) Recursive AI, LLC. Third-party data and fonts: [THIRD-PARTY-NOTICES.md](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/THIRD-PARTY-NOTICES.md). # Moving from Tesseract to OneOcr This guide maps what you do with the `tesseract` command line (or a wrapper around it) to the OneOcr API. It is based on Tesseract's public documentation (its manual pages and `--help` output) and on its observable behaviour, nothing else. Where a setting has no counterpart, the table says so rather than inventing one. The short version: OneOcr is a managed library, so there is no executable to install, no `tessdata` folder to point at, and no language code to choose for Latin, Greek and Cyrillic text. You load an engine once, hand it an image and read the page model. ```csharp // tesseract page.png out -l eng -> out.txt string text = Image.Load("page.png").ToText(engine); ``` Every code block in this guide compiles; the samples live in `tests/OneOcr.Samples/`. ## Concepts side by side | Tesseract | OneOcr | |---|---| | `tesseract in.png out` | `image.ToText(engine)` for text, `image.Ocr(engine)` for the page model | | `tessdata`, `TESSDATA_PREFIX`, `-l eng` | None. `OcrEngine.LoadBundled()` loads the recognizer, page detector and word model from `RecursiveAI.OneOcr.Models`. `OcrEngine.Load(path)` reads a model container you trained or fine-tuned | | `tessdata_fast` / `tessdata_best` | One bundled set of models. There is no speed/accuracy switch to flip | | Languages: `-l eng+deu`, `-l ell`, `-l rus` | One recognizer reads Latin (English, German, French, Turkish, Vietnamese and the other European languages), Greek and Cyrillic together, so mixed text needs no language list. CJK, Arabic and Indic scripts are not supported yet | | Page segmentation modes `--psm 0`–`13` | No modes. The page detector finds text lines and the layout code orders them into columns, tables and blocks. See "Page segmentation modes" below | | `--oem` | Not applicable: there is one engine | | `--dpi N` | `OcrOptions.Dpi`. It only sets the page's `UnitsPerInch`; recognition does not depend on it. When it is absent the DPI comes from the file (`OcrPages(path)`, PNG, JPEG, TIFF headers) or is estimated, and a diagnostic says so | | `OMP_THREAD_LIMIT` | `OcrOptions.MaxDegreeOfParallelism` (0 means min(4, cores)). The result is bit-identical at every setting | | `--psm 0` (orientation and script detection) | `OcrOptions.DetectOrientation` (on by default; 0, 90, 180 and 270 degrees) and `OcrOptions.Deskew` (on by default). The result is in `OcrPage.Orientation` and `OcrPage.SkewDegrees` | | `-c tessedit_char_whitelist=...` | No whitelist. Filter the words you get back | | Dictionaries, `load_system_dawg`, user words | No dictionary by default, so identifiers and codes are never "corrected". Beam search with a character language model and a lexicon is opt-in (`DecodeOptions`) | | `preserve_interword_spaces` | Not applicable. Word breaks are decided per line from the pixels, and a line's text is its words joined by one space; cells of a table row are joined by a tab | | Output `txt` | `image.ToText()` (every frame of a multi-page TIFF), or `OcrPage.Text` with blank lines between blocks | | Output `tsv` | Walk `page.Blocks[i].Lines[j].Words`, see the recipe below | | Output `hocr` | `OcrPage` → `OcrBlock` → `OcrLine` → `OcrWord`, serialized as `oneocr.page` JSON | | Output `pdf` (searchable PDF) | Not yet | | Output `alto`, `lstmbox`, `makebox` | Not provided | ## Page segmentation modes Tesseract asks you to say what the page looks like (a single block, a single line, a single word, sparse text), and the best mode differs from page to page. OneOcr takes no such hint. | Tesseract mode | In OneOcr | |---|---| | 3, fully automatic (the default), 1, 2, 4 | The normal path: `image.Ocr(engine)`. Columns, headers, footers and tables are found for you | | 0, orientation only | `DetectOrientation` and `Deskew` are options of the normal path; there is no call that only detects | | 5, 6, a single block | The normal path. Text boxes and screenshots read the same as pages | | 7, 13, a single line | Pass the crop to `Ocr` with orientation and deskew off, see the recipe below | | 8, 10, a single word or character | Same as a single line; the result has one word | | 11, 12, sparse text | The normal path. The detector looks for text lines anywhere on the page, not only in blocks | If you used a mode only because the default read your page badly, try the default first: it is the one OneOcr is built and measured on. ## Confidence | | Tesseract | OneOcr | |---|---|---| | Scale | Per word, 0 to 100 (TSV `conf`, hOCR `x_wconf`) | Per word, 0 to 1 (`OcrWord.Confidence`); lines, blocks and the page aggregate it | | Meaning | A score from the recogniser's internals | The calibrated probability that the word is exactly right | | Calibration | None documented; thresholds are tuned per project | Fitted per script on held-out data. Words at 0.9 are right about 90% of the time. The expected calibration error is published per category in the README | | Raw value | – | `OcrWord.RawConfidence` is the uncalibrated mean over the word's characters | Because the number is a probability you can set one threshold and expect it to mean the same thing on every document. To move an existing threshold over, divide it by 100 as a starting point and then look at the words just below it: ```csharp foreach (OcrWord word in image.Ocr(engine).Words.Where(w => w.Confidence < 0.8f)) { Console.WriteLine($"check: {word.Text} ({word.Confidence:P0}) at {word.Box.Left:F0},{word.Box.Top:F0}"); } ``` ## Output formats ### Plain text `image.ToText(engine)` returns the page in reading order: lines are separated by a newline and table cells by a tab; `page.Text` also puts a blank line between blocks (`page.ToText(OcrTextLayout.Paragraphs)` is the same thing). Reading order is worked out from the layout, so two-column pages come out column by column. ### TSV Tesseract's TSV has one row per page, block, paragraph, line and word, with `left top width height conf text`. The word rows map directly onto `OcrWord`. Boxes are top-left origin, y down, in pixels for images (in points for PDF pages), as Tesseract's are. ```csharp OcrPage page = image.Ocr(engine); for (int b = 0; b < page.Blocks.Count; b++) { for (int l = 0; l < page.Blocks[b].Lines.Count; l++) { foreach (OcrWord word in page.Blocks[b].Lines[l].Words) { // block, line, left, top, width, height, confidence 0-100, text: the columns of Tesseract's TSV Console.WriteLine(string.Join('\t', b, l, (int)word.Box.Left, (int)word.Box.Top, (int)(word.Box.Right - word.Box.Left), (int)(word.Box.Bottom - word.Box.Top), (int)Math.Round(word.Confidence * 100), word.Text)); } } } ``` ### hOCR hOCR describes a page as nested elements: `ocr_page`, `ocr_carea` (a block of the page), `ocr_par`, `ocr_line` and `ocrx_word`, each with a `bbox` and words with `x_wconf`. The page model has the same nesting: | hOCR | Page model | |---|---| | `ocr_page` (`bbox`) | `OcrPage` (`Width`, `Height`) | | `ocr_carea` / `ocr_par` | `OcrBlock` (role body, header or footer; `IsTable` for grids) | | `ocr_line` | `OcrLine` | | `ocrx_word`, `x_wconf` | `OcrWord`, `Confidence` | | `ocrx_cinfo` | `OcrChar`, only with `OcrOptions.IncludeChars`; character boxes are approximate and flagged as such | | `bbox x0 y0 x1 y1` | `OcrBox` (`Left`, `Top`, `Right`, `Bottom`) | | `textangle` | `OcrQuad` on rotated elements; `OcrPage.SkewDegrees` for the page | Nothing in the library writes hOCR. If you have a consumer that needs it, the walk above is all it takes. For storage and exchange, use `oneocr.page` JSON, which is documented in [docs/spec/oneocr-page-v1.md](https://recursiveai.org/oneocr/docs/page-json/) and reads back with `OcrPageJson.Parse`. ### PDF OneOcr does not write searchable PDFs yet. It does read PDFs: `PdfOcr.Read` uses a PDF's own text where it has any and reads the scanned parts (see the README). ## Recipes ### Many files at once The engine is immutable and thread-safe, so one engine serves every thread. Run files in parallel and leave each page at its default degree, or lower the degree per page when you run many pages at once. ```csharp Parallel.ForEach( Directory.EnumerateFiles(folder, "*.png"), new ParallelOptions { MaxDegreeOfParallelism = 4 }, path => File.WriteAllText(Path.ChangeExtension(path, ".txt"), Image.Load(path).ToText(engine))); ``` ### One line, one word Where you used `--psm 7` or `--psm 8`, pass the crop to `Ocr`. Turning orientation and deskew off stops the reader second guessing a short strip. ```csharp // A crop that is one line of text: the page reader finds that line and reads it. OcrPage page = Image.Load(cropPath).Ocr(engine, new OcrOptions { DetectOrientation = false, Deskew = false }); string line = page.Text; float confidence = page.Words.Count == 0 ? 0 : page.Words.Min(w => w.Confidence); ``` ### JSON for another program Where you ran `tesseract in out hocr` or `tsv` for a downstream step, write the page model instead: ```csharp string json = OcrPageJson.ToJson([page], new OcrPageJsonOptions { Indented = true }); IReadOnlyList again = OcrPageJson.Parse(json); // checked against OcrLimits; throws OcrFormatException on bad input ``` ### Tables, key–value pairs, Markdown These replace the post-processing most Tesseract pipelines grow. `page.Tables` holds cells with row and column spans that refer to words, `image.ToMarkdown(engine)` renders headings, lists and pipe tables, and `image.ExtractInvoice(...)`, `ExtractReceipt` and `Extract` return typed fields with their boxes. The README has the details. ## What is different - **No native binaries.** The library is managed code over the .NET base class library, so it runs wherever .NET 10 does, including NativeAOT, with no `libtesseract`, no `leptonica` and no data files to deploy beside the application. - **Deterministic.** Text, boxes and confidences are bit-identical on AVX-512, AVX2, 128-bit vector and scalar hardware, on every operating system, and at every degree of parallelism. `engine.KernelPath` says which path is running. - **Calibrated confidence.** See above. - **A page model, not a flat result.** Blocks, lines, words, tables, key–value pairs and diagnostics come back in one object, with a `Source` on each element, and the same shape is used for text read from a PDF and text read by an LLM. - **Limits and typed errors.** Image size, lines per page and model sizes are bounded by `OcrLimits` before anything large is allocated, and the public methods throw only the typed exceptions in the README. - **Optional LLM vision, never by default.** `RecursiveAI.OneOcr.Llm` adds grounded reading through a provider you configure. The core package has no networking code. ## What is not there yet Searchable-PDF output, CJK, Arabic and Indic scripts, handwriting, scene text (signs and photos of the world rather than documents), curved-page dewarping, word whitelists and a word dictionary. Tesseract `.traineddata` files cannot be loaded or converted: oneocr runs only its own models, and you fine-tune those with `oneocr train`. See "Not yet" in the README. # oneocr.page, version 1 The JSON form of an OCR result: pages of blocks, lines, words and characters, with the layout elements a layout model can add (regions, tables, key–value pairs). It is written by `OcrPageJson.ToJson`/`Write`, read by `OcrPageJson.Read`/`Parse`, and is deterministic: the same pages give the same bytes on every platform and path. It is built on the OnePDF `onepdf.extraction` JSON (member names, order, rounding, text joining). Every member that has a counterpart there keeps its name, its place and its meaning; what OCR needs beyond that is added at the end of each object. Three things differ on purpose, and a converter must handle them: 1. **y runs down.** Origin at the top-left of the page, x to the right, y down. `onepdf.extraction` is y up. 2. **Box order is `[left, top, right, bottom]`** (OnePDF: `[left, bottom, right, top]`). With y down, `top <= bottom`. 3. A word has no `glyphs` list: OCR has no glyph indices. It has `chars` instead (below). ## Format - `format` is `"oneocr.page"` and `version` is `1`. A reader must check both. New members keep the version; renaming, removing or changing the meaning of a member raises it. - UTF-8, no byte-order mark (a reader rejects one). Strings are escaped the way OnePDF's writer does (the relaxed .NET JSON encoder: text in the Basic Multilingual Plane is written as it is, characters beyond it as `\uXXXX` surrogate-pair escapes). Unpaired surrogates are written as U+FFFD. - Numbers: coordinates, sizes and `unitsPerInch` are rounded to **three decimals**, confidences to **four**, `toPdf` entries to **six**; ties to even; invariant form, no exponent, no trailing zeros; negative zero and non-finite values are `0`. Integers are plain. - Member order is the order below. Whitespace is insignificant. - Duplicate member names are an error. Unknown members are ignored by a reader (they may be additive members of a later minor revision). ## Coordinates Page units: pixels for images, points for PDF pages. `unitsPerInch` says how many units make an inch (the image's DPI, or an estimate, with a diagnostic, when the caller gave none; 72 for PDFs). When a path, stream or bytes are given to `OcrPages`, the file's own resolution is used: PNG `pHYs` (unit metre), JPEG JFIF density (units 1 = per inch, 2 = per centimetre) or the first TIFF directory's `XResolution` with `ResolutionUnit`; only the first 4 MiB are looked at and a value outside 10..10000 is ignored. `toPdf`, present only for pages that came from a PDF, is the matrix `[a, b, c, d, e, f]` taking a point `(x, y)` of this page to PDF user space: `x' = a x + c y + e`, `y' = b x + d y + f`. ## Shape ``` { "format": "oneocr.page", "version": 1, "pageCount": , must equal the length of "pages" "pages": [ Page, ... ] } Page = { "index": , "width": , "height": , "unitsPerInch": , "toPdf": [a, b, c, d, e, f], optional "orientation": 90 | 180 | 270, optional, written only when not 0 "skew": , optional, written only when not 0 "diagnostics": [ , ... ], the messages only; the API also has a stable code and a severity per diagnostic, which the JSON does not carry (a reader gives them code "OCR000") "text": , blocks joined by "\n\n", lines by "\n", cells by "\t"; derived, ignored on read "blocks": [ Block, ... ], reading order "regions": [ Region, ... ], "tables": [ Table, ... ], "keyValues": [ KeyValue, ... ] } Block = { "role": "body" | "header" | "footer", "table": , "bounds": [l, t, r, b], "lines": [ Line, ... ], "source": "ocr" | "textLayer" | "llm", "confidence": , "quad": [x0, y0, x1, y1, x2, y2, x3, y3] optional } Line = { "direction": "ltr" | "rtl", "vertical": , "bounds": [l, t, r, b], "cells": [ , ... ], indices into "words" where a cell starts; first is 0; empty only without words "text": , words joined by " ", cells by "\t" "words": [ Word, ... ], "source", "confidence", "quad" as in Block } Word = { "text": , logical order, never empty "bounds": [l, t, r, b], ink extent for recognized words "visible": , "source": ..., "confidence": , calibrated probability the word is exactly right "rawConfidence": , uncalibrated (mean of the characters' largest softmax probability) "chars": [ Char, ... ], optional "quad": [...] optional } Char = { "text": , "bounds": [l, t, r, b], "confidence": , "source": ..., "approximate": , true when the horizontal edges come from CTC alignment, not the glyph outline "quad": [...] optional } Region = { "role": "text"|"title"|"list"|"table"|"figure"|"header"|"footer"|"formField"|"checkbox", "bounds", "confidence", "source", "blocks": [ , ... ], "quad" optional } Table = { "bounds", "rows": , "columns": , "cells": [ Cell, ... ], "confidence", "source", "block": , "quad" optional } Cell = { "row", "column", "rowSpan", "columnSpan", "bounds", "words": [ , ... ] } KeyValue = { "key": [ , ... ], "value": [ , ... ], "bounds", "confidence", "source" } ``` **Orientation and skew.** `orientation` is how far the page content was turned clockwise from upright when the image was read: 90 means the tops of the lines point to the right. `skew` is the tilt, in degrees (three decimals), of the text lines that was removed before reading, positive when the lines descend to the right; it is within 45. Both are additive members of version 1, written only when they are not 0 (so a page that needed neither has the same bytes as before) and read as 0 when absent. A reader rejects an `orientation` other than 0, 90, 180 or 270 and a `skew` beyond 45 with `OcrFormatException`. All coordinates stay in the image as given, whatever the turn; text that was turned or straightened carries a `quad` (corners in the text's own frame: top-left, top-right, bottom-right, bottom-left), and its `bounds` is the box around the quad. How the turn and tilt are found and mapped back is in `page-orientation-v1.md`. Pages from a PDF have neither member. **Lines read along their own axis.** A line whose own lean differs from the page's (after any page-wide deskew) carries a `quad` on its words, characters and the line even when the page itself was not turned or straightened: the quad is the line's tilted rectangle, and `bounds` is the box around it. Straight lines carry none, as before. See `detector-v1.md`, "Line axis". **Word index.** Words of a page are numbered in reading order: blocks in order, lines in order, words in order, from 0. `words` of a cell and `key`/`value` of a key–value pair refer to that numbering. Block indices count from 0 in `blocks`. **Line confidence** is the words' calibrated confidences averaged with each weighted by its character count; **block confidence** averages its lines weighted by their word counts; both are 0 without words. ## What the reader checks Everything is checked before the structure it guards is built. Errors are `OcrFormatException` (shape, types, ranges, indices, unknown names for `role`/`source`/`direction`, the format and version members, a byte-order mark, invalid JSON, duplicate members, `pageCount` disagreeing with `pages`, non-finite numbers, a box with `right < left` or `bottom < top`, probabilities outside 0..1, `cells` not starting at 0 and strictly increasing, indices out of range, spans leaving the table) and `OcrLimitException` (document bytes, `MaxJsonDepth`, pages per call, blocks, lines, words, characters and other elements per page, string length; see `OcrLimits`). Nothing else leaves the reader. ## Reading order Local OCR finds text lines as ink strips, then arranges them with the algorithm of OnePDF's text layout (recursive X–Y cut over the strips, Nagy & Seth 1984; whitespace columns; line merging; grid detection for tables), applied to OCR words instead of PDF fragments: - Strips are found by a recursive X–Y cut of the ink mask: a vertical gutter at least 1.25 line heights wide that runs through at least 2.5 lines splits columns; rows inside a column split at gaps of half a line. Ruling lines are removed from the mask first. A page with no gutter and no rule is read as one column exactly as a single-column page always was. - Strips on one row from different columns join into one line when their vertical centres agree within 0.3 em. - A horizontal cut is refused when more than one column lies in the region and a column has text on both sides of the gap (columns that share paragraph gaps are not interleaved). - Sections of columns are cut as a whole. When the horizontal cut would run through a page with several columns, every empty band that crosses all of them (paragraphs that end at the same height) would cut the columns into slices read left, right, left, right. Consecutive bands that share an empty vertical stripe of at least 1.25 em are therefore merged back into one section, which is then cut into its columns. A section that would be a table is not merged. - Narrow gutters (`LayoutParameters.Soft*`). Print gutters are about one em of the font, a little over the measured text height, so a gutter of 1.25 text heights is missed by a pixel or two on tightly set pages and more on blurred scans, and the page was read row by row across its columns (the cells of one line joined by tabs). When a region has no stripe of 1.25 text heights and no fragment of any of its lines crosses a stripe of at least 0.4 text heights, and the region has at least 10 distinct lines, it is cut there into columns if every column has at least 5 distinct lines and the widest quarter of its fragments is at least 12 text heights wide (full lines of running text; a column of bullets, labels or table cells is not). These columns skip the label, table, sidebar and tied-row rules and are read one after the other, top to bottom. Regions that already had a stripe of 1.25 text heights are cut exactly as before. The bar is 0.4 and not higher because word boxes are ink extents, a little short of the print: a page whose gutter measured 0.46 text heights was just under 0.5 with one detector and over it with another, and its columns flipped. A stripe that nothing in a dozen lines touches is not a gap between words, so the lower bar costs no false cuts on the tune sets. - Gutters with a few words in them (`LayoutParameters.Robust*`, `ReadingOrder.Columns.cs`). On scans a stray word, a speck read as a word, or a justified line that runs a few pixels into the next column keeps every stripe from being empty. When a region of at least 30 distinct lines still has no cut after the narrow-gutter rule, the words of the region are swept left to right counting how many cover each position. A stripe is a run, inside the text (not at its margins) and at least 0.2 text heights wide, where at most a tenth of the peak count (at least one word) lies, and where the mean count over the 3 text heights on each side is at least a quarter of the peak (text stands up to it on both sides; the ragged right edge of long lines, with a few longer lines reaching beyond, is not a gutter). Its position is the middle of the longest run at the lowest count. If no stripe leads to a cut, the sweep is repeated with the words' positions sheared, x minus k times the height of the word's centre, for k = 0.0025 to 0.0125 in steps of 0.0025, both signs, least lean first (a skew of up to 0.7 degrees: a page is deskewed before it is read, and the estimate leaves a little lean, so the gutter is clear along its slant but no vertical stripe is empty from top to bottom). The first k that gives columns is used; the upright reading is always tried first, so pages that had columns keep them. Words are assigned to columns by their centres, and a fragment that runs across a stripe is cut between its words. The columns must pass the same running-text test as above (5 lines, widest quarter at least 12 text heights) and be at most 1.6 times as wide as each other (a wider group has a gutter missing in it); if not, the stripe with the most words in it is dropped and the rest tried again, until none is left. When more than a tenth of the lines (at least 5) have words on both sides of a stripe, the region is not cut into columns here at all: a table or figure across the columns is cut off by the horizontal cut first. Regions that had a cut by the earlier rules are cut as before. *Header and footer.* Before the stripes are sought, a band of at most 5 distinct lines at the top (bottom) of the region, set off from the rest by an empty band of one text height (fragments of any region, not only of one detector region), is read before (after) the columns when its fragments fill at most 40% of the region's width on each line and it either spans at least 60% of that width or is a single line (a footer of a few labels, or a page footer at the left, not the first lines of two columns). A region of at least 30 lines that has no cut at 1.25 text heights has such a band cut off first, before the narrow-gutter rule, so that a footer at the left is not taken for the end of the left column. A line with words on both sides of a stripe is also a header (footer) when at most 5 lines lie above (below) it, or a tenth of the lines when an empty band of half a text height sets it off from the lines below (above); every fragment from there outwards, within half a text height of the line, goes with it. The header and footer groups are cut as ordinary regions. - Specks between columns. A group of at most 2 fragments, and at most 3% of the fragments, that the 1.25 em cut leaves between two others in a region of at least 30 fragments (a mark in the margin, a noise word) is not a column when it lies inside the vertical extent of the nearer neighbour, at least 2 text heights from its top and bottom: it joins that neighbour. Without this it looks like a column embedded between parts of another and the page is read in bands. A menu item at the top of a screen is not inside anything and stays a group. When the cut leaves only two groups, one of them such a speck at the edge of the region (a mark in the left or right margin), the speck is cut off on its own and the text is cut into columns in its turn, skipping the band reading that a group embedded in another would bring: with it, the paragraph gaps two columns share cut the page in slices (left slice, right slice, next left slice). - Screens and forms (`ReadingOrder.Sections.cs`). When a region has empty vertical stripes and is not a table, three rules decide whether the columns are the structure, in this order: 1. *Header row*: the topmost band is read first, whole, when it is at most 2.5 em tall, has at least two fragments, spans at least 60% of the region's width and is separated from the rest by at least 2.5 em (a menu bar whose last item lines up with the body's right column). The label row of a field grid has a smaller gap and is not taken. 1b. *Sidebar*: the first column is read before the rest when it has at least three single-fragment lines at an even pitch (each gap within 30% of the median), is at most 35% of the region's width, the body starts within 25% of the width after it, and at most half of its lines (and half of the body's lines) share a line with the other side. A label column with its values has partners on both sides, or far off, and is not a sidebar. The menu bar, the sidebar and the cells of a row of fields are marked standalone: no detected table region merges them into a table. 2. *Tied rows*: the region is cut at empty bands of at least 1.5 em, bands being joined across a gap of at most 0.8 of both neighbouring gaps (the label above its value inside one cell). If the smaller column is not the first one, has at least three items in at least half as many bands, and every band holding one of them also holds other text, the bands are read in order and each is cut into its columns: a setting then its value, a field's label and value then the next field. A narrow first column (a sidebar) is read before the body instead. 3. *Embedded column*: if one column lies wholly between two parts of another (text at least 2 em above and below it), the columns are not the structure and the bands of rule 2 (or, without them, of half an em) are used (cards between a title and a list). - Fields. A label above its value is a cell; the cells of a row of fields (equal numbers of fragments in a label line and the line under it, each value starting within 2 em of its label) are read one cell at a time, left to right, not all labels and then all values. This is applied to the last two lines of a region (under at most one single-fragment heading) and to pairs of bands whose gap is at most 0.8 of the gap after the pair (before it for the last pair). Labels that end in a colon ("Total: 45.00") are list rows, not fields. - Rows of `label: value`. A column whose fragments mostly (60%) end in a colon is a label column. With one label column and one or more value columns, rows are read across even when there are only two, and uneven spacing does not make them fields. With several label columns (two blocks side by side) each label column and the columns up to the next form a block; the blocks are read one after the other, each across its rows. A run of bands is not merged into a section of columns when one of its columns is made of cells (one or two lines at a time, with gaps of at least 3 em between them). - A table needs more than most of its rows having several fragments: its columns must also be filled in 60% of the rows on average (short blocks side by side that share a few rows are not columns), no row of fragments may be 1.6 times taller than the median, nor set in a text size 1.6 times that of another row (a card's large number), and the distance between consecutive lines must not alternate short and long by a factor of 1.25 or more (a label above its value). These tests only remove tables that the earlier tests accepted. - A run of rows whose cells align in columns is a table block: `table` is true, each line is a row, and a `Table` entry carries its cells. Table `confidence` is the share of rows with more than one cell and is not calibrated. Tables drawn with ruling lines, and tables in a detected `table` region, get spans, empty cells (`words: []`) and multi-line cells; see `table-structure-v1.md`. The members are the same. In a table cut into rows by rules, a cell of several lines is read whole (its lines top to bottom) before the cell beside it, so the text of a row comes cell by cell, not line by line across the cells; the lines of one cell are joined by a space in the `Line.text` of the band's row. Word boxes are the ink extent inside the horizontal range the recognizer aligned the word to. Character boxes tile that range by the alignment of the network's output and carry `approximate: true`. ## Word segmentation The recognizer emits a space class where it believes there is a gap, and it is wrong in both directions: it cuts words (`immedi ately`, `$40, 181.65`, `2014- 02-15.`) where the ink is continuous, and it fuses words (`Iagree`) across an empty stretch. After a line is read and before its words get boxes, the reader re-decides the spaces from the line image. Only spaces change: every character, its probability and its extent stay as the network gave them, so numbers, amounts, dates and addresses keep exactly the characters that were read. A word whose characters do not change keeps its edges and confidence; a word made by joining or cutting gets text from the characters, edges halfway across the neighbouring gaps (the line's ends where the network first and last fired), and a raw confidence that is the mean of its characters' probabilities (calibrated as any word). **Evidence.** The scaled line image (the recognizer's input, 32 rows) gives the ink columns: the line's background is the median pixel value; `top` is the 99.5th percentile of the distance from it; a pixel is ink when its distance exceeds `(top + 1) / 2`, and a column is ink when any pixel in it is. A line whose `top` is under `MinContrast` (64) is left as read. An *empty stretch* is a maximal run of non-ink columns with ink on both sides (the margins are not stretches). A stretch belongs to a pair of neighbouring characters when it touches the span between the two characters' centres (a character's run of steps is not always inside its glyph); `widest[p]` is the widest stretch touching pair `p`. **Taking a space out.** A space the network emitted after character `p` is removed when `widest[p]` is small and one of: 1. *Number or date* (`widest[p] <= NumberGap` = 6): the two words are the halves of one number. The left word ends in a number (its longest suffix of digits, `,` `.` `/` `:` `-`, from its first digit) and the right word begins with a digit or a date separator, and either (a) the left number ends in `,` after `1-3 digits (,3 digits)*` and the right word begins with exactly three digits (`$40,` `181.65`); (b) the left number's last comma group has one or two digits and the right word begins with exactly the digits that complete three (`16,0` `80.65`); (c) the left number is only digits and `/ - : .` with at least two digits and ends in `/` or `-`, and the right word begins with a digit (`2014-` `02-15.`); (d) the left number holds a `/` or `-` and the right word begins with `/` or `-` followed by a digit (`24/07` `/2010.`). 2. *Letters* (only when the engine has a word model): the left word ends and the right word begins with a letter; with their outer non-letters set aside, neither piece has an upper-case letter other than the first letter of the left piece (capitals inside mark codes, abbreviations and headings, which the model cannot judge); when the line has at least `MinSpacesForGap` (4) network spaces, `widest[p]` is at most `ModelRatio` (0.7) times the median stretch at those spaces; with `Score` the word score of the character model below, `gain = Score(joined) - (Score(left) + Score(right))`, and the space goes when either - `widest[p] <= ModelGap` (5) and `gain >= ModelMargin` (2.0 nats): plain evidence; or - `widest[p] <= ModelGapWeak` (6) and `gain >= ModelMarginWeak` (1.0 nat) and the lower of `Score(left)` and `Score(right)` is at most `WeakPieceMax` (-2.0): weak evidence, enough when one piece is clearly no word (`Janu` + `ary`, `Fieldn` + `ote`). A piece that is a rare real word (`tax` scores about -1.7) is not bad enough, and neither are two pieces that both score as words: they need the plain evidence. A piece the model has no score for (digits, code symbols, letters outside the script) never joins: the code-word rule. 3. *Word frequencies* (only when the engine has a word frequency table, `word-lm-v1.md`): the same piece rules as 2 (letters at the cut, outer marks set aside, no capital inside a piece other than the first letter of the left piece) and, in addition, no digit and no code symbol (`@ # _ / \ = % + * ~ ^ | < > [ ] { }`) anywhere in either word, the marks around the letters included. Both pieces are lowercased by the table's folding, and `a`, `b` and `ab` are looked up (`q` is the stored quarter-bits of a count; a missing word has `q_missing`). The space goes when `widest[p] <= FrequencyGap` (9), the stretch is at most `FrequencyRatio` (0.85) times the line's median word gap (on lines with at least `MinSpacesForGap` network spaces), `ab` is in the table with `q(ab) >= FrequencyMinQuarter` (10), and q(ab) + q_total - q(a) - q(b) >= FrequencyMargin (28) + 24 if a piece is one letter + 16 if both pieces have q >= 24 The extras keep out pairs of real words that happen to make a word (`dare` `say`, `a` `gain`). Integer arithmetic only. Word breaks the network left out are put in by two further rules, after the ink-gap rule above (which needs a stretch of `SplitGap` (9) pixels and `SplitRatio` (3.5) times the line's letter gap): - *Vertical bar, middle dot and bullet.* A separator mark (`|`, `·` or `•`) and the letter, digit or closing mark before it, or the letter, digit or opening mark after it, are cut when the stretch between them is at least `BarGap` (5) pixels and at least `BarExtra` (2) more than the line's median letter gap (`Genealogy|` `Search`, `Alvarez·` `Oct`). The same address-word guard as for the ink-gap rule applies. Two separator marks in a row stay together. - *Fused capital I.* A network word that is a capital `I` followed by two or more lower-case ASCII letters is cut after the `I` when the stretch is at least `FusedIGap` (4) pixels and `FusedIExtra` (2) more than the line's letter gap, the rest is in the word frequency table with `q >= FusedIWordQuarter` (20; `agree` has 23) and the whole word, lower-cased, is not in the table with `q >= FrequencyMinQuarter` (10) (`Iagree` → `I agree`; `Index` and `Ian` are words and stay). It needs a word frequency table. 4. *Addresses, web addresses and amounts* (`widest[p] <= CodeGap` = 6 and, on lines with at least `MinSpacesForGap` network spaces, at most `FrequencyRatio` times the median word gap): none of these has a space in it, so a break goes where the two words are the halves of one. Only the text of the two words is judged (no word model or table is needed), by the first rule that applies: (a) the right word is `@` followed by a domain with a point inside it, and the left word is made of letters, digits and `. _ % + -`, ending in a letter or digit (`elena.nair` `@example.com`); (b) the left word holds `@` with a point after it and ends in a letter and a full stop, and the right word is two or more lower-case letters, a closing mark allowed after them (`dad@corp.example.` `net`); (c) the left word is made of letters, digits, points and hyphens, is longer than two characters, and ends in a full stop after a letter or digit, and the right word (a closing mark allowed after it) is one of `com gov org net edu io uk mil info biz app dev example`, all lower-case or all upper-case (`IRS.` `gov`; a sentence would continue with a capital; `us` and `co` are left out because they are words); (d) the left word is letters and `:`, and the right word starts with `//` (`https:` `//irs.gov`); (e) the left word holds `//` or starts with `www.`, ends in `/`, and the right word starts with a letter or digit; (f) the left word ends in `$`, `£` or `€` that no digit precedes, and the right word starts with two digits, or a digit and a point or comma (`$` `17,906.36`; a lone digit is not an amount). Characters never change: only the break is taken out. 5. *Contractions and possessives* (`widest[p] <= CodeGap` = 6): the left word ends in a letter and the right word is an apostrophe (`'` or `’`) followed by one of `s t d m ll re ve`, closing marks allowed after it (`brother` `'s`). Nothing else starts with an apostrophe and a bare ending, so no model is needed. Rules 1 to 5 are tried in that order and any one of them takes the space out. **Putting a space in.** A stretch that no network space accounts for (no pair it touches has a network space) becomes a space at the pair it overlaps most when its width is at least `max(SplitGap, SplitRatio x letterGap)` (`SplitGap` = 9 pixels, `SplitRatio` = 3.5, `letterGap` the median `widest` over the line's pairs inside words, so letter-spaced print is not cut) and the two characters may be separated: both sides are letters, digits or marks that can end or start a word (a closing mark `. , ; : ! ? ) ] } % ' " » …` and its quotes may end a word, a currency sign or an opening bracket or quote may start one), not two digits, not a point or comma that follows a digit, and not inside a word that contains `@` or `//`. **Word score.** `Score(letters)` (letters only) is the sum, over each letter and the word end, of the character model's `LogProb(context, symbol) - MeanLogProb` floored at the decoder's floor (-1), with the case-folded symbols of the script that covers the most of the word's letters (the first on ties; no score when it does not cover all of them) and the word start as the first context symbol, as in `decoding.md`, in `double` in that order. A typical word scores about zero. The word model is the model file that ships with the models package (`charlm-v1`); an engine without one skips rule 2 above and nothing else. The frequency rule's parameters were chosen on the held-out tune sets (thirteen sets of generated and real-web pages) and checked on five newly generated sets that were not looked at while choosing; no set got worse in CER, WER or reading order. The first parameters were chosen on pages that are not part of the benchmark sets (a degraded-scan set and a screenshot set rendered with new seeds). The weak-evidence values were chosen on the held-out tune sets, by counting joins that the truth text confirms against joins of two real words over every candidate space the engine met there; the benchmark sets were counted in the same tables as information and agree. `OcrOptions.FixWordSpacing = false` turns the whole step off. Pages from a PDF text layer or an LLM are not re-segmented. ## Digits and the letter O After word breaks are settled, the one step that changes a character: the recognizer reads `0` as `O` in identifiers and dates (`ACC-71O3O5`, `2020-04-O7`), and the neighbours of the letter settle which it is. For each word: 1. A word with fewer than `MinDigits` (3) ASCII digits is left as read (`O2`, `No.7`, `COVID`). 2. The word is cut into runs of ASCII letters and digits (the pieces between punctuation). A run that holds only digits and the capital letter `O`, with at least one digit and at least one `O`, has each `O` replaced by `0`. A run with any other letter (`ACC`, `CO2`, `COVID19`) or without a digit (`OO`) is left alone, and so is a run that is not separated from letters by punctuation (`ACC71O3O5`). Characters keep their extents and probabilities; the word keeps its confidence and box. Only the text of the word and of the changed characters differ. The rule is deliberately narrow: over about 100,000 truth words of the benchmark and tune sets (receipts included, with their many `0`s) it changes none, and on the engine's output it changes five words, all corrections. It was written from the three `O`-for-`0` errors of two benchmark pages and checked on the held-out tune sets, which hold two more. `OcrOptions.FixDigitLookalikes = false` turns it off. Beam decoding with the character model keeps words that contain digits exactly as greedy decoding reads them (`decoding.md`, "code-word rule"); this step runs afterwards on either result, so the two do not conflict: the code-word rule decides what the decoder may change, this step is a separate post-recognition fix. The reverse (`l` or `I` for `1`) is not handled: the sets hold no such error. ## Separators The recognizer often reads the middle dot, the bullet and the vertical bar between menu items, breadcrumbs and bylines as a hyphen or another small mark (`By Ana Costa - 3/2/2021 - 4 min read`). After word breaks are settled, each one-character word between two other words of a line is judged again from the pixels of the scaled line image. Like the digit step, this changes characters only; the word keeps its box, confidence and per-character extents, and nothing is added or removed. All arithmetic is on integers, so the result is the same on every path and at every degree. **Candidates.** A word of exactly one character, neither first nor last in its line, read as one of `- – — . , ' * ° ~ _ : ; ^ + = ¨ ˙ ‧ ∙ ⋅ ● ○ ▪ ◦ ¤ º ª` or the grave accent (a dot), `·` or `•` (a dot the network already decided), or `I l 1 ! ¦ / \ ( ) [ ]` (a bar). Other letters and digits are never candidates for a dot, and a word of two or more characters (`-x`, `--`, `a-b`) is never touched, so hyphens and dashes inside words stay. The first and the last word of a line (a list marker, a trailing mark) are candidates only when read as one of `º ª ° ˙ ‧ ∙ ⋅ ● ○ ▪ ◦ ¤`, `·` or `•`, and only for the dot test below: a hyphen, a full stop, a letter or a bar mark there is left as read. A line needs at least two words. **The glyph.** Ink pixels are those the word segmenter uses (distance from the line's median grey above half the line's strongest contrast; lines fainter than `MinContrast` are left as read). The glyph is the run of ink columns that contains the first ink column of the word's extent, grown over every adjacent ink column, so an extent that cuts the glyph short does not matter. A run wider than the line image is high is not a separator. In it, the largest 8-connected piece must hold at least 80% of the ink; its bounding box is `w` by `h` with `area` ink pixels. The reference is the line's other ink columns, leaving out the columns of all candidates (a row of bars is no reference for itself); at least 8 are needed. `L` is the median over them of the column's ink height (top to bottom ink row), `mid2` the median of top + bottom, and `tallest` the largest ink height. **A dot or bullet** (the candidate is not a bar mark) when all hold: - `w` and `h` are at least 1; `45 <= 100 w / h <= 160`; or, for a blob at most 25% of `L` wide, `w <= 2 h`. A hyphen or dash the network read with confidence 0.8 or more needs `100 w / h <= 130`: a short hyphen and a dot look alike, and the network is usually right about a hyphen it is sure of. - `area >= 0.55 w h`. - The blob's centre (`2 * top + h - 1`, twice the row) is within 25% of `L` of `mid2`. - The diameter `(w + h) / 2` is between 10% and 90% of `L`. The result is `•` when the diameter is at least 38% of `L`, else `·`. A word the network read as `·` becomes `•` only from 45%; one read as `•` becomes `·` only up to 28%; otherwise the network's own mark stays. **A bar** (`|`) when `w <= max(2, 0.15 h)`, `area >= 0.8 w h`, `h >= 1.12 *` the tallest other ink column and `h` is at least half the line image's height. A capital `I` or a `1` is not as tall as a vertical bar, which reaches from above the ascenders to below the baseline, and is thicker. The constants are in `SeparatorGlyphs` and were set on held-out screenshot, layout and generated interface-text pages; they are not tuned on the benchmark. `OcrOptions.FixSeparators = false` turns the step off. ## Dot leaders Contents pages, lists of figures and forms join a title to a page number or a line number with a run of full stops. The recognizer reads text, and a long run of dots in the middle of a line makes it lose the words in front of the run and invent letters. So a line is searched for a leader from its pixels before it is read; a line with one is cut at the leader, its two parts are read apart, and the leader is put back as words, the way a text layer has it. **Finding.** In the line's ink box (8-connected components of the page's ink at the line's own threshold; more than 20,000 components means the line is not searched; lines narrower than 3 line heights are not searched), components whose box covers 3 pixels or fewer are grain and are left out (they stand between dots in a scan and must not end a run). A *dot* is a component 2 pixels or more on each side, at most 0.3 of the line height wide and high, with width/height between 0.5 and 2, in the lower half of the box. Neighbouring dots in left-to-right order form a run while each has the first dot's baseline (within max(2 px, 0.3 of its size)) and size (ratio 0.6 to 1.7) and sits one pitch from the last, within 30% of the run's pitch (the second dot: 1.3 dot widths to 2 line heights). A pair of dots that does not grow into a run of at least 3 uses up neither of its dots: the next candidate starts at the second one (a full stop and the first dot of a leader pair up under the wide limit, and the leader must still start at its first dot). A gap of two or three pitches between two runs of the same pitch is a dot too faint to see, and the runs join. A leader has at least 6 dots, 6 of them seen, a pitch of at most 2 line heights (stops set half an em apart are about 1.5 line heights apart when the box holds no descenders) and at least 1.3 mean dot widths. When its first dot touches the ink in front of it (less than half a pitch away), that dot is the sentence's full stop and is not counted. A run that reaches the right edge of the box continues: the search window grows by 4 line heights until the run ends inside it, bounded by the page width. The ink left of the run and the ink right of it (up to the first gap wider than 1.5 line heights) become the boxes of the two parts. Only upright lines of the ordinary finder are searched; tilted lines and lines recovered from faint text are read as before. Two checks keep small text from being taken for a leader. Other components in the box are *strays* when they are no bigger than 1.5 times the dots (plus a pixel) and stand on their baseline, and *letters* otherwise; components of other runs of dots count as neither. A run with more strays than max(2, a quarter of its dots) is a few letters of a line of text set as small as dots, and is not a leader. A box shorter than 0.6 line heights is taken to hold the leader alone (the detector cut it into a strip of its own); it may hold at most max(4, a third of the dots) letters. **Faint leaders.** On a page whose typical line is at most 48 pixels high (up to about 200 to 250 dots per inch), the dots of a leader are one or two pixels wide and blur takes their middle below the page's threshold, so the search above does not see them; and the detector, which looks at a shrunk page, often boxes the text up to the dots and not the dots. For each upright line in which the search found nothing, the search is made again over the line's box and 16 line heights past its right end, with the threshold halfway from the page's threshold to the paper, but not past the left edge of the next strip on the same row (a strip lying mostly right of the line's middle and sharing more than half of the smaller height), which is read on its own. The rules for a leader are the same. The line's own box up to the first dot is the left part. Pages with taller lines do not make this second search. **Reading.** The crop of each part has the columns of the leader (and of the other part) painted with the crop's median grey before it is scaled. Detector strips that lie inside an extended leader are not read again. Pages and lines without a leader take the old path and are byte-identical. **Words.** The leader is one word of as many full stops as the run has dots, with the confidence 0.995 (counted from pixels, no network read it) and a box around the run. If the page number starts within 2.5 pitches of the run, it is part of the same word (`.........12`, as the text layer of a typeset contents page is). Full stops read right of the run are dropped. A run whose pitch is at least 0.37 of the line height is *spaced* (`. . . .`: a full stop and a space advance about half an em, a full stop alone about a quarter of one): each dot is a word of its own, and the number stays a word of its own. Words from the two parts are joined in reading order. The constants were set on generated contents pages, lists of figures and tables in training-pool fonts, in both styles, at 300 dots per inch and again reduced to 150 and 200 with blur and JPEG. The step reads pixels only and adds no parser. ## Noise filter Before lines are arranged, the reader drops detections that are noise read as text: specks, scan grain and edges that were cut as tiny lines and read as a few unsure characters, and lone dots and ticks. Two rules work on the recognized strips (one strip is one detected line, its height is the height of its ink box). *Small unsure words:* 1. A strip is a reference line when the mean calibrated confidence of its words is at least `AnchorConfidence`. The reference height is the median ink height of the reference lines. With fewer than `MinAnchorLines` reference lines the page is left alone: there is no text to be small against. 2. A strip is small when its height is under `min(RelativeHeight x reference, MaxHeight)` pixels. Small is relative to the page's own text, so a page set entirely in small type keeps everything; the pixel limit keeps proportionally small text on high-resolution pages. 3. In a small strip every word whose calibrated confidence is under `MaxConfidence` is dropped. In a tiny strip (height under `min(RelativeHeight x reference, TinyHeight)`: specks and grain a few pixels high) the bar is `TinyConfidence` instead, because such strips hold no real text but the recognizer reads a lone punctuation mark in them with middling confidence. A strip left with no words is dropped. Words in other strips are never touched by this rule, whatever their confidence. *Dots and ticks* (after the first rule, in strips of any height): a word made only of the characters . , : ; ' " ` ´ ‘ ’ “ ” (a speck read as a mark) is dropped when its calibrated confidence is under `SpeckConfidence`, but only on a page that has at least `SpeckPageMin` such words: dust comes in numbers, while a clean page has none or a stray one, and the stray one may be a real mark. Dashes, bullets, middle dots and other separators are words of their own on real pages and are never dropped by this rule. *Controls* (any strip height): radio buttons and checkboxes are drawn rings and squares that the detector finds as lines of their own and the recognizer reads as a letter or a letter and a bracket. A word whose text is one character from `O Q L D C E Г [ ]`, such a character followed by one from `] ) [ l I |`, or such a character followed by one of `l I | 1` and a closing bracket `] )` (a box with a stroke inside, read as `Dl]`), is dropped when its calibrated confidence is under `ControlConfidence` and it is alone in its strip, or first or last in it with a gap to its neighbour of at least half its own height (a control sits apart from its label; a letter in running text is a word space away). A word that ends in a closing bracket is shaped like nothing else on a page, so it needs only `ControlBracketGap` of its height (a checkbox's label often starts closer than half a height). A letter read with confidence (a grade, a column label) stays, as does any word with a letter outside this set (`Do`, `OK`) and a digit. *Strokes* and *contained fragments* below apply only on a clean page: one where at least `CleanPageShare` of the characters are in words read with a calibrated confidence of 0.9 or more. On a photo of a receipt or a noisy scan thin real characters (`1`, `I`, `T`) are read with little confidence as well, and dropping them costs recall. *Strokes* (any strip height): a table rule's edge, a letter's stem or a line at the page edge is cut by the detector as a thin line of its own, and the recognizer reads it as one narrow character (`1`, `I`, `l`, `]`) with the lowest confidence it gives. A word of one letter, digit or bracket (`[ ] ( ) |`) is dropped when its calibrated confidence is under `StrokeConfidence` and its box is at most `StrokeAspect` times as wide as it is high. Dashes, dots and bullets are not strokes. A real narrow character (a lone `I` or `1`) is read with confidence and stays. *Contained fragments* (any strip height): when a word of at most `ContainedMaxLength` characters has at least `ContainedFraction` of its box area inside the box of a word with more characters, it is the same ink read twice (the detector cut a stem or a letter of the word as a line of its own: `p` over `perfect`, `(` over `(with`). Two printed words never share a box like that, so the short word is dropped, whatever its confidence. Candidates are found through a grid with 128-pixel cells, so a page costs one pass. *Repeats* (any strip height, any page): when a word has at least `RepeatFraction` of its box area inside the box of another word, has at least two letters or digits, and its letters and digits (case-folded, marks left out) are a contiguous run of the other word's, it is the same ink read again (the detector boxed a stretch of a line a second time, so `firea` and `ms.` appear over `firearms.`, or a second `in` over `in`). The repeat is dropped; of two readings with equal letters the less confident goes (the later one when equally confident). A misread repeat (`arn` for `arm`) is not a run of the word's letters and stays. Same grid as above. Unlike the two rules above it does not wait for a clean page: it needs a shared box and shared letters, which two printed words never have. The parameters are `MaxConfidence` = 0.5, `RelativeHeight` = 0.6, `MaxHeight` = 12 px, `TinyHeight` = 8 px, `TinyConfidence` = 0.99, `AnchorConfidence` = 0.9, `MinAnchorLines` = 3, `SpeckConfidence` = 0.9, `SpeckPageMin` = 3, `ControlConfidence` = 0.9, `ControlBracketGap` = 0.2, `StrokeConfidence` = 0.2, `StrokeAspect` = 0.5, `ContainedFraction` = 0.6, `ContainedMaxLength` = 1, `RepeatFraction` = 0.6, `CleanPageShare` = 0.95. The control rule's glyph set and bar were read off the unmatched words of two benchmark sets (form and dialog screenshots, form pages) and then checked on held-out screenshot and layout sets rendered with other seeds, where the same glyphs appear. The other values were chosen on pages that are not part of the benchmark sets: a degraded-scan set and a screenshot set rendered with new seeds, and the validation split of the generated page set. The dropped words never reach reading order, tables, `text` or the JSON; the page gets one diagnostic stating how many words were dropped. The stroke and contained-fragment rules and the bracket gap were found on the unmatched words of held-out layout and screenshot sets (pages rendered with other seeds than the benchmark sets), where most of the extra words on a clean page are these three kinds. Together the new rules removed 80 words from the four held-out and two benchmark synthetic layout and screenshot sets, all of them extra or wrong; 108 from real-PDF pages, 11 of them right (form `X` marks and a capital letter); and 36 from degraded, phone and receipt pages, none right. Three values were set after reading dropped words on benchmark development pages, not on held-out sets: the confidence bound of 0.2 and the clean-page gate (the first version lost recall on receipt photos), and the one-character limit of the contained-fragment rule (two-letter words such as `If` and `be` were dropped on real forms). `OcrOptions.DropNoise = false` turns the filter off. Pages from a PDF text layer or an LLM are not filtered. ## Stray accents On a badly scanned page a speck above or below a letter is read as part of it: the network writes `thé` for `the`, `dọwn` for `down`, `budġet` for `budget`, usually with middling confidence. Right after recognition, before the noise filter, a page-wide pass takes such marks off. It changes characters, never boxes or confidences, and it is part of the noise cleanup (`OcrOptions.FixStrayAccents`, default true, does nothing when `DropNoise` is false). It needs the word frequency table (`word-lm-v1`); without one nothing changes. *Page gate.* Count the words that hold a letter, and among them the words that hold a precomposed Latin letter with marks (a letter whose canonical decomposition is an ASCII letter followed only by combining marks). When the second count exceeds `AccentPageShare` of the first, the page is in a language that uses accents and nothing changes. *Word rule.* A word changes when all of these hold: 1. Its calibrated confidence is under `AccentConfidence`. 2. Trimming characters that are neither letters nor digits from both ends leaves at least two characters, and every one of them is an ASCII letter or a precomposed Latin letter with marks. A digit, a hyphen or apostrophe inside, a letter of another script and a letter without a plain form (`ß`, `æ`, `œ`) leave the word as it is. 3. The lowercase word with its marks taken off is in the frequency table, and its count, in quarter-bits (four times the base-2 logarithm), is at least `AccentMargin` above that of the lowercase accented spelling (a word the table lacks has the table's missing value). Eight times as frequent is `AccentMargin` = 12. Each marked letter is replaced by its ASCII letter in the case it had; the word's characters (when kept) change the same way. The page gets one diagnostic with the number of words changed. The decomposition comes from the pinned normalization tables (`text-normalization-v1`), so the result does not depend on the platform's globalization settings. Values: `AccentConfidence` = 0.97, `AccentMargin` = 12, `AccentPageShare` = 0.12. They were fixed in advance and kept after trying margin 8, a bound of 0.99 and no page gate on three held-out degraded scan sets, where the choice moved the word error by under 0.02 points. The page gate never fires on the degraded sets used, so its value is a precaution for accented languages and was not tuned. ## Compatibility A reader of version 1 accepts documents with additional members. Documents written by a future version 2 are rejected with the version in the message. # Local Markdown, version 1 `OcrMarkdown.ToMarkdown(page, options)` (and `image.ToMarkdown()`, and the same on PDF pages and documents) writes a page of the page model as Markdown without a model or a network. The text is a function of the page and the options only: no clock, no random choice, no culture. It is written for CommonMark with pipe tables (GFM). ## Elements The blocks of the page are visited in reading order and each gives zero or more elements, separated by one blank line. The output ends with one line feed, or is empty when nothing was written. Pages of a sequence are separated by a thematic break (`---`) between pages that have content. Roles come from the page: `Block.Role` (header, footer), the first region that lists the block in `BlockIndices` (title, text, list, table, figure, header, footer, …), and table entries. A page read without the page detector has no regions and no block roles; everything below works without them, using sizes and positions. 1. **Furniture is dropped.** A block with the header or footer role, or in a header or footer region, is left out unless `IncludeHeadersAndFooters`. With `DropPageNumbers` (default), a block of one line that is only a page number (`7`, `Page 7`, `7 of 12`, `7 / 12`, `- 7 -`, one to five digits) and lies in the top or bottom 8% of the page is left out too. Not done on pages that were turned or straightened. 2. **Tables.** A table entry whose `BlockIndex` names the block is written as a pipe table in the block's place; tables without a block go before the first element that starts below them. Cell text is the cell's words (by `WordIndices`) joined by a space. The first row is the header row. **Spans:** a cell's text goes into its top-left position; the other positions it covers stay empty. Columns in which every cell is empty are left out. `|` in cell text is written `\|`. A block flagged as a table that has no table entry becomes a table of its lines, cut into cells at `Line.Cells`. 3. **Headings.** A block in a title region is a heading. Without a region, a block of at most two lines, at most 160 characters, whose median line height is at least 1.25 times the page's body line height, that does not end with `,` or `;` and does not start with a list marker, is a heading (`HeadingsFromSize`; blocks that sit in any other region never are). The body line height is the median over the lines of body blocks that are not tables, titles, headers or footers. **Levels** follow size: the heading heights of the page are grouped (a height within 8% below the group's largest is in it); the largest group is `#`, the next `##`, down to `######`. Line height is the length of the quad's left edge for turned text, otherwise the box height. 4. **Lists.** A line that starts with a bullet (`•`, `●`, `○`, `◦`, `▪`, `▫`, `■`, `□`, `‣`, `⁃`, `∙`, `·`, which may be followed directly by the text), or `-`, `*`, `+`, `–` followed by a space, or one to three digits followed by `.` or `)` and a space (or the same in parentheses), starts a list item. Bullets are written `- `; numbers are kept as printed and written `N. `. A line without a marker that is indented by at least 0.3 line heights past the marker's left edge, and no more than 1.5 body line heights below the item, continues the item (joined with a space). **Nesting:** an item whose left edge is more than 0.8 line heights to the right of the current level's starts a new level; one more than 0.8 to the left returns to a level that fits. Each level indents by four spaces. Pages with turned text have one level. Lettered and Roman markers are not recognised. 5. **Paragraphs.** The remaining lines of a block make paragraphs: lines are joined with one space (cells of a line, which are separated by a tab, by one space as well), and a vertical gap of more than 0.75 body line heights between two lines starts a new paragraph. A hyphen that ends a line and follows a letter is removed, without a space, when the next line starts with a lower-case letter (a hyphenation break); otherwise lines join with a space. 6. **Escaping.** `\ ` * _ [ ] < >` are escaped with a backslash wherever they occur. At the start of a paragraph, `# + - = ~` are escaped too, and the `.` or `)` of a leading `123. ` / `123) `. Headings and list item text escape the first set only. ## What it does not do No emphasis (bold, italics), links, images, code blocks, footnotes, math, definition lists or lettered lists. Figures contribute only the text read inside them, as paragraphs. Table cells hold one paragraph of text (no breaks). Multi-column text is whatever reading order gave: one column after the other. # Table structure v1 How the local reader turns a table on an image page into a `Table` of the page model: cells with row and column spans, empty cells, and the words in each cell. No member of `oneocr.page` v1 changes; this document says what the existing members hold for a table the reader recognized. It is deterministic: integer pixel arithmetic and fixed tie rules, independent of kernel path, degree and operating system. ## Ruled tables A table drawn with ruling lines is read from the lines, before any text finding, on the page's own binarization (Otsu; ink is the smaller class). 1. **Runs.** One pass over the page collects horizontal and vertical runs of ink at least 24 pixels long. More than 2^17 runs of either direction means the page is noise, and no table is looked for. 2. **Lines.** Runs in neighbouring rows (or columns) that overlap by at least 60 % of the shorter one join into a line. A line is a rule when it is at most 12 pixels thick and at least 10 times longer than thick (the strokes of large letters are not slender enough). At most 4096 lines per direction are considered. 3. **Grids.** Rules that touch (the end of one within `max(3, 2 × thickness)` pixels of the other) form a component. Rule positions closer together than `max(3, 2 × thickness)` are one boundary; a boundary's position is the mean of its rules' centres. A component is a table when it has at least two rows and two columns of at most 2^16 cells in all, every row and column is at least 16 pixels, at least two rules each way run 60 % of the table's width (height), and at most 40 % of the pixels in its box are ink (a light panel read as ink is not a table). 4. **Cells.** The grid has one elementary cell per row and column. An edge between two neighbouring elementary cells is *drawn* when rules at its boundary cover at least 60 % of the edge, ignoring `thickness + 1` pixels at each end. Elementary cells whose shared edge is not drawn are one cell; a group that is not a rectangle falls apart into elementary cells. That gives the cell's `row`, `column`, `rowSpan` and `columnSpan`. A cell's `bounds` are the rectangle between its boundaries. 5. **Reading.** The rules are painted out of the page (each rule pixel and its fringe takes the paper value next to it), text lines that cross a drawn vertical rule are cut at the rule, and every piece is read on its own, so a word is never read across a column. Text belongs to the cell that holds the centre of its fragment. **A grid that is only part of a table.** A component of rules is not taken as a grid when at least two of its horizontal rules run on beyond its outermost vertical rule on the same side by more than `FragmentOverhang` (0.25) times the grid's width: the table has columns whose rules are faint or not drawn, and a grid of the verticals found would leave them out. Nothing is erased or cut for such a component; the table is read as a table without a rule grid (next sections), with the horizontal rules cutting its rows. The table's block holds one line per row that has text; a line's cells are the table cells with text in that row, left to right (`Line.Cells` marks where each cell's words start); a cell with several lines of text lists them top to bottom, joined by a space in the line's text; a cell with a span is listed once, in its first row. `Table.Rows` and `Columns` are the grid's; `Cells` lists every cell in row-major order, empty ones with `words: []`. `Table.Box` is the grid's box; `Confidence` is 0.95 (not calibrated); `Source` is `ocr`. ## Tables without a rule grid For a table block the reading order found (rows with several aligned fragments), and for the blocks inside a detector region of class `table` (merged into one table whose rows are the page's text lines, so that columns found as separate blocks read row by row): - **Columns** are the stretches of text between separators. A separator is a stretch of at least half a text height that is free of text in all but a quarter of the rows (a spanning header crosses it). A table needs two columns; with fewer the block is left as the reading order made it. A learned source of separators can replace this analysis (`ITableSeparators`, internal); none ships. - **Rows** are the text lines. A cell that wraps onto a second line is two rows. - **Cells.** A fragment belongs to a column when it covers at least 30 % of the smaller of itself and the column's text; a fragment that belongs to several columns is a cell with a column span. Fragments that reach the same column in one row are one cell. A column with nothing in a row is an empty cell (`words: []`) over the column's extent. - `Confidence` is the share of rows with more than one cell, as before. The text of the page does not change. ### Rows cut by rules, cells of several lines Where the page has horizontal rules across the table, the rows are not the text lines but the **bands** between the rules, and a band is read cell by cell. (Page rules are the horizontal rules of the ruled-table finder, found whether or not they form a grid.) - **Separators.** A rule separates two bands when it lies between the table's first and last fragment and its extent overlaps the table's text extent by at least `BandRuleShare` (0.6) of its width: a rule under one column is the border of one cell, a short rule an underline. Rules closer together than `BandRuleMerge` (0.5) times the median fragment height are one separator. A table with no such rule is read exactly as before. - **Bands.** Each fragment goes to the band that holds the centre of its box, so the lines of a cell stay together even when a cell of another column ends higher or lower. A band is a row of the table (`row` of its cells, `Table.Rows` counts bands). - **Columns** come from the gaps as above, with the bands as the rows: a gap is free when at most a quarter of the bands have text across it. - **Cells.** A band is read cell by cell only when it holds cells of several lines. It must use at least `MinCellColumns` (3) columns (two columns are mostly a name and its value, and cells that wrap in both are rare), and its lines must not run across its columns together: with `present` the number of (line, column) pairs that have a fragment, `lines` the band's lines (fragments sharing a baseline, across columns) and `used` the columns with text in the band, a band with `present >= AlignedShare (0.75) * lines * used` is a set of rows that the rules happen to fence together (a list of a name and a price on every line) and is read line by line, each line a row. Otherwise every fragment belongs to the cell of the first column it reaches, so all fragments of one column of the band (the cell's lines) are one cell, listed top to bottom, and the cells are listed left to right. A band in which a fragment spans columns that a fragment of another line also reaches cannot be split into cells; it is read by its lines, as without rules. A band holding one line is the same row as without rules. - **Text the reading order cut out.** A block of a table region that the reading order marked as a field of a form (a wrapped paragraph in a cell reads like one) is part of the table when rules cut the table into bands and no cutting rule runs through the block's text; without rules, or when a rule runs through it (a label above a field's box and the value in it), such a block stays out, as before. Blocks are merged only if the merged table has a band read cell by cell that holds a fragment of a merged block; otherwise the table is built without them (a form's labels and input boxes are bands too, and read line by line they would lose the label–value pairing of the fields). A word that the recognizer read across a column gap stays one word and goes to one cell. Words are never split to fit a grid. ## Order and downstream Reading order inside a table is row by row, cell by cell, header first. The table block takes the place in the page's block order of the first block below the table that overlaps it horizontally. `OcrPage.ToText` joins the cells of a line with a tab; Markdown writes the `Table` entry as a pipe table (spans leave the covered cells empty). # Text normalization v1 The library normalizes text in three places: the extraction model's character ids (NFC), the grounding fold (NFKC) and the training agreement check (NFC). The runtime's `string.Normalize` cannot be used for this: under invariant globalization it leaves non-ASCII text unchanged, under ICU it composes, and ICU's Unicode version follows the OS. Normalization is therefore our own and reads no data at run time. A port must produce the same strings. ## Forms NFC and NFKC exactly as UAX #15 defines them (https://www.unicode.org/reports/tr15/), with the default (not stream-safe-limited) behavior: 1. **Decompose** each code point: Hangul syllables arithmetically (Unicode Standard 3.12), everything else through the tables, recursively and to the end. NFC uses canonical decompositions; NFKC uses compatibility decompositions, which include the canonical ones. 2. **Reorder** each run of code points with a non-zero canonical combining class by that class, stably. 3. **Compose**: walk the result; a code point combines with the last starter before it when no code point between them has a class greater than or equal to its own (adjacent always qualifies) and the pair has a primary composite. The composite replaces the starter. Primary composites are the canonical two-code-point decompositions minus `CompositionExclusions.txt`, singletons and non-starter decompositions; Hangul L+V and LV+T compose arithmetically. Both forms are idempotent. Text already in the form is returned unchanged (the same string instance in C#). ## Coverage and pass-through The tables are generated from the full `UnicodeData.txt` of **Unicode 17.0.0** (`UnicodeData.txt` SHA-256 `2e1efc1dcb59c575eedf5ccae60f95229f706ee6d031835247d843c11d96470c`, `CompositionExclusions.txt` SHA-256 `2f239196ef3b5b61db5cc476e9bd80f534d15aa1b74e1be1dea5d042a344c85f`): 403 combining-class ranges, 2,081 canonical and 3,849 compatibility decompositions, 961 compositions. That covers every character of charset-v1 (Latin, Greek incl. polytonic, Cyrillic), every combining mark that composes into them, and the compatibility characters the NFKC fold depends on (ligatures, full-width forms, superscripts, no-break and other spaces, fractions, roman numerals). A code point the tables do not list (unassigned, or added after 17.0.0) has combining class 0, no decomposition and no composition, and **passes through unchanged**. An unpaired surrogate is such a code point: it is copied as it is and never causes an error. Noncharacters pass through too. The pinned version changes only with a spec revision; a newer Unicode never alters the result of text that already normalizes under 17.0.0 except for newly assigned characters. ## Regenerating `dotnet run --project tools/NormalizationTables -- src/OneOcr/Unicode/NormalizationTables.g.cs` with the two files in ``; the tool checks their SHA-256 against the pinned values above. ## Quick paths NFC returns its input when every UTF-16 unit is below U+0300; NFKC when every unit is below U+00A0. These are optimizations; the result is the same as running the three steps. ## Checks The test suite compares both forms against the runtime's ICU-backed `Normalize` over every Unicode scalar value, every base in Latin, Greek, Cyrillic and Latin Extended Additional / Greek Extended with every combining mark (and with every ordered pair of the marks Vietnamese and polytonic Greek stack), whenever it runs with ICU; the same inputs are asserted in both modes by the explicit cases. # Page orientation and deskew, version 1 What the local reader does to a page image before it looks for text lines, so that scans fed in sideways, upside down or crooked are read like upright ones. Every step uses integers or the IEEE basic operations (`+ − × ÷ √`), so the result is the same on every machine, kernel path and degree of parallelism. A port reproduces the numbers below. The steps run in this order, each on the 8-bit gray page: 1. **Orientation** (one seam, two sources): how many quarter turns clockwise (0–3) make the text upright. With a page detector loaded, its orientation head decides and the classical decision below is not run; without one, the classical decision runs. 2. **Quarter turn**: the page is turned by exactly that many quarter turns (pixel moves, no resampling). Both sources use the same rotation step (`(x, y) → (h − 1 − y, x)` per clockwise quarter). 3. **Skew**: the tilt of the text lines is estimated on the turned page. 4. **Deskew**: when the tilt is large enough to act on, the page is turned by it on a larger canvas, with bilinear sampling. 5. The turned and straightened page is read as usual; every box is then carried back to the input image (below). 6. **Angle search** (only when the first reading is weak, below): the page is read again at other whole-page angles and the best reading is kept. `OcrOptions.DetectOrientation`, `OcrOptions.Deskew` and `OcrOptions.SearchAngle` (all default true) switch steps 1–2, 3–4 and 6 off (with a detector, `DetectOrientation = false` also stops the head from turning the page). A page that needs neither (no turn, tilt under 0.3°) is read from the very same pixels as with both switched off, so its text is the same. ## Ink grid The estimators work on a coarse count of ink pixels. Ink is the minority class of Otsu's threshold (the same binarization the line finder uses; a page with no contrast has no ink). With `f = ceil(max(width, height) / 1200)`, cell `(cx, cy)` holds the number of ink pixels in the `f × f` block at `(cx·f, cy·f)`. A grid shrunk by 4 adds up 4 × 4 cells. A quarter turn clockwise of a grid maps cell `(x, y)` to `(height − 1 − y, x)`. ## Skew (projection profile) After Postl (1986) and Baird (1987). The ink of cells that are not solid (`0 < count ≤ max(1, ⌊0.6 · f²⌋)`) and lie outside a 2% border on every side is projected on the axis perpendicular to a candidate line direction. For a slope `k / 1024` (lines descend by `k` per 1024 across), the cell `(x, y)` goes into bin `⌊(1024·y − k·x) / 1024⌋` (arithmetic shift) and adds its count. The sharpness of a candidate is the sum of squares of the bins; text lines pile ink into few bins, a wrong direction smears it. - **Coarse search**: on the grid shrunk by 4, `k = −280 … 280` in steps of 2 (±15.3°); the best sharpness wins, ties go to the smaller `|k|`. - **Fine search**: on the full grid, `k` from `coarse − 6` to `coarse + 6` in steps of 1 (and 0, to report the gain); same tie rule. - **Gain** = sharpness at the best `k` ÷ sharpness at 0. **Contrast** = `best · bins ÷ (Σ counts)²`, where `bins` is the grid height. Contrast is 1 for evenly spread ink and large for ruled lines; it is comparable between a page and the same page turned a quarter. - The tilt is acted on only when `|k| ≥ 5` (0.28°) and gain ≥ 1.03, and only when the lines run across the page: the contrast of the grid turned a quarter must not exceed the grid's (otherwise the slope is noise from vertical text). The tilt in degrees is `atan(k / 1024) · 180/π`, computed with the series `x − x³/3 + x⁵/5 − …` (16 terms, in this order, doubles). It is reported as `OcrPage.SkewDegrees`: positive when the lines descend to the right (clockwise in y-down coordinates). ## Deskew With `t = k / 1024`: `cos = 1 / √(1 + t²)`, `sin = t · cos`. The turned page `Wq × Hq` is put on a canvas of `⌈Wq·cos + Hq·|sin|⌉ × ⌈Wq·|sin| + Hq·cos⌉`; canvas pixel `(i, j)` samples the page at ``` dx = i + 0.5 − Wd/2 dy = j + 0.5 − Hd/2 qx = dx·cos − dy·sin + Wq/2 qy = dx·sin + dy·cos + Hq/2 ``` with bilinear interpolation between the four pixel centers around `(qx − 0.5, qy − 0.5)` (the page edge is clamped), the result rounded half up to 8 bits. Points outside the page take the page's most common gray value (lowest value on ties). If the canvas would exceed `OcrLimits.MaxImagePixels` the page is read as it is and a diagnostic says so. ## Orientation: the classical decision `IOrientationDetector.Detect` returns 0, 90, 180 or 270 (degrees clockwise to turn the page upright). A model-provided detector can replace the classical one behind this interface. The classical decision: 1. **Line direction.** Contrast of the grid and of the grid turned a quarter clockwise. If the turned grid's contrast is at least 3× the grid's, text runs vertically: the page is a quarter turn off (candidate axis 1). If the grid's is at least 3× the turned grid's, the lines are horizontal (axis 0). Otherwise the profiles do not decide. 2. **Which way up.** If the lines are horizontal by the 3× rule, the page is not tilted enough to act on, and its strips (found with the page finder, ruling lines left in place) have clearly more ink above their body band than below it, the page is upright and nothing is read. Per strip at least 10 pixels tall and 3× as wide as tall, the body band is the rows with at least half the ink of the densest row; over all such strips `above` and `below` are the ink above and below their bands. The test passes when `above + below ≥ total/20` and `10·above ≥ 14·below` (ascenders and capitals against descenders; digits, capitals only and upside-down text fail it and are read). Otherwise (on the axis chosen, turned and straightened by that axis's skew estimate, when that is acted on): the page's strips are found with the page finder (ruling lines left in place), the 24 biggest by area whose width is at least 3 times their height are taken, those with a height between 0.6× and 1.7× of the median height of those are kept, and up to 8 are read twice by the recognizer: as cropped and with the crop turned half way round (the crop's pixel array reversed). The weight of a line is the longer reading's length. The page scores are the weighted means of the two confidences, and *share* is the weighted share of lines that read better turned. The first 3 lines are read first and the rest skipped when all 3 read better upright and the upright mean is at least 0.85. The page is turned half way round when `turned > upright + 0.05`, `turned ≥ 0.75` and `share ≥ 0.6`. 3. **Undecided profiles** (neither contrast is 3× the other): axis 0 is read first; axis 1 is read too and wins only if its better reading is at least 0.75 and at least 0.15 above axis 0's. Photographs, receipts and forms end up here and are kept upright unless the recognizer clearly reads them better the other way. The result is `quarter · 90` plus 180 when turned half way round, modulo 360. `OcrPage.Orientation` reports how far the content was turned clockwise from upright: 0, 90, 180 or 270 (a page whose text tops point to the right has 90, and is turned 270° clockwise, that is 90° counter-clockwise, to read). ## Orientation from a page detector With a detector loaded (`OcrEngine.WithDetector`), step 1 is the detector's orientation head: the page is detected as given; when the head's winner is at least 0.9 confident the page is turned clockwise by the winning quarter count (the shared rotation step) and detected again on the turned page, otherwise the first detection is the one used. The classical decision is not run. How the turns compose: quarter turn (head) → skew estimate and deskew on the turned page → detection on the turned, straightened page. The page model is always in the input image's coordinates, with or without a detector: `Width`/`Height` are the input's, the quarter turn is reported in `Orientation` (the same 0/90/180/270 as above), and a diagnostic (`OCR103`) names the turn and the head's confidence and says positions stay in the image as given. The elements read on the turned page are carried back by the mapping below, so they get quads. Deskew, when it acts, is carried back the same way and reported in `SkewDegrees`. A page with neither a confident turn nor a tilt is detected once, exactly as without the orientation step. ## Angle search Photographs taken with the camera turned by 10 to 30 degrees are beyond the skew estimate (±15°) and the per-line tilt reading; the page reads as garbage. The search costs nothing on pages that read well. - **Trigger.** After the first reading (steps 1–5, mapped to the frame the caller gets), let *confident* words be those with calibrated confidence ≥ 0.9. Let the *share* be the characters in confident words over all characters of all words. The search runs when the share is under **0.9** (a page without words has share 0 and is searched only when at least 0.2% of its pixels are ink, so a blank page is not). It is the **full search** when the share is under **0.6** and the **light search** from 0.6 up to 0.9 (a photograph turned by a few degrees reads mostly right and still loses lines). Pages with share ≥ 0.9 are never read again. - **Score** of a reading: the sum, over its confident words in reading order, of `confidence · (number of UTF-16 code units of the text)`, accumulated in double precision. - **Candidates.** The page as it was before step 4 (turned upright, not straightened) is turned by a slope `k / 1024` exactly as in Deskew, with `k` taken from the table for 5°, 10°, … 35°: `90, 181, 274, 373, 477, 591, 717` (round(1024·tan a)). The search is coarse to fine: 1. 10°, then −10°, and in the full search 20°, −20°, 30°, −30° (positive slopes first), each read like the first reading (detection, lines, recognition, reading order, the same options) and mapped back through the frame of the candidate; 2. if one of these won, 5° either side of the winner (the smaller magnitude first). A candidate whose canvas would exceed `OcrLimits.MaxImagePixels` is skipped. - **Winner.** A candidate replaces the best so far only if its score is strictly greater, and the first candidate must beat the first reading's score times **1.1**; ties therefore keep the earlier (smaller-angle, positive-sign) reading. If none wins, the first reading is the result unchanged. - **Result.** The winning page's boxes and quads are in the input frame (the turn is part of the frame, so quads are produced by the mapping below), `SkewDegrees` is the candidate's angle (`atan(k/1024)`), and a diagnostic says the page was read at several angles. The full search costs at most 8 more readings, the light search at most 4 (10°, −10°, and the two 5° steps around a winner). Every reading is an ordinary page reading, so the search is bit-identical on every path and degree like the rest. ## Mapping back The page model is in the input image's coordinates: `Width`/`Height` are the input's. A point `(u, v)` of the straightened frame maps to the turned frame by `R(s)` about the canvas and page centers (the sampling formula above, `qx, qy`), then to the input by the inverse of the quarter turn: for 1 quarter `(x, y) = (qy, H − qx)`, for 2 `(W − qx, H − qy)`, for 3 `(W − qy, qx)` (`W`, `H` the input's size). Every box becomes the corner quad of its four mapped corners, in the order top-left, top-right, bottom-right, bottom-left of the text's own frame, plus the axis-aligned box around that quad. Words, characters, lines, blocks, regions and tables carry the quad (`quad` in the JSON); table cells and key–value pairs keep only the bounding box. A page that needed no turn has no quads. # Page detector `dbnet-lite-v1` `dbnet-lite-v1` is [`dbnet-lite-v0`](https://recursiveai.org/oneocr/docs/detector-v0/) with three optional changes. Everything else is as in v0: inputs, outputs, pre- and post-processing, layer types (convolution, 3x3 depthwise convolution, ReLU, nearest 2x upsampling, addition, channel mean, linear), tensor naming, the evaluation-order freedom and the numeric rules in [numerics.md](https://recursiveai.org/oneocr/docs/numerics/). There is no new layer type, kernel size or dilation. A v1 file with `layout16_blocks` absent or 0 and `orient16` absent or false is exactly the v0 graph. The v1 members are an error in a file whose architecture is `dbnet-lite-v0`. ## 1. Stride-16 context branch for the layout head With `layout16_blocks` = *n* > 0 and width `layout16` = *w16*: ``` p16 = lat2 + up2(lat3) (as in v0; p16 is not smoothed) g = DW(fpn -> w16)(p16) layout16.0 g = DW(w16 -> w16)(g) layout16.1 .. layout16.(n-1) m = conv1x1(g), w16 -> layout, bias, no ReLU layout_mix h = DW(fpn -> layout)(p8) + up2(m) layout.0 output plus the upsampled mix h = layout.1 ... layout.(layout_blocks-1) as in v0 layout = layout_out(h) as in v0 ``` `layout` is the config's layout width, `fpn` its FPN width. The 1x1 convolution commutes with nearest upsampling, so the multiply runs on the map that is four times smaller. All branch maps are at stride 16: height `H/16`, width `W/16`. Because the padded input is a multiple of 32, `H/8` and `W/8` are always exactly twice `H/16` and `W/16`, so the upsample never meets a partial pixel; the stride-32 map may be odd (`W = 160` gives 5 columns), which the tests cover. The addition is one fp32 addition per element and commutes, so its order does not matter. Tensors: `layout16..dw.weight/.bias` (`[c,1,3,3]`, `[c]`) and `layout16..pw.weight/.bias` for `j < n` (the first block has `fpn` inputs, the others `layout16`), and `layout_mix.weight` `[layout, layout16, 1, 1]`, `layout_mix.bias` `[layout]`. ## 2. Orientation from two scales With `orient16` true the four logits are one linear layer over the concatenation `[mean(stage2 output), mean(stage3 output)]`, stage 2's channels first; `orient.weight` is `[4, widths[2] + widths[3]]`. Each mean is the row-major fp32 sum over the map divided by the position count ([numerics.md](https://recursiveai.org/oneocr/docs/numerics/)); each logit is one dot product over the `widths[2] + widths[3]` joined values in the 16-lane order (the lanes continue across the join: the concatenation is one vector), plus the bias. ## 3. Widths and depths They come from `config` only. The 0.2.0 configuration is `widths [32, 72, 176, 288]`, `depths [2, 3, 6, 4]`, `fpn 48`, `text 32`, `layout 96`, `layout16 160`, `layout16_blocks 4`, `orient16 true`. ## Line axis (post-processing, any detector version) A text line's component of the text map also gives the line's own axis, used to read lines that lean (a photographed page keeps some perspective lean after the page-wide deskew). The text map and the component are those of [detector-v0](https://recursiveai.org/oneocr/docs/detector-v0/#post-processing); nothing about the boxes changes. For a component of `n >= 12` cells (cell `(x, y)` has its centre at `(x + 0.5, y + 0.5)` in cell units, one cell = 2 network pixels), with mean `(mx, my)` over the cell indices and `a = Var x`, `b = Cov(x, y)`, `c = Var y` (divisors `n`, sums in the order the flood fill visits the cells): - `h = (a - c) / 2`, `d = sqrt(h*h + b*b)`; no axis when `d <= 1e-9`. - `cos 2t = h / d`; `cos t = sqrt(max(0, (1 + cos 2t) / 2))`, `sin t = sqrt(max(0, (1 - cos 2t) / 2))`, negated when `b < 0`. The axis points right (`cos t >= 0`); `sin t > 0` means the line descends to the right. - Eigenvalues `major = (a + c)/2 + d`, `minor = max(0, (a + c)/2 - d)`; `elongation = major / (minor + 1/12)` (one cell's own variance). - The line frame has its origin at the centroid `(mx + 0.5, my + 0.5)`, `u` along the axis and `v` across it, pointing down: `u = dx*cos t + dy*sin t`, `v = dy*cos t - dx*sin t` for a cell centre offset `(dx, dy)`. Its extent is `[min u - 0.5, max u + 0.5] x [min v - 0.5, max v + 0.5]`, then grown on every side by `0.75 * text_expand * min(length, thickness)` like the axis-aligned box. - Everything is scaled to page pixels by `2 / scale` (the text map stride over the page scale). The page point of `(u, v)` is `(cx + u*cos t - v*sin t, cy + u*sin t + v*cos t)`. Only `+ - * /` and square roots are used, so the axis is bit-identical on every platform and degree of parallelism. A line is **read along its axis** when its axis passes all of these; otherwise it takes the axis-aligned path described in the engine's line-crop rules, unchanged: | test | value | |---|---| | cells | at least 20 | | elongation | at least 100 (the component is about ten times longer than thick) | | `|sin t|` | at least `sin 0.3 degrees`, and at least `0.15 * thickness / length` (the angle of a short, thick component is not believed), and at most `sin 15 degrees` | The values were chosen on held-out camera-simulated pages and on straight pages (so that straight text stays on the axis-aligned path). Reading a line along its axis: 1. **Ink band.** The detected rectangle is sampled into an upright strip of one-pixel cells (below), and the usual ink-band rule (the main band and any band holding at least 40% as much ink) finds the line's ink rectangle in the line frame; each band is one line. 2. **Crop.** The crop rectangle has the margins of the axis-aligned rule (a quarter of the line height above and below, half of it left and right, the height raised to 0.85 of the page's median) but is taken in the line frame and is not clamped to the page. It is sampled (below) and scaled to the model height exactly as an axis-aligned crop. 3. **Words.** A word's extent along the line comes from the alignment; its rectangle is the ink inside that extent and the line's rows, found on the line-frame pixel grid with the page pixel under each cell centre (nearest pixel). Its `quad` is that rectangle's corners (top-left, top-right, bottom-right, bottom-left of the text's own frame) mapped to the page, and its box is the box around the quad. Characters use the same rows and their alignment edges. A line's `quad` is the smallest rectangle in the first word's direction that holds all its words' corners. 4. **Reading order** sees the straightened ink rectangle, unrotated, placed at the centre of the tilted one. **Sampling.** The value at a page point `(px, py)` (pixel `(x, y)` covers `[x, x+1)`) is bilinear between the four pixel centres around it: `fx = px - 0.5`, `x0 = floor(fx)`, `wx = fx - x0`, likewise `y`; neighbours outside the page repeat the edge pixel; `top = p00*(1-wx) + p10*wx`, `bottom = p01*(1-wx) + p11*wx`, `value = top*(1-wy) + bottom*wy` in double precision, rounded half up to a byte. No fused multiply-add is used, so the bytes are the same on every platform. ## Ink on camera photos (page reader, any detector version) The ink inside a detected line (the ink band, the line's ends, the word rectangles) is found with the page's Otsu threshold: ink is the smaller class. On a photo whose light falls off across the page one threshold cannot fit every line, and a line the threshold does not fit loses its ends or its words. The page reader therefore binarizes lines of **camera photos** by itself. A page is a camera photo when the detector's noise estimate `sigma` (see the page-conditional threshold below) is above `0.1`; screenshots and clean renders have flat colours, and pages without a noise estimate (a model without the rule) are never treated as photos. On every other page nothing here applies and the output is unchanged. On a photo, for each detected line (axis-aligned rectangle, or the straightened strip of a line read along its axis): 1. **Line threshold.** Let `H` be the histogram of the line's gray values and `(t, bright)` Otsu's threshold over `H` with ink as the smaller class. The line uses `(t, dark ink)` instead of the page's when - the page reads ink as bright, at least half of the page's detected lines read ink as dark by their own Otsu threshold, and this line does too (the page threshold was fixed by the dark majority of a shadowed photo; a dark-mode screen has bright ink in most lines and keeps the page's reading); or - the page reads ink as dark, more than half of the line's pixels are at or below the page's threshold (the threshold lies inside the line's paper), and the line's own reading is dark ink with `t` different from the page's. The line's own reading is used only when it is credible: the pixels at or below `t` are at least 1.5% of the line, and the mean gray of the rest exceeds the mean of those by at least 40. Otherwise the page's threshold stands. The threshold then serves the ink band, the line's ends and the word rectangles of that line. 2. **Line ends.** After the ink band is found, an end of the line's ink that is more than 0.8 of the band's height short of the detected rectangle's end (left or right, along the line) is moved to the detector's end. Print at the end of a line is where blur and uneven light leave it faintest, and a crop that stops there loses the words. 3. **Word rectangles.** On every page: when the ink found inside a word's aligned extent is narrower than a quarter of the extent (and the extent is at least half the line's height), the word's rectangle is the aligned extent over the line's rows, as when no ink is found. A rectangle of a pixel or two otherwise tears the word out of its line in reading order. ## Line ends on flat pages (page reader, any detector version) The detector's box is a few pixels uncertain at the ends of a line, and a box that stops inside the last letter (or just before it) leaves the ink band, and with it the crop the recognizer reads, a letter short: `Nairobi` read as `Nairob`. Measured on exact-truth pages, the box ends about 1 px inside the ink on typical lines and a letter or more inside on a few percent of them. After the ink band of a line is found, each end is therefore followed outwards on **pages that are not camera photos** (photos keep the rule above). Let `B` be the band's height in pixels. 1. **Follow.** From the band's last ink column, scan columns outwards over the band's rows. A column has ink when any of its pixels is on the ink side of the threshold the band was found with. Scanning stops at the first run of more than `max(1, round(0.25 * B))` columns without ink (a letter gap is below this, a word gap above it), at `1.0 * B` columns from the old end, at the page edge, or at the box of the nearest other detected line on the same row (two lines overlapping vertically by at least half the smaller height; the line on the side of this one's centre that the end is on). The candidate end is the last column with ink. 2. **Letters only.** The candidate is taken only when the stretch gained is at least `ceil(0.25 * B)` columns and its ink pixels are at most 0.65 of its area (stretch columns times `B`). A shorter stretch is the letters' anti-aliased edge, and moving the crop for it only shifts what the recognizer sees by a pixel. A stretch that is mostly ink is a button, a rule or a border, not letters. Otherwise the end stays. 3. The two ends are decided independently. Nothing else about the strip changes; the crop rule (margins of half and a quarter of the line's height, scale) applies to the new extent. Without step 2, about one strip in ten on screenshot pages grew by a whole line height onto the fill of buttons and bars. The constants (0.25 gap, 1.0 reach, 0.25 minimum, 0.65 fill) come from a sweep of seven variants over generated layout and screenshot pages. ## Joined lines on flat pages (page reader, any detector version) Lines set solid (leading barely above the type size, as in statute and report text) touch row by row: a descender reaches the next line's ascenders, so no pixel row between two lines is empty. The detector can then box two or three lines as one, and the ink-band rule above (a gap of more than an eighth of the box height ends a band) cannot cut it. A recognizer line crop that holds two lines reads as noise. On **pages that are not camera photos**, with at least 8 detected lines: 1. **Typical box.** `T` is the median height of the page's detected boxes (faint-text recoveries left out). 2. **Tall band.** A band of ink rows (found as above, rows counted over the box's width) at least `ceil(1.5 * T)` rows tall is cut. 3. **Valley.** Let `count[r]` be the ink pixels in row `r` over the box's width, `above(r)` and `below(r)` the densest rows on either side within the band. A row is a valley when `count[r] <= 0.2 * min(above(r), below(r))` (so it lies between two dense bands, not in one line's ascender or descender rows) and at least `max(2, floor(R / 2))` rows remain on each side, `R` being the reference height (`T` at first). The valley with the smallest ratio is the cut (ties: nearest the band's middle); the cut row belongs to neither part and blank rows at the cut ends are trimmed. 4. **Again.** Each part is cut again by steps 2 and 3 while it is at least `ceil(1.5 * R')` tall, with `R'` the smaller of `R` and its sibling's height. A box of three lines of small type is cut twice; two lines of normal type once. 5. **No duplicates.** A box is often drawn over a line that has a box of its own (the lines of a shaded note beside running text). A part is dropped when another detected line that is shorter than this box and shares at least half of the narrower width covers at least half of the part's rows. When every part would be dropped, all are kept. 6. The parts go through the ink-band mass rule with a lower bar than ordinary bands: a part is kept with a tenth of the heaviest part's ink (ordinary bands need 40%), because a short last line has little ink beside a full one. Each part's horizontal extent is taken from its own rows. 7. A box this tall is cut upright first (the principal axis of a component that joins lines leans with them, and straightening along it smears the rows between the lines). If it was cut, the parts are used; otherwise the box goes on as any other, along its axis if it leans (see "Line axis"), where the same cut is tried in the straightened strip with `T` unchanged and without step 5. Without step 5 a line already read through its own box was read twice. The constants (1.5, 0.2, the cover share 0.5) were chosen on held-out real-PDF pages (IRS layout, scans) and on generated solid-set pages; on those, a tall-band factor anywhere from 1.2 to 1.7 gives identical output, and a valley share of 0.35 made generated layout pages worse by splitting single lines. ## Evaluation order and memory As for v0, every output value is an independent sum, so a port may evaluate whole or in bands. The C# reader keeps the v0 banded stages; the stride-16 branch runs on whole stride-16 maps (a quarter of the stride-8 map's size), `p16` is kept until the branch has consumed it, and stage 2's channel means are taken before stage 2 is dropped. On an A4 page at the 0.2.0 widths the scratch in use at once peaks at 24.3 MB at degree 1 (22.8 MB for v0 at its own widths). ## Reader checks Before a tensor is read or a buffer allocated: `layout16_blocks` is in `[0, MaxConvBlocks]` and counts toward the total block limit with the backbone and the layout head; `layout16` is positive and within `MaxLayerWidth` when the branch exists; the stride-16 buffers are checked against `MaxActivationElements` with the other activation sizes; every tensor shape is checked against `config`. `orient16` must be a boolean. ## Tolerances As in [numerics.md](https://recursiveai.org/oneocr/docs/numerics/), with these additional layers against the float64 evaluation: `p16` Atol 4e-6, `layout16` 5e-6, `layout_mix` 5e-6 (Rtol 1e-5). `layout_feat`, `layout` and `orient` keep their v0 values. Worst error measured on random-weight models at the 0.2.0 widths and a narrow model, on every path, as a fraction of the tolerance: `p16` 0.24, `layout16` 0.38, `layout_mix` 0.29, `layout_feat` 0.55, `layout` 0.39, `orient` 0.14. Against PyTorch fp32 fixtures at the 0.2.0 widths (and a second model with the branch only) every layer differs by about PyTorch's own fp32 error. ## Page-conditional text threshold (post-processing, any detector version) Camera pages carry sensor noise that the detector sometimes turns into faint speckle; on them a higher text threshold reads better, while on scans, screenshots and clean pages the manifest's `text_threshold` is right. A model may carry a rule that picks the text threshold per page. It needs all three members of `config.postprocess`; with none of them the rule is off, with some of them the file is rejected. | member | meaning | |---|---| | `rule_sigma_min` | the page's noise estimate must be above this | | `rule_junk_max` | the page's speck density must be below this | | `rule_text_threshold` | text probability above which a cell is on, on pages the rule selects (0.5 is logit 0) | **Noise estimate `sigma`.** Take the gray page at detector scale: the page resized so that its long side is 1024 pixels (the resize of the network input, before padding), unpadded area only. Apply the 3x3 kernel `[[1,-2,1],[-2,4,-2],[1,-2,1]]` to every interior pixel (`(h-2) x (w-2)` responses; a page under 3 pixels in either direction has `sigma = 0`). Build the exact integer histogram of the absolute responses (0 to 2040), take the lower median (the value at rank `(n+1) div 2`, counting from 1, n the number of responses) and divide by 4.047. This is Immerkaer's noise estimate (Immerkaer 1996, "Fast noise variance estimation") with the median of a half-normal distribution, 0.6745 x 6, in place of the mean. The median is an integer, so `sigma` is exact. **Speck density `junk`.** In the text map (probabilities, one per cell, padding included), count the 4-connected components of cells with probability above 0.3 that have 2 to 12 cells and a mean cell probability below 0.5. This count always uses 0.3, whatever `text_threshold` is. Divide by the map's cell count times 4, over 10^6 (the map has stride 2, so a cell stands for 4 input pixels). **Rule.** If `sigma > rule_sigma_min` and `junk < rule_junk_max`, label the text map with `rule_text_threshold`; otherwise with `text_threshold`. Nothing else changes: boxes, axes, layout regions and orientation are computed as before from the chosen labelling. `sigma` is computed only for models that carry the rule. The integer histogram makes it identical on every platform; the speck density depends on the text map, so it is as reproducible as the map. Constants of the first model with the rule, chosen on held-out pages (camera-simulated pages gain; clean, layout and screenshot pages have `sigma` under the minimum): `rule_sigma_min` 0.1235, `rule_junk_max` 0.6358, `rule_text_threshold` 0.5. The script `training/export/add_postprocess_rule.py` adds the three members to an existing container without touching the tensors. `dbnet-lite-v1` 0.2.2 is 0.2.1 with these three members added and the manifest version changed: the tensor bytes are identical to 0.2.1's, so the forward-pass parity fixtures of 0.2.1 apply unchanged. Only the page-level behaviour differs. `dbnet-lite-v1` 0.3.0 has the 0.2.x architecture and tensor layout with new weights (a fine-tune of 0.2.1 on windows of real web pages and light-on-dark synthetic pages, blended 1:3 with its starting weights). Its page rule is `rule_sigma_min` 0.1235, `rule_junk_max` 1,000,000 (the junk-density condition never holds back the rule) and `rule_text_threshold` 0.5. It has its own forward-pass fixtures. ## Faint text on dark backgrounds (post-processing, any detector version) Light text a few percent brighter than a dark background (small menu and footer text on dark bands) reaches the text map with probabilities of only a few percent, below `text_threshold`, so the ordinary labelling misses it. A model may carry a second, much lower threshold that is used only where the text sits on a dark background. All five members of `config.postprocess` are given together, or none (some of them: the file is rejected). Without them nothing below applies and the output is as before. | member | meaning | |---|---| | `faint_text_threshold` | text probability above which a cell is on in the second labelling (0.02 in the first model) | | `faint_bg_max` | largest median gray value (0 to 255) of a candidate's cell box that counts as a dark background (110) | | `faint_min_cells` | fewest cells of a candidate (6) | | `faint_min_aspect` | smallest width over height of a candidate's cell box (1.5) | | `faint_min_confidence` | smallest calibrated word confidence kept from a recovered line (0.9) | **Candidates.** Skipped on pages where the page-conditional rule above chose its threshold (camera pages). Otherwise: the cells of every ordinary component and their 8-neighbours are removed from the map (so a line's own fringe is never a candidate), the rest is labelled with `faint_text_threshold` (4-connected, as the ordinary labelling) and a component is a candidate when it has at least `faint_min_cells` cells (and at least the ordinary minimum), its cell box is at least `faint_min_aspect` times as wide as tall, it does not overlap an ordinary component's cell box by more than a third of the smaller box, and the median gray value of the pixels under its cell box (detector-scale page, one cell is two pixels) is at most `faint_bg_max`. **Extension.** The map holds only the most visible stretch of a faint line, so the candidate is followed along its rows on the gray page: a column of the candidate's rows has ink when one of its pixels is at least 5 above the median and the column's mean is not above `faint_bg_max`; the box grows left and right over ink columns, across gaps of up to its own height (at least 6 pixels), and stops before an ordinary line that shares at least half its rows. The box is then grown like an ordinary line (`k * text_expand` of its shorter side), and dropped when it covers more than a quarter of the smaller of itself and an ordinary line's grown box. **Order.** Recovered lines come after the ordinary lines; they take no part in snapping layout regions to lines. **Reading.** A recovered line is cropped and recognized like any other, but (a) it takes no part in the page's typical line height, so the other lines are cropped exactly as without it, (b) words under `faint_min_confidence` and single-character words are dropped from it, (c) a line that has no word left, or whose ink box covers more than a quarter of the smaller of itself and an ordinary line's ink box, is dropped. The page therefore changes only where a recovered line survives. The script `training/export/add_faint_rule.py` writes the five members into a container's manifest and leaves the tensors untouched. Constants were chosen on held-out tune pages only (screenshots of real pages, synthetic screens, scans, layouts, phone pages and receipts): on every tune set except the real-screenshot one the output is byte-identical to the container without the members. # CTC decoding Input: per-step class scores (`logits`, `T` steps of `C` classes, class 0 the CTC blank) from a recognizer, and the charset. Output: text and a confidence. Every mode gives the same bits on every CPU path, OS and degree of parallelism. ## Greedy (default) Arg-max per step (lowest index on ties), merge repeats, drop blanks (Graves et al. 2006, sec. 3.1). Confidence: mean over emitted characters of the largest softmax probability seen while that character was the winning class (0 for empty text). ## Beam search Prefix beam search over CTC alignments (Graves et al. 2006; Hannun et al. 2014, "First-pass large vocabulary continuous speech recognition using bi-directional recurrent DNNs"). ### Arithmetic All beam arithmetic is IEEE double precision with `+ - * /` and comparisons only, evaluated in the order written here, no fused multiply-add. The exponential and logarithm are our own: * `exp(x)`: `x <= -700` (or NaN) gives 0, `x > 700` is clamped to 700; `k = floor(x * 1.4426950408889634 + 0.5)`; `r = (x - k * 6.93147180369123816490e-01) - k * 1.90821492927058770002e-10`; the degree-13 Taylor polynomial of exp in Horner form with coefficients `1/n!` (`n = 13 … 0`), each step `p = p * r + c`; the result is `p * 2^k` built from the exponent bits. * `ln(x)`: `x = m * 2^e` with `m` in `[1, sqrt 2)` (halve `m` and add 1 to `e` when it is above `sqrt 2`); `s = (m - 1) / (m + 1)`, `z = s * s`, `q` the Horner form of `1 + z/3 + z²/5 + … + z⁹/19` (coefficients `1/(2j+1)`), result `e * 0.6931471805599453 + 2 * s * q`. * `logadd(a, b) = a + ln(1 + exp(b - a))` for `a >= b`; `a` when `b` is minus infinity or `b - a < -40`. Log probabilities of classes: `lp(j) = (z_j - max) - ln(S)`, with `z` the float logits, `max` their largest, `S` the float sum of `exp(z_j - max)` accumulated as in `numerics.md` (the same 16-lane order as the softmax elsewhere). The `float` subtraction is exact; the rest is double. ### Steps State: a set of prefixes (at most `BeamWidth`), each with `pb` and `pnb`, the log probabilities of the steps so far with the prefix emitted and the last step blank / non-blank. Start: the empty prefix with `pb = 0`, `pnb = -inf`. For each step `t`: 1. Candidates: the classes `j >= 1` whose logit is within 9.2 of `max`, at most 8, best first (equal logits keep the lower class). 2. For each prefix `p` with `total = logadd(pb, pnb)` and last class `c` (none for the empty prefix), in beam order: * `p` gets `pb += total + lp(0)`; * if `c` exists and `pnb` is not `-inf`: `p` gets `pnb += pnb + lp(c)` (the run of `c` continues); * for each candidate `j`, the extended prefix `p·j` gets `pnb += (pb if j == c else total) + lp(j)`. Contributions to the same prefix are combined with `logadd`; new prefixes are listed in creation order. 3. Rank the resulting prefixes by `logadd(pb, pnb) + R(prefix)` (below), best first, ties to the prefix created earlier, and keep the first `BeamWidth`. After the last step each kept prefix gets `final = logadd(pb, pnb) + R_end(prefix)`; the best (ties: created earlier) is the text. If a language model is set (whether or not a script fits the line), the best prefix whose code words (below) match the greedy reading's is taken, and the greedy reading if none does. Confidence: `exp(logadd(pb, pnb) / n)` of the chosen text with `n` emitted characters, clamped to `[0, 1]` (0 for empty text). It is the per-character geometric mean of the probability of the text over all alignments, and is not calibrated. ### Prefix score `R` A *word* is a run of non-space characters in the prefix (the space is the charset entry `" "`). `R` is the sum of a part for the finished words, `InsertionBonus` per character, and a part for the open word. * **Language model** (when set): for a script chosen once per line (the model whose alphabet contains the most letters of the greedy text's characters; the first on ties; none if no letter fits), each character `s` of a word that is not *protected* adds `LanguageModelWeight * max(LogProb(context, fold(s)) - mean, -1)`, where `fold` is the simple lowercase mapping (see `char-lm-v1.md`), `mean` is the file's mean and the context is the word's earlier symbols led by the word start. A finished word additionally adds the same for the word end. The open word counts without the word end in `R` and with it in `R_end`. * **Protected words**: a word is protected, and gets no language-model score, if it contains a decimal digit, one of `@ # _ / \ = % + * ~ ^ | < > [ ] { }`, or a character whose folded form is outside the script's alphabet. * **Lexicon** (when set): the open or finished word adds `LexiconBoost` when it spells a lexicon word exactly (same code points) and `LexiconBoost / 2` when it is a proper prefix of one and at least two characters long; nothing otherwise. Words with characters not in the charset are dropped from the lexicon. ### Code words The code words of a text are its space-delimited words that contain a decimal digit or one of the symbols above, in order. Beam search with a language model returns a reading only if its code words are those of the greedy reading, position by position, except that a word listed in the lexicon may stand in for the one it replaces. Numbers, identifiers, amounts and dates are therefore read exactly as greedy decoding reads them; the language model only changes the other words. ## Parameters | name | default | meaning | |---|---|---| | mode | greedy | greedy or beam | | BeamWidth | 8 | 1 to `OcrLimits.MaxBeamWidth` (64) | | LanguageModelWeight | 0.8 | at least 0 | | LexiconBoost | 2 | at least 0 | | InsertionBonus | 0 | per emitted character | The candidate count (8), the 9.2 window and the −1 floor were chosen on held-out synthetic lines (see the decoding evaluation in `bench/README.md`) and are part of this version. ## Limits Before the search starts, `steps * BeamWidth * 9 + 1` (the most prefixes it can create) is compared with `OcrLimits.MaxBeamNodes` (default 2^22), and `BeamWidth` with `OcrLimits.MaxBeamWidth`. ## Words and approximate boxes (greedy only) Words, their raw and calibrated confidences and their script are defined in [model-container-v1.md](https://recursiveai.org/oneocr/docs/model-container/#calibration-version-1). Boxes come from the greedy alignment: a character occupies the steps `[first, last]` of its run, and a step is `width / T` pixels of the line image (`width` the scaled line's width, `T` the number of steps). A word's left edge is the first step of its first run, or halfway across the gap to the previous run when there is one: `(prev.last + 1 + first) / 2`, in steps; its right edge is `last + 1` of its last run, or halfway to the next run, `(last + 1 + next.first) / 2`. Neighbouring words therefore tile the line, and the line's first and last word stop where the network first and last fired. Edges are computed in double precision, scaled by `width / T`, mapped back to page pixels by the crop's own ratio and rounded to float32. Vertical edges are those of the whole line's ink. These boxes are approximate and flagged so; beam search keeps no alignment, so words are always read greedily. # Character language model file, version 1 A file holds one n-gram model of words per script. The decoder (`decoding.md`) uses it to rank alternative readings of a text line. It is produced by `training/charlm/build_charlm.py` and read by `CharLanguageModel`. All integers are little-endian. ## What the model scores A word is a run of non-space characters, case-folded: each character is looked up by its simple lowercase form (Unicode one-to-one lowercase mapping; a character without one, and U+0130, stay as they are), so `NASI`, `Nasi` and `nasi` are one word and the alphabet holds lowercase letters. The model scores one word at a time, one symbol after another, starting from a word-start symbol and ending with a word-end symbol. There is no context across spaces. Symbols are small integers: | id | meaning | |---|---| | 0 | word start (only ever the first symbol of a context) | | 1 | word end (only ever the last symbol of an n-gram) | | 2 … 251 | the script's alphabet, in the file's order (`id = index + 2`) | The alphabet holds at most 250 characters. A character whose folded form is not in a script's alphabet makes its word unscorable for that script (the decoder gives such words no language-model score). ## Layout ``` magic 4 bytes "OCLM" version u16 1 scripts u16 number of script models, 1 or more script model, repeated `scripts` times: name_len u8 1 or more name ASCII printable, no spaces ("latin", "greek", "cyrillic") order u8 highest n-gram order N, 1 to 8 alphabet u8 number of characters A, 1 to 250 characters, A times: len u8 (1 to 8), then len bytes of UTF-8 (one charset class) mean f32 average log probability per symbol on the training words, in nats, between -50 and 0 tables, for n = 1 .. N: count u32 rows, `count` of them: key n bytes (symbol ids), prob u8, and for n < N also bow i8 ``` Rows of a table are sorted by key, comparing bytes from the first to the last (strictly increasing, no duplicates). Every byte of a key is below A + 2; byte 0 may only be the first byte of a key, byte 1 only the last. * `prob` q means ln P = -0.1 q (q = 0 … 254). * `bow` q (signed) means a back-off weight ln b = 0.1 q. ## Probabilities The builder estimates interpolated Kneser–Ney probabilities (Kneser and Ney 1995; Chen and Goodman 1998): absolute discounting with D = n1 / (n1 + 2 n2) per order from the counts of counts, lower orders on continuation counts (the number of distinct symbols that precede the n-gram; n-grams that begin a word use their plain counts). It then prunes n-grams by a minimum count per order, keeps every context of a kept n-gram, and writes the model in back-off form, as ARPA files do: each kept n-gram stores its full interpolated probability, and each context stores the weight that makes the pruned model sum to 1 over the words that follow it, `(1 - Σ P(w|h)) / (1 - Σ P_lower(w|h'))` over the kept extensions `w` of the context `h`. The first table holds every symbol of the alphabet and the word end, and the word start as a context (its row only carries a back-off weight). ## Scoring `LogProb(context, s)`: let `ctx` be the last `min(len, N-1)` symbols of the context (the word start counts as the first symbol of a word). Starting with `back = 0` and `L = len(ctx)`, down to 0: 1. If the table of order `L + 1` has the row `ctx[len-L..] ++ s`, return `back - 0.1 * prob`. 2. Otherwise, if `L > 0` and the table of order `L` has the row `ctx[len-L..]`, add `0.1 * bow` of that row to `back`. 3. Continue with `L - 1`. If no row is found (a file that lacks a unigram), the result is `back - 14`. The sum is accumulated in `float` in that order. ## Limits Reading checks, before allocating, the file size (`OcrLimits.MaxLanguageModelFileBytes`), scripts, order, the n-gram total over the file, and that the bytes of a table are present. Violations are `OcrLimitException` (a bound) or `OcrModelException` (structure). ## Sizes The model shipped with the recognizer (order 4, minimum counts 1, 3, 4, 6; the 187 training books, case-folded) is 1.42 MB: latin 1,015,999 bytes (130 characters), greek 301,157 bytes (57), cyrillic 104,747 bytes (57). ## Use for word breaks Besides ranking beam-search prefixes, the model judges whether two neighbouring words are really one: `docs/spec/oneocr-page-v1.md`, "Word segmentation", compares the word score of the joined letters with the sum of the two pieces' scores. The shipped file is a SHA-256-pinned asset of the models package (role `language-model`). # Word frequency file, version 1 A file holds how often words occurred in running text. The recognizer's word segmentation (`oneocr-page-v1.md`, "Word segmentation") uses it to take back a space the network put inside a word. It is produced by `training/wordlm/build_wordlm.py` and read by `WordFrequencyModel`. All integers are little-endian. Another implementation must reproduce the lookup and the comparison below exactly: they use integers only. ## What the table holds Words of Latin letters only, each lowercased by the simple one-to-one lowercase mapping of its code points (a character without one, and U+0130, stay as they are), as in `char-lm-v1.md`. A word with a digit, an apostrophe, a hyphen, a code symbol or a letter of another script is not in the table. A word is 1 to 40 UTF-8 bytes. The builder reads the Latin-script books marked `train` in `training/data/texts.yaml` (the books the character language model is built from; Alice and Pride and Prejudice, the evaluation page sets' texts, and the held-out books are not among them) and keeps words seen at least `min_count` times (default 3). It can add counts from the visible text of public-domain US federal web pages (`training/wordlm/README.md`); the file format does not change. A count is stored as `q = clamp(round(4 * log2(count)), 1, 255)`, a quarter of a bit: the count is about `2^(q/4)`. `q` is never 0. ## Layout ``` magic 4 bytes "OCWF" version u16 1 flags u16 0 (a reader rejects any other value) total u32 the number of word tokens the counts come from words u32 the number of words W in the table q_total u8 clamp(round(4 * log2(total)), 1, 255) q_missing u8 the q taken for a word that is not in the table blocks u32 ceil(W / 16) offsets u32 x blocks byte offset of each block in the word data word data: entries, W of them, in ascending byte order of the word (strictly increasing): shared u8 bytes the word shares with the previous entry's word (0 for the first entry of a block) suffix u8 bytes that follow; shared + suffix is 1 to 40 bytes suffix bytes of UTF-8 q u8 1 to 255 ``` Blocks hold 16 entries (the last holds the rest); block `i` starts at `offsets[i]`, which is where block `i - 1` ended (the first at 0). A reader checks all of this while it parses: lengths stay inside the word and the file, words strictly rise, no bytes remain after the last entry. The builder sets `q_missing = max(1, q(min_count) - 4)`, a word seen about half as often as the rarest word in the table. ## Lookup Binary search over the blocks on the first word of each (the last block whose first word is not above the target), then a scan of that block, rebuilding each word from the previous one's prefix. Words compare as unsigned bytes, a prefix before its extensions. ## Use The page reader's word segmentation (`oneocr-page-v1.md`, "Word segmentation", rule 3) takes two neighbouring pieces `a` and `b` (lowercased, letters only) with joined word `ab`, and compares, in quarter-bits, gain = q(ab) + q_total - q(a) - q(b) (q of a word that is not in the table is q_missing) which is four times the base-2 log-odds that the letters are one word and not two independent words. All comparisons are on integers, so every platform decides alike. The bundled table holds 219,000 words from the Latin-script books of 15 languages and modern web text, and is 1.3 MB. It knows nothing of numbers, codes, names or words with an apostrophe or hyphen, and such words are never joined with it. # Numeric rules The operation order that makes the library's output bit-identical across CPU paths (scalar, 128-bit vectors, AVX2 and AVX-512), operating systems and degrees of parallelism. A port that follows these rules reproduces the C# results exactly. All arithmetic is IEEE 754 binary32 unless stated; `fma(a, b, c)` is the fused multiply-add with a single rounding. Weights are stored as binary16 and widened exactly to binary32 on load. ## Dot products A dot product of two length-*n* rows is computed with 16 partial sums, initially +0. Element *k* is fused into partial *k mod 16*: `p[k mod 16] = fma(a[k], b[k], p[k mod 16])`, for all *k* in order, including a final group shorter than 16. The partials are then added in a fixed tree: 1. `p[i] = p[i] + p[i+8]` for i = 0..7 2. `p[i] = p[i] + p[i+4]` for i = 0..3 3. `p[i] = p[i] + p[i+2]` for i = 0..1 4. result `p[0] + p[1]` A bias, when present, is added to the finished sum. Vector paths keep the 16 partials in registers (one 512-bit, two 256-bit or four 128-bit); padding a row with zeros to a multiple of 16 does not change the result. Computing several outputs at once (a tile of A rows against B rows), reading one row in several contiguous segments whose lengths are multiples of 16, reducing four outputs' partials together with the same pairwise additions, or, for a dot product of at most 16 taps, keeping each tap's product in its own vector across output channels (lane k holds `fma(w[k], x[k], 0)`, the missing lanes are +0) are all the same computation: every output sees the same fused multiply-adds in the same order and the same tree. Parallel work is split across independent outputs; no reduction crosses threads. ## Layers - **Convolution** output value = dot product of the weight row with the zero-padded input patch, plus the bias, with both flattened in the order (kernel row, kernel column, input channel): tap index `k = (ky * kw + kx) * c + channel`, so the lane of a tap is `k mod 16`. (The container stores weights `[out][in][kh][kw]`; readers reorder them at load. The order is chosen so that, with channels-last activations, the taps of one kernel row are adjacent in memory.) Then ReLU `v > 0 ? v : 0`, then max pooling (a value that is not greater than the running maximum, starting at 0, is ignored). - **LSTM** step, per direction, hidden size H, gate order input, forget, cell, output: `proj[t] = dot(W_ih row, x_t) + (b_ih + b_hh)` (the two biases are added in fp32 once); `z = dot(W_hh row, h_prev) + proj[t]`; `i, f, o = sigmoid(z slice)`, `g = tanh(z slice)`; `c' = fma(f, c, i * g)` with `i * g` rounded first; `h = o * tanh(c')`. State starts at zero. The reverse direction visits steps from last to first and writes its output at the step's own position. - **Classifier** logits = dot product + bias, per step. - **Softmax denominator** (for confidence) = sum over classes of `exp(z - max)` accumulated into 16 partials by class index mod 16 with plain additions, combined with the tree above; the winning probability is `1 / sum`. ## Evaluation grouping that leaves every sum alone These rearrangements change what is computed together or which terms are visited, never the value of any output. A port may use them or not. - **Taps on the zero border.** A tap whose input value is an exact zero (a kernel row that falls on the zero border above or below the image rows) adds nothing: `fma(0, w, s) = s` for finite `w`, and a partial sum that starts at +0 is never -0, so the +0 or -0 product leaves it unchanged. Such taps may be left out of the dot product while every other tap keeps its lane `k mod 16` and its place in the order. This holds only when every weight of the layer is finite (a zero times an infinity or NaN is NaN); a layer with a non-finite weight visits all taps. Zeros that come from the data are not skipped. - **Batched recurrence.** The recurrent product `dot(W_hh row, h_prev)` of several lines at the same step may be one matrix product (rows of `h_prev` against the rows of `W_hh`); each output is the same dot product, then `+ proj[t]` as one fp32 addition. A line's step never reads another line's values, so which lines share a product, and in what order, does not matter. The input projections of a run of steps may likewise be computed ahead as one product. - **Three-by-three single-channel convolution** with 9 taps keeps the 16 partials in registers; lanes 9 to 15 are +0, and a partial `fma(w, x, +0)` is never -0, so adding one of those lanes in the tree returns the other operand unchanged: only lane 0 + lane 8 is a real addition, and the remaining additions are the tree's, in its order. ## Transcendental functions No platform math library is used. With `Limit = 80`: - `exp(x)`: clamp `x` to [-Limit, Limit] (`x < Limit ? x : Limit`, then `x > -Limit ? x : -Limit`, so NaN becomes Limit); `n = roundToNearestEven(x * log2e)`; `r = fma(-n, ln2_hi, x)`, then `r = fma(-n, ln2_lo, r)`; `p = c7`, then `p = fma(p, r, c6)`, `c5`, `c4`, `c3`, `c2` in turn; `y = fma(p, r * r, r) + 1`; result `y * 2^n`, with `2^n` built from the exponent bits `(n + 127) << 23`. - `sigmoid(x) = 1 / (1 + exp(-x))`; `tanh(x) = 1 - 2 / (exp(x + x) + 1)`. - Constants (binary32): `log2e = 1.44269504088896341`, `ln2_hi = 0.693145751953125`, `ln2_lo = 1.42860677e-6`, `c2 = 0.5`, `c3 = 1/6`, `c4 = 1/24`, `c5 = 1/120`, `c6 = 1/720`, `c7 = 1/5040`, each rounded to binary32. Measured accuracy: exp and sigmoid within 4e-7 relative error, tanh within 2e-7 absolute. ## Detector layers The page detector (`dbnet-lite-v0`, [detector-v0.md](https://recursiveai.org/oneocr/docs/detector-v0/)) uses activations laid out channels-last, `[row][column][channel]`, with the channel count padded up to a multiple of 16 by zeros that stay zero. Nothing about this layout is visible in the results; it is how the sums below are kept in the order of the dot-product rule. - **1x1 convolution** is a dot product over the input channels for every position: the rule above, with the channel count zero-padded to a multiple of 16 (no change to the result), then the bias, then ReLU where the graph has one. - **3x3 depthwise convolution** (zero padding 1, stride 1 or 2, one filter per channel): the nine products of a position and channel are the elements of one dot product, tap *k* = `ky * 3 + kx` in lane *k*: `p[k] = fma(w[k], x[k], +0)`, lanes 9 to 15 stay +0, then the tree above and the bias. A tap that falls outside the image multiplies a zero padding value and contributes `fma(w, 0, +0) = +0`, so it is simply +0. Vector paths hold the nine products for 4, 8 or 16 channels at a time; a channel is never split across lanes, so the width cannot change a result. - **Stem** (3x3, stride 2, one input channel, ReLU) follows the same rule per output channel. - **Nearest 2x upsampling plus addition**: `dst[y][x][k] = dst[y][x][k] + src[y / 2][x / 2][k]` (one fp32 addition; commutative). - **Global average** over a stage output of *n* positions, per channel: the positions are added one after the other in row-major order starting from +0 in fp32, then divided by *n* (IEEE division). The orientation logits are dot products of that vector with each row of `orient.weight`, plus the bias; with `dbnet-lite-v1` `orient16` the vector is the stage-2 means followed by the stage-3 means, one dot product over the joined vector ([detector-v1.md](https://recursiveai.org/oneocr/docs/detector/)). - **ReLU** `v > 0 ? v : 0` (NaN and -0 give +0). Sigmoid and softmax appear only in post-processing, with the exp above; thresholds are compared on logits. Rows of an output are split across threads; no reduction crosses a thread boundary. ## Tolerance of the detector against float64 and PyTorch For each layer the C# fp32 result is compared with a float64 evaluation of the same graph and fp16 weights: `|ours - exact| <= Atol + Rtol * |exact|`, with `Rtol` 1e-5 for every layer and `Atol` as in the table (about three times the worst error seen over the test models on every path). Against PyTorch's fp32 fixtures the allowance is twice that, plus twice PyTorch's own recorded fp32-versus-float64 error for the layer (`fp32_vs_fp64_floor` in `fixtures.json`). | layer | Atol | | layer | Atol | |---|---|---|---|---| | input | 1e-6 (Rtol 1e-6) | | layout_feat | 5e-6 | | stem | 1e-6 | | layout | 3e-6 | | stage0 | 1e-6 | | text4 | 5e-6 | | stage1, stage2, stage3 | 2e-6 | | text2 | 8e-6 | | p8 | 4e-6 | | prob | 5e-6 | | p4 | 5e-6 | | orient | 2e-7 | | p16 (v1) | 4e-6 | | layout16, layout_mix (v1) | 5e-6 | ## Tolerance against PyTorch Agreement with a PyTorch fp32 computation from the same fp16 weights is a per-layer tolerance, not bit equality. Both sides carry float32 rounding error against the exact answer, so the difference between them is bounded by the sum of the two errors. For layer *L* and every element: `|ours - pytorch| <= Atol(L) + 2 * Floor(L) + Rtol(L) * |pytorch|` - `Floor(L)` is PyTorch's own fp32-versus-float64 maximum absolute error for layer *L* over the fixture lines, measured by the fixture generator and stored in `fixtures.json` as `fp32_vs_fp64_floor`. It scales with the activation magnitudes of the model, so a trained model with large, saturating activations gets a proportionally larger allowance than a random-weight one. Our own float32 error is of the same size (both use float32 accumulation over the same number of terms), hence the factor 2. - `Atol` / `Rtol` are twice the tolerances our kernels must meet against a float64 reference of the same network on random-weight models (error measured on models of the intended size, 3-5x headroom): conv 4e-6 / 2e-5, LSTM 3e-6 / 2e-5, classifier 2e-6 / 2e-5; so 8e-6 / 4e-5, 6e-6 / 4e-5, 4e-6 / 4e-5 here. The input layer must match to 1e-6. - On top of the layer tolerances, our greedy decoding of our own logits must equal the fixture's `decoded` text for every line. Observed on the 50 held-out lines of a trained model: the largest difference per layer is 0.9-2.0x the PyTorch floor (conv.5 2.6e-6, lstm.l1 6.6e-5, fc 7.0e-5, within the bound through its relative term). ## Document extraction kernels The `docextract-v0` forward pass (graph in [docextract-v0.md](https://recursiveai.org/oneocr/docs/docextract/)) uses the rules above and these. Every sum of products is a dot product in the 16-lane order, so a vector path gives the same bits as scalar; everything else is element-wise IEEE arithmetic (add, subtract, multiply, divide, square root, fused multiply-add where stated), which is the same on every width. No sum is split across threads: rows, heads and words are split between threads, each output is computed by one call. - **Linear layer** `y = x Wᵀ + b`, `W` stored `[out, in]`: one dot product of `in` terms per output, bias added to the finished sum. Inputs whose length is not a multiple of 16 continue the same lanes (the tail is part of the order); `feat_proj` (8 inputs) and `pair1` (13) are zero-padded to 16, which changes nothing because a zero product leaves every partial sum as it was. - **Embedding rows**: `x[i] = x[i] + table[id]`, one addition per element, applied in the order x0, x1, y0, y1 of the position tables after the character projection; then the projected features are added; then layer norm. - **Character CNN**, per word: the embeddings of its characters (an id of 0 reads row 0 of the table) with one zero row before and after; output position *p* is the dot product, in the order of taps (k, channel) with k = 0 the previous character, of the weight row `[k][channel]` (the container's `[out, channel, k]` reordered at load) with the three rows from *p* - 1 to *p* + 1, plus the bias. ReLU (`v > 0 ? v : 0`), then the maximum over the word's valid positions (characters with an id other than 0), starting from +0. Positions after the last valid character are not computed. `char_pool` is that maximum. - **Layer norm** over `d` (epsilon 1e-5 as fp32): `mean = (Σ x) / d` with the sum a dot product with ones; `dx = x - mean`; `variance = (Σ dx²) / d`, a dot product of `dx` with itself; `inv = 1 / sqrt(variance + eps)` (IEEE square root and division); `y = fma(dx * inv, weight, bias)`. - **GELU** (tanh form): `inner = c0 * fma(c1, (x * x) * x, x)` with `c0 = 0.79788456f`, `c1 = 0.044715f`; `y = (0.5f * x) * (1 + tanh(inner))`, `tanh` being the library's own (see Transcendental functions). - **Pair features**: with `geo` the box in units of the median word height and the centre `c = (lo + hi) * 0.5f`, `d = a_j - a_i`, `squash(d, s) = d / (s + |d|)` (one division): centre x, centre y, x0, x1, y0, y1 each at s = 1 and s = 16, then the height difference at s = 1; 13 values, in that order. The pair MLP is `pair1` (13 to `pair_hidden`), ReLU, `pair2`; row *i*, column *j* of channel *c* is stored at `c * n * n + i * n + j`. - **Attention**, per head: `s = q·k` (dot product over the 32 head dimensions), `s = s / sqrt(head size) + bias` (the divisor is the fp32 square root of the head size, 5.656854f for 32; division then addition, two roundings); softmax per row: `m = max s`, `e = exp(s - m)` with the library's exp, `sum = Σ e` as a dot product with ones, `p = e / sum`; `a = Σ_j p_j v_j` as a dot product over `j` with the head's V transposed to contiguous rows, no bias. - **Pointers**: `rel[r, i, j] = (q_i,r · k_j,r) / sqrt(rel_dim) + pair_bias[layers * heads + r, i, j]`, the division by the fp32 square root (8 for 64). - **Document logits**: the mean over words of `final` (each column summed in word order in double precision, divided by the word count, rounded to fp32) through `doc_head`. - **Windows**: see docextract-v0.md; the merge is selection, not arithmetic, except the document logits of a page of several windows, which are a double-precision weighted mean rounded to fp32. ### Tolerance of the document extraction model Per layer (`char_pool`, `emb`, `pair_bias`, `block.`, `final`, `tags`, `roles`, `doc`, `rel`), against PyTorch's float64 run of the same fp16 weights: `max |ours - exact| <= 2 * Floor(L) + 8 ulp * max |exact|`, where `Floor(L)` is PyTorch's own fp32-versus-float64 maximum error for the layer (`fp32_vs_fp64_floor` in the fixtures) and the second term (8 times 2^-23 of the layer's largest value) covers layers where PyTorch happens to be nearly exact. Against PyTorch's fp32 result: `max |ours - pytorch| <= 3 * Floor(L) + 8 ulp * max |exact|`. Observed on two random-weight models over pages of 24 to 300 words on all four paths: our error is 0.4 to 1.13 times PyTorch's for every layer except the document logits (2.05 times a very small floor, inside the ulp term); the difference from PyTorch's own result is at most 1.44 times the floor. Results are bit-identical across paths and degrees. ## Line sampling Reading a leaning line (see [detector-v1](https://recursiveai.org/oneocr/docs/detector/#line-axis-post-processing-any-detector-version)) samples the page at points off the pixel grid. The sampler is bilinear in double precision with a fixed order of operations (`top`, `bottom`, then their blend), edge pixels repeated and the result rounded half up to a byte. It uses no fused multiply-add and no library trigonometry: the axis comes from `sqrt`, `+ - * /` only, so every platform and every degree of parallelism produces the same bytes. The sampled crop then goes through the same resize as any line crop. ## Libraries outside the kernels Every crop is scaled with `RecursiveAI.OneImage` (linear light, 8-bit gray), which is outside these rules. Its sRGB tables are built with the platform's `powf`; macOS's differs from glibc's by one binary32 ulp in 2 of the 256 decode entries, and that is enough to change one pixel of a crop, a logit, and the last bit of a raw word confidence (text and boxes were unchanged on the sampled pages). Taking the power in double and rounding once, `(float)Math.Pow(x, 2.4)`, gives tables equal to glibc's on x64 and arm64 and to macOS's, and leaves every x64 output unchanged. Until OneImage ships that, raw confidence can differ in the last bit between macOS and Linux; `OneOcr.Bench mathhash` prints the table hashes. The platform identity check is described in `bench/README.md`. Two library `Math.Log` calls remain outside the kernels (the detector's logit thresholds and a column-width comparison in the page reader); they are double precision, rounded to binary32 or compared, and agreed on every platform tried. # Document extraction model, `docextract-v0` A small transformer over the words of a page. It labels words; deterministic code joins the labelled words into fields, so a value is always the text of words that are on the page. The container section is in [model-container-v1.md](https://recursiveai.org/oneocr/docs/model-container/), the numeric rules of the kernels in [numerics.md](https://recursiveai.org/oneocr/docs/numerics/). The model's graph and training are described in `training/export/docextract-architecture.md`; this document repeats what a reader of the weights file needs and specifies what the library does around it: featurization, windows, assembly and normalization. ## Input The words of an `OcrPage` in reading order, with empty texts dropped; box = the word's box in page units (origin top left, y down); page size W x H (each at least 1). `unit` = the median word height (the mean of the two middle values for an even count), at least 1; word height and width are at least 1e-3. All of this is in double precision until the values are stored as fp32 (`feats`, `geo`). Non-finite coordinates count as 0 and every coordinate is limited to ±1e9. | input | definition | |---|---| | `chars` `[N, max_chars]` | the word in NFC; each code point (not UTF-16 unit) is its index in the container `charset`, or `unk_char` when absent; the first `max_chars` kept; the rest 0. A text that cannot be normalized (unpaired surrogates) is read as it is, an invalid unit as U+FFFD. | | `bins` `[N, 4]` | `clamp(floor(bins * v), 0, bins - 1)` for `v` = x0/W, y0/H, x1/W, y1/H | | `feats` `[N, 8]` | x0/W, y0/H, x1/W, y1/H, w/W, h/H, `min(h / unit, 8) - 1`, `a / (1 + a)` with `a = w / h` | | `geo` `[N, 4]` | the box divided by `unit` | ## Graph Width `d` (192), `layers` (5) pre-layer-norm blocks with `heads` (6) heads and an `ffn` (512) feed-forward layer with tanh GELU; `char_dim` 48, `char_conv` 128, `pair_hidden` 32, `rel_dim` 64. 1. Word vector: character embedding, convolution (kernel 3, zero padding), ReLU, maximum over the word's valid positions (`char_pool`), linear `char_proj`. Add the rows `pos_x0[bins0]`, `pos_x1[bins2]`, `pos_y0[bins1]`, `pos_y1[bins3]` (in that order) and `feat_proj(feats)`; layer norm `emb_ln` gives `emb`. 2. Pair bias `[layers * heads + 2, N, N]`: 13 features of word *j* relative to word *i* (numerics.md) through `pair1`, ReLU, `pair2`. Channels `6l .. 6l + 5` are the head biases of block *l*; the last two are the biases of the two pointer relations. 3. Block *l*: `u = ln1(x)`; `q, k, v = qkv(u)` (each `[N, heads, head size]`, in that order along the 3d axis); per head `softmax(q kᵀ / sqrt(head size) + bias) v`; `x += proj(heads)`; `x += fc2(gelu(fc1(ln2(x))))`. 4. `final = final_ln(x)`. Heads: `tags = tag_head(final)` (`[N, 1 + 2 * classes]`: 0 = O, `1 + 2c` = B-class *c*, `2 + 2c` = I-class *c*), `roles = role_head(final)` (O, B-KEY, I-KEY, B-VAL, I-VAL), `doc = doc_head(mean of final)`, and the pointers `rel[r, i, j] = q_i,r · k_j,r / sqrt(rel_dim) + pair_bias[layers * heads + r, i, j]` with `q = rel_q(final)`, `k = rel_k(final)` viewed `[N, 2, rel_dim]`. Relation 0 (item): a word of a line item or tax row points to the first word of that item or row (the anchor), the anchor to itself. Relation 1 (key/value): a VAL word points to the first word of its key, a KEY word to itself. ## Windows A page of at most `max_words` words is one window. A longer page is run in windows of `max_words` words starting every `max_words * 3 / 4` words (512 and 384), the last window ending at the last word: starts 0, s, 2s, … while `start + max_words < N`. A word belongs to the window whose centre is nearest the word's centre (`(start + end) / 2` against `index + 0.5`; the earlier window on a tie) and takes its tag logits, role logits and pointer row from it. Pointers are scored inside a window only; the pointer target is the window-local argmax (the lowest index on a tie) shifted by the window start, so a pointer never leaves its window. The document-type logits of a multi-window page are the mean of the windows' logits weighted by the number of words each window owns, in double precision, rounded to fp32. ## Assembly 1. Document type = argmax of the document logits (lowest index on a tie); its probability is the softmax. A caller that asks for a schema (`ExtractInvoice`, `ExtractReceipt`) reads with that schema's classes whatever the model predicts and gets a note when the two differ; the predicted type is still reported. 2. Tag classes of the other schemas are masked: for `invoice` the classes without the prefix `receipt.`, for `receipt` those without `BT-`, for `other` those with neither prefix. A class name with neither prefix belongs to every type (a model with its own classes). 3. Per word the tag is the argmax over the allowed tags (the O tag included; the lowest index on a tie); its probability is `exp(l - max) / Σ exp(l' - max)` over the allowed tags, summed in index order. 4. A span is a B tag plus the following I tags of the same class on consecutive words; an I tag whose previous word is not in a span of its class starts a span. Span confidence = the mean probability of its words' tags (double precision). 5. Header classes (not item or tax classes) keep their highest-confidence span (the first on a tie). Item classes (`BT-126 BT-129 BT-130 BT-131 BT-146 BT-152 BT-153`, `receipt.itemName receipt.quantity receipt.unitPrice receipt.amount`) and tax classes (`BT-116 BT-117 BT-119`): the span's first word follows relation 0 to an anchor (the pointer of the pointer, at most 4 steps; stop at a word that points to itself; on a cycle, the smallest index on it; after 4 steps, wherever it is). Spans with the same family and anchor form one item or tax row, each class once (highest confidence), ordered by anchor; fields in an item are ordered by their first word. 6. Roles: the argmax per word, spans as above for KEY and VAL; a VAL span pairs with the KEY span that contains the word its first word points to by relation 1; otherwise it is dropped. The pair's confidence is the smaller of the two spans'. 7. The text of a span is its words joined by one space. Differences from taking the argmax of float32 probabilities: the library takes the argmax of the logits (softmax is monotone; it differs only where two probabilities round to the same float), and its probabilities come from the library's own exp, so confidences agree with a float32 NumPy softmax to about 1e-6, not bit for bit. ## Normalization Each field carries the text as read and a normalized `Value` derived from it by the same code the grounding of LLM answers uses ([grounding.md](https://recursiveai.org/oneocr/docs/grounding/)): dates (`BT-2 BT-9 BT-72 receipt.date`) to `yyyy-MM-dd` (`--MM-dd` without a year; a numeric date that reads two ways under the page's date order stays as read), amounts (the money classes) to a canonical decimal with `.` as the decimal mark, quantities and rates (`BT-129 receipt.quantity`, `BT-152 BT-119`, the sign `%` dropped) likewise, `BT-5` to an ISO 4217 code when the text names exactly one. The number style of the page is the one its unambiguous numbers use (point decimal when none is). ## Limits `OcrLimits.MaxWordsPerPage` (words read from a page), `MaxWordsPerWindow`, `MaxCharsPerWord`, `MaxPositionBins`, `MaxTransformerLayers`, `MaxExtractionClasses` and the existing model-file, tensor and activation limits bound the model file and the work before anything sized by them is allocated. Memory per window is dominated by the pair bias, `(layers * heads + 2) * max_words² * 4` bytes (33 MB at 512 words, 5 layers, 6 heads), so a page of any length costs the same memory and time linear in the number of windows. ## Fine-tuning a custom schema A custom model is a `docextract-v0` file whose `config.classes_list` holds the schema's field names (`classes`, `tags = 1 + 2 · classes` follow), with a `tag_head` of that size and every other tensor as in the model it was tuned from. The container format, the manifest members and the tensor names do not change, so every reader that reads the shipped file reads a custom one. Rules for the class names: not empty, at most 128 characters, no `/` or `~` (they are JSON pointer paths), no duplicates, and none starting `BT-` or `receipt.` (those prefixes select the invoice and receipt rules). A class with neither prefix is a once-per-page field allowed in every document type; the item and tax groups exist only for the built-in classes. The new head starts with weights uniform in ±1/√d from a SplitMix64 stream of the seed and zero biases, rounded to fp16 like every stored weight. A tuned file is written from the fp32 master weights rounded to fp16 once, with the runtime layouts mapped back (character-convolution taps from (tap, channel) to (channel, tap), the padded feature columns dropped). Training itself is specified in [docextract-backward.md](https://recursiveai.org/oneocr/docs/docextract-backward/). # PDF bridge (`RecursiveAI.OneOcr.Pdf`) How a PDF page becomes an `oneocr.page` page. All PDF parsing is OnePDF's and all drawing is the OneImage PDF bridge's; this package only converts, decides what to trust, and calls the engine's page entry point (`OcrEngine.Ocr`) on rendered regions. ## Versions `RecursiveAI.OnePdf` 1.10.1 and `RecursiveAI.OneImage.Pdf` 1.10.4 (the versions in `src/OneOcr.Pdf/OneOcr.Pdf.csproj`). OneImage.Pdf pins the exact `RecursiveAI.OneImage` version, so 1.10.4 is the one that matches the core package's OneImage 1.10.4 floor. ## Coordinates A PDF page is `OcrPage` with `Width`/`Height` in points of the viewed page (box, `/Rotate` and `/UserUnit` applied; the crop box is used), `UnitsPerInch` 72, origin top-left, y down. OnePDF's extraction is y up in the same upright space, so a text-layer box `[l, b, r, t]` becomes `[l, H − t, r, H − b]`. `ToPdf` maps a page-unit point back to user space (`x' = a x + c y + e`, `y' = b x + d y + f`), for the crop box `[l b r t]`, width `w`, height `h`, user unit `u` and `k = 1/u`: | `/Rotate` | `[a b c d e f]` | |---|---| | 0 | `[k 0 0 −k l b+h]` | | 90 | `[0 k k 0 l b]` | | 180 | `[−k 0 0 k l+w b]` | | 270 | `[0 −k −k 0 l+w b+h]` | Text-layer glyph boxes are OnePDF's: 0.9 em above the baseline to 0.2 em below, from the glyph origin to its advance. Mapped through `ToPdf` a word's left edge lands on the glyph origin and its edges on the baseline ∓ those distances; the tests hold this to **0.6 pt** in user space for upright, rotated, cropped and `/UserUnit` pages. Recognized words come from a render at `RenderDpi` and carry the recognizer's ink boxes, so their left edge is only held to within 4 pt (CTC alignment). ## What is read how 1. The text layer is extracted in reading order with glyphs retained. Words, lines, blocks, roles (header/footer), tables and cell starts carry over; words and characters have `Source = TextLayer`, confidence 1 (the file says so; it is not a calibrated probability). 2. **Invisible text** (render mode 3 or fully transparent) is somebody else's OCR layer. By default it is dropped and every image it sat on is read by the recognizer. `TrustExistingOcrLayer` keeps it as `TextLayer` and the images under it are not read. 3. **Garbled text**: a line with a word that contains U+FFFD, a private-use code point or a control character is dropped and its box (padded by 15% of its height) is rendered and read. 4. **Cross-check**: up to `CrossCheckLines` (3) evenly spaced, visible, left-to-right lines are rendered, read, and compared with the text layer on letters and digits (case-folded, Levenshtein distance more than 40% of the longer string = disagreement). If at least half the checked lines disagree, the text layer is dropped and the whole page is read from pixels. This catches a `ToUnicode` map that is wrong but plausible. 5. **Images**: every image the page draws (from the interpreter's device calls, not inside Type 3 glyphs) of at least `MinImagePoints` (24) in both sizes, whose area carries less than 10% text-layer word boxes and fewer than three trusted invisible words, is read. A page with no text at all therefore reads its images, and a scan drawn full-page is one region. 6. A region is the crop of the page rendered at `RenderDpi` (300), passed to `OcrEngine.Ocr` with that DPI, and its blocks, tables and regions are scaled back to points. Recognized words that overlap an accepted word by at least 40% of the smaller box (text drawn over a picture, or two regions that overlap) are dropped, text layer first; tables of a region that lost words are not kept. 7. **Order**: text-layer blocks keep OnePDF's reading order; each recognized region goes before the first text-layer block that starts below its top. Blocks, lines and words keep their own `Source`. Failures (a page that cannot be rendered, an image that is too large, a region over an OCR limit) leave a diagnostic on the page and the rest of the page is still read; `Strict` throws instead. Damaged files raise `OcrFormatException`, files over a limit `OcrLimitException`; nothing else comes out of a public method except argument errors, I/O errors and cancellation. Not done: outlined text (glyphs drawn as paths), annotations (they are neither read nor rendered), vector text-as-art. # Grounding, text merge and read outputs (v1) Behavior of `LlmClient.ReadAsync` in `RecursiveAI.OneOcr.Llm`. Everything here is deterministic given the model's answer and the page model: no clock, no locale, no randomness. A port must reproduce it exactly. ## Read outputs `LlmOutputs` flags: `Text`, `Markdown`, `Json`, `PageModel`. One model call produces every requested output. The call asks for one JSON object with these members, present only when asked for: | member | when | content | |---|---|---| | `pages` | `Text` or `PageModel` | array of strings, one per page image sent: the exact text of the page, lines separated by a newline | | `markdown` | `Markdown` | array of strings, one per page image sent: the page as Markdown | | `data` | `Json` | the caller's schema with every scalar replaced by `{ "value", "words", "derived" }` (below) | The page images and the word list are the existing request: words are listed as `[id, text, left, top, right, bottom]`, ids `w0`, `w1`, … in the order of the pages' word sequences, masked words left out. Results: `Text` (merged), `LlmText` (the model's own), `Markdown` and `MarkdownFlags`, `Json` and `Fields`, `Pages` (merged when `PageModel` was asked, otherwise the input pages), `Disagreements`, `Calls`, `Usage`, `CostUsd`, `Diagnostics`. Pages are joined by a blank line (`\n\n`). An answer that does not have the requested shape never throws on its own: a missing or short `pages`/`markdown` array gives empty pages and a diagnostic; a missing `data` gives all-null values. ## Wrapped schema The caller's schema is read into a tree: objects with `properties`, arrays with `items`, scalar leaves (string, number, integer, boolean, enums). Supported shapes: `$ref` to `#/$defs/…` or `#/definitions/…` (inlined, at most 8 levels, at most 20,000 nodes), `anyOf` of exactly one type and `null`, type arrays containing `null`. Not supported (the call throws `ArgumentException` before anything is sent): `oneOf`, `allOf`, `prefixItems`, objects without `properties`, arrays without `items`. Every object lists all its properties as required and forbids others. A leaf becomes ```json { "value": , "words": ["w3","w4"], "derived": false } ``` `value` is `null` when the value is not printed. `words` are the ids of every word the value was read from. `derived` is true only for values computed or inferred rather than printed. The plain JSON returned to the caller is rebuilt from the tree and the answer; it has the caller's shape (missing values are `null`, arrays keep the model's items). Paths are JSON Pointers: `/items/0/amount` (`~` is `~0`, `/` is `~1`). ## Grounding one value Input: the value as text (strings as they are, numbers and booleans as their JSON text), the cited ids, the derived flag, and all listed words with their calibrated confidence. Steps, first match wins: 1. **Boolean**: state `Derived`, confidence 0.5, never boxed. 2. **Cited words**: look for the value in the cited words (ids sorted ascending, duplicates and unknown ids dropped; unknown ids are noted). A match of kind *exact* or *normalized* is taken: `Grounded` or `GroundedNormalized`. 3. **Page search**: look for the value in all listed words. Take the best match: exact before normalized before part-of-word before interleaved; among equals the one nearest a cited word (smallest word-index distance), then the first. The field is marked `CitationCorrected`, with a note saying whether nothing was cited or the cited words did not hold the value, and how many places matched. If the cited words held only a part-of-word or interleaved match and the page has no better one, the citation is kept and not marked corrected. 4. **Derived**: if the model said `derived`: for an amount, check whether the cited amounts reproduce it (below); state `Derived`, confidence 0.9 when reproduced, 0.3 otherwise. Derived values are never boxed. 5. **Ungrounded**: confidence 0. The note names the closest page text when one is within 40% edit distance (page of at most 5,000 words, value of at most 64 folded characters). ### Matching a run of words A *run* is consecutive words in the scope (the cited list, or the page). Strategies, all tried; their hits are pooled: - **Text.** Fold both sides, then compare the run's folded concatenation with the value's folded text after trimming `. , ; :` from both ends. Fold = Unicode NFKC, typographic quotes to `'` and `"`, dashes and minus to `-`, lower case (invariant), every whitespace, format and control character removed. A value that is only punctuation is compared untrimmed. Runs grow word by word, up to 64 words, and stop at the first word where the concatenation can no longer be a prefix of the value. The hit is **exact** (state `Grounded`) when the run's text with whitespace removed equals the value with whitespace removed, case and punctuation included; otherwise **normalized**. - **Amount.** If the value parses as an amount: single words and pairs of words parsed as amounts; equal decimal value and compatible currency (either side without currency is compatible; otherwise the sets of ISO codes must intersect). Normalized form: below. - **Date.** If the value parses as a date: runs of 1 to 4 words (starting at a word with a digit or a month name) parsed as dates. Match when month and day are equal and the years are equal or either side has none. A year-less run is rejected when the next word would extend it into a date that has a year (`24 October` inside `24 October 2024`). Hit flags: `YearInferred` when the page has no year, `DateAmbiguous` when the page's reading of day/month was not unique. - **Currency.** If the value is an ISO 4217 code from the table: any word that is that code or a symbol standing for it. - **Interleaved.** Only when nothing else matched and the value has several words: each word of the value, folded and trimmed, equals a word in order with at most 3 other words between neighbours. Only the matching words are boxed. - **Part of a word.** Only when nothing else matched and the folded value has at least 4 characters: one word contains it with no letter or digit directly before or after. ### Amounts An amount is optional currency, optional sign, digits with separators, optional currency, optional sign. Accepted: a currency symbol or ISO code at the start or end (`€`, `$`, `US$`, `£`, `¥`, `₹`, `R$`, `zł`, `kr`, `CHF`, …; `$` stands for USD, CAD, AUD, NZD, MXN, SGD, HKD, ARS, CLP, COP); a leading `-`, `−`, `+`, a trailing `-`, or parentheses for negative; digit groups separated by `.`, `,`, space, no-break space, narrow no-break space, thin space or `'`. Decimal mark, given the page's **number style** (`PointDecimal` or `CommaDecimal`): - both `.` and `,` present: the last one is the decimal mark; - one kind, more than once: group marks (every group after the first has exactly three digits); - one kind, once: group mark when exactly three digits follow it, one to three digits come before it and the number does not start with `0` (`1,234`, `1.234`); then it is decimal only if the style's mark is that character. In every other case (`12,5`, `0,125`, `1234,567`) it is the decimal mark; - spaces and apostrophes are always group marks. The **page number style** (`NumberStyle.Auto`) votes over listed words that look numeric: both marks present, last is the decimal; one mark repeated: the other mark is the decimal; one mark once followed by one, two, or four or more digits: that mark is the decimal. More comma votes than point votes: comma; otherwise point. A caller can pin the style. A JSON number value is read as an invariant decimal. Canonical text: `.` decimal mark, no grouping, trailing zeros removed, at least two decimals when it has any (`12.5` is `12.50`, `12` is `12`, `0.125` is `0.125`). ### Dates Tokens are digit runs and letter runs; other characters (space `.` `,` `/` `-` and Unicode hyphens, no-break space) only separate; any other character rejects. Ordinal suffixes (`st nd rd th er º`) and the filler words `of de del der the le den am on` are ignored. At most three tokens remain, at most one letter token, which must be a month name or abbreviation in English, German, French, Spanish, Italian, Portuguese, Dutch or Turkish (table in the source). Readings: eight digits → year, month, day. Three numbers: first of four digits → year, month, day; otherwise day/month first two numbers (1–2 digits) and a last number of two or four digits as the year. Two-digit years: 00–69 → 2000–2069, 70–99 → 1970–1999. If the first number is above 12 it is the day; if the second is above 12 it is the day; equal numbers read the same either way; otherwise the **date order** decides, `DayFirst` or `MonthFirst`. With `DateOrder.Auto` the order is the page's: among listed words of the shape `n.n.yyyy`, `n/n/yyyy`, `n-n-yyyy` with only one number above 12, the majority; with no evidence both readings are kept and the field is flagged `DateAmbiguous`. With a month name: day and year (`3 Oct 2026`, `Oct 3, 2026`, `2026 Oct 3`) or day only (year missing). A date must exist (`31.02.2026` is rejected; 29 February is accepted without a year). Canonical: `yyyy-MM-dd`, or `--MM-dd` without a year. Times of day are not normalized (they match as text). ### Derived amounts Over at most 16 cited words that parse as amounts: the value equals the sum of any subset of two or more; or `a − b`; or `a × b` rounded to two decimals; or `a × b / 100` rounded to two decimals (percentages). Midpoints round to even. ### Confidence `confidence = base × mean(word confidence)`, clamped to 0..1, where the mean runs over the boxed words' calibrated confidence and `base` is the product of: exact 1.0, normalized 0.95, part of a word 0.9, interleaved 0.8; × 0.85 when the citation was corrected; × 0.8 more when the page search found several places; × 0.7 when the year was inferred; × 0.8 when day and month were ambiguous. Ungrounded 0; derived 0.9 or 0.3; boolean 0.5. ### Boxes One box per line of backing words: consecutive words on the same page whose vertical overlap with the previous box is at least half the smaller height are united. ## Re-ask After the first answer, fields in state `Ungrounded` or `RoleConflict` (not `Derived`) are re-asked once if `Reask` is true (default). The second call carries the same images and word list and a second delimited block `<<>>` with one JSON array per field (at most 64): `[path, value given, text of the cited words, closest page text or null]`; for a role conflict the last item is the value the local model reads for that path, when it has one. It asks for the data member only, in full. For each re-asked path present in the new answer, the new leaf replaces the old and is grounded again (`Reasked` true). A path absent from the new answer, or null there, is withdrawn: its value is `null`, it is not in `Fields`, and a diagnostic says so. Other fields keep the first answer. The second call goes through the same client as every call: spend caps, per-call bounds, concurrency and retries apply. If it is refused (`OcrBudgetException`) or fails (`OcrProviderException`), the first results stand and a diagnostic says why. There is never a third call. `UngroundedPolicy` then applies to fields still `Ungrounded` or `RoleConflict`: `Flag` keeps the value in the JSON (the field says it is ungrounded); `Null` writes `null`; `Throw` throws `OcrProviderException` with `LlmFailure.Ungrounded`, naming up to eight paths. ## Text merge (`LlmOutputs.Text`) Per page sent: the OCR words (the page's word sequence) and the model's tokens (split on whitespace) are aligned. **Keys.** A word's key is its fold with `. , ; :` trimmed from both ends. **Alignment.** Global alignment over keys, minimizing cost: a gap costs 1000; a pairing costs 0 for equal keys, otherwise `400 + floor(1600 × d / max(len))` with `d` the Levenshtein distance of the keys; ties prefer pairing, then dropping the OCR word, then the model's word. At most 2^22 cells are computed in full; larger pages use a band of half-width `max(256, |n − m| + 64)` around the diagonal; if that still needs more than 64M cells the page keeps its local text and a diagnostic says so. **Merge**, per aligned pair of OCR word `o` and model token `t`: - same text (ordinal): the word, source of the OCR word, confidence unchanged; - different text and calibrated `o.Confidence ≥ ConfidenceThreshold`: the OCR word wins (source of `o`); - different text otherwise: the model's text, source `Llm`, confidence 0.5, box of `o`. No calibrated estimate exists for model words; 0.5 marks "uncertain". Only the model: kept with source `Llm`, confidence 0.5 and a zero-width box at the right edge of the previous OCR word (the left edge of the next one at the start of a page) unless `KeepLlmOnlyWords` is false. Only OCR: kept (after the previous word) when confident, dropped otherwise. Each of these, and each substitution, is a `TextDisagreement` with the choice made; `Start` and `Length` locate the chosen word in `Text` (a dropped word has length 0). Text is rebuilt in the model's order with the model's separators; OCR-only words that are kept are inserted after the previous word with one space. **Merged page model.** `Pages[i]` is the input page with each word replaced by its merged word (`Source`, `Confidence`; characters are kept only where the text is unchanged), dropped words removed, model-only words added to the line of the previous word (cells unchanged), empty lines removed, table cells and key–value pairs remapped to the new word indices (removed words leave them; pairs left with no key or no value go). Blocks and regions are kept. A diagnostic is added. ## Markdown flags The model's Markdown per page is split on whitespace. In each piece, leading `# > * _ | ~ ` ` and trailing `* _ | ~ ` ` are syntax; pieces made only of `- + | : # > * _ ~ ` =` are not words. The remaining words are aligned with the OCR words as above. A flag is raised for a pair whose keys differ (so case and edge punctuation do not flag), for a model-only word, and for a confident OCR word the Markdown lacks (zero length, at the position of the next word). `Start`/`Length` are offsets into `Markdown` covering the word without its syntax characters. `MarkdownWithFlags` wraps flagged spans in `⟦ ⟧` by default. ## Routing - `LlmOnly` (default): the model is called for every requested output. - `LocalOnly`: the model is never called, even if configured or when offline. `Text` and `PageModel` are the local text and pages; `Markdown` is not produced (diagnostic); `Json` needs an `ILocalFieldExtractor` (none ships) or the call throws `ArgumentException`. - `LocalFirst`: `Json` is routed per field when a local extractor is set (`LlmReadOptions.LocalExtractor`; `DocumentFieldExtractor` wraps a `DocumentExtractor`, the shipped invoice and receipt model or a custom one). `Markdown` always needs the model. For `Text` and `PageModel` only pages with a word below the threshold are sent; the others keep their local text. No page to send means no call. ### Per-field `LocalFirst` The extractor returns the schema's values as JSON plus a list of fields (schema path, value, words, confidence). Then: 1. A scalar is **local** when the list has it at `ConfidenceThreshold` or above, in state `Grounded`/`GroundedNormalized`, with a non-null value in the JSON. A scalar the extractor lacks, is unsure of, or reads as null is **missing** and goes to the model. 2. An array is local only **as a whole**: the extractor has at least one element and every value of every element is local. Otherwise the whole array goes to the model, because the model and the local reader may disagree on how many elements there are. 3. The model is asked for the **reduced schema**: the same schema with the local scalars and arrays removed (objects left empty are removed too). No local values are sent as hints. If nothing is left, no call is made, nothing is sent and nothing is spent. 4. The result merges both: local values keep their fields with `Source = Ocr` (the model labelled words on the page; the words' own source is on the page model), model values are grounded as usual with `Source = Llm`. In the JSON, a local array is the extractor's, a model array the model's. The estimate, the cost check and `SpentUsd` are those of the reduced request, because that is the request that is built. Diagnostics say how many values were kept. 5. An extractor that lists no fields (only a JSON and a `MinConfidence`) keeps the earlier whole-document rule: its result is used when `MinConfidence` reaches the threshold, otherwise the model answers every value. `LocalOnly` returns the extractor's JSON and fields and never calls the model. `Offline` makes every call throw, so a `LocalFirst` read that needs the model throws before anything is sent, while one that is fully local makes no call and does not throw. `DocumentFieldExtractor` matches a schema property to a class of the model: first by the property's name equal to a class name (case and punctuation ignored), then by a class named as a whole token in the property's description (`BT-112`), then by a short list of common words for EN 16931 terms and receipt fields (`total`, `invoiceNumber`, `vendor`, `iban`, `qty` and so on). Where an invoice and a receipt class share a word, the one the model found is used. An array of objects takes the model's line items, or its tax rows when its name contains `tax` or `vat`; element *i* is group *i*. A property without a class, or whose class was not found, is missing. A number-typed property is written as a JSON number from the field's parsed number (missing when the text did not parse). The field's confidence is the model's mean tag probability, not a calibrated one. ## Role check Grounding checks that a value is on the page, not that it is in the right role. When a local extractor is set (any routing except `LocalOnly`; `LlmReadOptions.RoleCheck`, default true), every model value in state `Grounded` or `GroundedNormalized` is also checked against the local reading, using only fields the local model holds at `ConfidenceThreshold` or above: - one of the cited words is a word of a local field of **another path** (or of a field outside the schema, `LocalFieldResult.OtherFields`); or - the local model reads **this path** from other words (none shared with the citation) and its value, compared after normalization, is different. Only a property the local model has a class for is judged (`LocalFieldResult.MappedPaths`: the schema paths, array indices left out, that some class of the model answers to, whether or not it found a value on this page). For a property with no class the local model says nothing about what it means, so its cited words belonging to another class is no contradiction and the value stays as grounding left it. A local extractor that gives no set is taken to have a class for every property. `DocumentFieldExtractor` fills the set with the properties its name, description and alias matching maps to a class. The field then gets state **`RoleConflict`** (a state of its own rather than a flag on `Grounded`, so code that trusts `Grounded` cannot overlook it). It keeps its words and boxes, has confidence 0 and a note saying what the local model reads. It counts like `Ungrounded` for the re-ask (the hint is the local value), for the check after the re-ask, and for `UngroundedPolicy`. The check costs no call. A value the local model does not know (a missing field, a low-confidence one) is not checked: no local reading is not a conflict. Derived values and values without cited words are not checked. ## Prompt injection Page text and the model's own earlier answer are untrusted. They reach the model only as JSON-escaped array elements, one per line, between marker lines whose tag is the SHA-256 of the block (so a page cannot forge a closing marker), with a system prompt saying to treat them as data; the model has no tools and answers in a fixed schema. Nothing from a model answer is executed or used as a path, id or URL except as a lookup key into the word list. What grounding catches: a value the page does not contain (invented, leaked from the instructions, an injected number that is not printed), a citation that points elsewhere (corrected and flagged), a computed value that nothing reproduces (`Derived`, 0.3). What it cannot catch: the model choosing a different number or text that is really printed on the page, including hostile text printed on it; and a printed value in the wrong role (a vendor name in the total field). Those are grounded because they are on the page; with a local extractor the role check above catches them whenever the local model confidently reads the field (the `printed-other-number`, `tiny-text-read` and `list-spoof` cases of the injection set become `RoleConflict`); only the cases the local model is unsure of remain. Mitigations are the user's: redaction, a schema with descriptions, and checking boxes and confidences for fields that matter. The model-answer set in `tests/OneOcr.Tests/LlmInjectionTests.cs` records, per attack, what grounding reports. # Review queue and fine-tuning work directory, version 1 Status: current. The recognition fine-tuner (`OcrTrainer`, `oneocr train`) keeps everything in one work directory, and the review queue is a pair of JSON files around it. ## Agreement rule A page's lines come from the local reader (line finder, then the recognizer with greedy decoding). The labelling model's transcription of the page is split at whitespace. Walking the lines top to bottom, each line is compared (NFC, whitespace collapsed, case and punctuation kept) with every stretch of the model's words that starts within 80 words of the end of the previous match and is within 3 words of the line's own length; the stretch with the lowest character error rate wins (the earliest on ties). Character error rate is edit distance in code points over the line's length. - **Agreed**: the best rate is at most `MaxAgreementCer` (default 0: the same text). The model's text is the training label. - **Disagreed**: the best rate is above `MaxAgreementCer` but at most `AlignMaxCer` (default 0.5). The line goes to the review queue. - **Unaligned**: no stretch is within `AlignMaxCer`, or the model gave no text. The line goes to the review queue. Only agreed lines and lines a reviewer corrected train the model. Lines whose label has a character outside the model's charset are left out and counted. ## Work directory `labels/.json` (`oneocr.labels`, version 1): one file per document, `` = first 16 hex digits of the SHA-256 of the image file. Holds each line's box, local reading, confidence, the matched model text, its state and the crop the recognizer saw (8-bit gray at the model's input height, base64). `reviewed.json` (`oneocr.reviewed`): the corrections by line id. `checkpoint.bin`: step, weights and AdamW moments (fp32, little-endian), tied to the run by a SHA-256 over the starting model, the settings and every label. `report.json`: the last run's report. A line id is `-`. ## Review file (`oneocr.review`, version 1) ```json { "format": "oneocr.review", "version": 1, "items": [ { "id": "0123456789abcdef-3", "document": "scan.png", "box": [10, 20, 300, 30], "local": "Tota1 due", "llm": "Total due", "state": "disagreed", "confidence": 0.81, "crop": "crops/0123456789abcdef-3.png", "text": null } ] } ``` The export writes this file and one PNG per line. A reviewer sets `text`. The importer reads only `id` and `text` and skips everything else. It is untrusted input: the whole file must be valid UTF-8 and at most `MaxReviewImportBytes` (16 MiB); there are at most `MaxReviewItems` items (100,000); `MaxJsonDepth` bounds nesting; a text is at most `MaxReviewTextLength` (1,024) UTF-16 units and, like an id (at most 128, not empty), holds no control characters and no U+FFFD; `items` appears once. A `text` that is missing or `null` leaves the line in the queue; an empty `text` says the line has no text and drops it from training. Ids that name no line of the work directory are ignored and counted. Text is stored with whitespace normalized. ## Training schedule Batches are drawn from a seeded (SplitMix64) permutation of the training lines, reshuffled each pass. The learning rate rises linearly over the warm-up and then falls linearly to a tenth of the peak at the last step. Batches, schedule, loss and weights depend only on the seed, the labels and the settings, so a resumed run ends with exactly the weights of an uninterrupted one. # Backward numerics How gradients, the CTC loss and the optimizer step are computed, in an order that makes them bit-identical across CPU paths (scalar, 128-bit vectors, AVX2, AVX-512), operating systems and degrees of parallelism. A port that follows these rules reproduces the C# results exactly. It extends `numerics.md`: all arithmetic is IEEE 754 binary32 unless stated, `fma` is the fused multiply-add, and a "dot product" is the 16-lane dot product defined there (element *k* into partial *k mod 16*, then the fixed pairwise tree). Gradients are for the `crnn-v0` recognizer; weights are the fp32 values widened from the container's fp16, and fine-tuning updates fp32 master copies of them. ## Principle Every sum over many terms is a dot product in the library's order; every other operation is element-wise. A gradient is therefore a set of independent outputs, each with one fixed order, so tiling and threading cannot change a bit. Sums that would otherwise be scatter-adds are written as gathers (each output reads its terms in a stated order), and the per-line gradients of a batch are added in line order. No reduction crosses threads. ## Layers Notation: `dY` is the gradient of a layer's output, `T` the number of positions or time steps, activations are channels-last as in the forward pass. - **Linear** (`y = x Wᵀ + b`, `T` rows): `dW[o][i]` = dot over t of `dY[t][o]` and `x[t][i]`; `db[o]` = dot over t of `dY[t][o]` and 1; `dx[t][i]` = dot over o of `dY[t][o]` and `W[o][i]`. The dot products read the operands as rows of the transposed matrices, so the lane of term *t* (or *o*) is *t* mod 16 (or *o* mod 16). - **Sum of a row** (bias gradients) is always the dot product with a row of ones: `fma(x, 1, p)` adds exactly, so this is the 16-lane sum. - **ReLU and max pooling.** Forward: the pooled value is the maximum over the window starting from 0, scanning rows then columns, replacing the running maximum only when strictly greater (so the first of equal maxima wins and a NaN never wins). The winner's flat index in the convolution output is recorded, or "none" when no value was positive. Backward: the pooled element's gradient is assigned to the winner; "none" passes nothing (this is the ReLU's zero gradient at and below 0). Trailing partial windows are dropped, as in the forward pass. - **Convolution** (taps in (ky, kx, channel) order, rows padded with zeros to a multiple of 16, as in `numerics.md`). With `P` output positions, `dC[p][o]` the gradient of the convolution output (after the pool backward) and `col[p][k]` the input windows used by the forward pass: - `db[o]` = dot over p of `dC[p][o]` and 1; - `dW[o][k]` = dot over p of `dC[p][o]` and `col[p][k]` (zero in the padding columns, whose window values are zero); - `dcol[p][k]` = dot over o of `dC[p][o]` and `W[o][k]`; - `dx[iy][ix][c]` = the sum of `dcol[p][(ky·kw + kx)·C + c]` over the taps (ky, kx) whose window covers the input element, p being the output position `(iy + padH − ky, ix + padW − kx)`; started from +0 and added in ascending ky, then ascending kx, with plain additions. The first block's input gradient is not computed. - **LSTM** (per direction, hidden size H, gate order i, f, g, o; `b = b_ih + b_hh` is one parameter, and its gradient is the gradient of either bias). The forward pass stores the activated gates, the cell state, `tanh` of the cell state and the hidden state each step started from (zeros for the first step visited). The backward pass visits the steps in the reverse of the forward visiting order. For each element j, with `dh = dOut[t][column + j] + dhRec[j]` (`dhRec` from the step visited next, zero for the last): ``` dtc = dh * o do = dh * tc dc = dcNext + dtc * (1 - tc * tc) di = dc * g dg = dc * i df = dc * cPrev dcNext = dc * f dz_i = di * (i * (1 - i)) dz_f = df * (f * (1 - f)) dz_g = dg * (1 - g * g) dz_o = do * (o * (1 - o)) ``` every operation rounded separately (no fused multiply-add), `cPrev` = 0 for the first step visited. `dhRec` for the previous step is the dot product over the 4H gates of `dz` and `W_hh[·][j]`. After the recurrence, with `dz` stored by time step: `dW_ih[g][i]` = dot over t of `dz[t][g]` and `x[t][i]`; `dW_hh[g][j]` = dot over t of `dz[t][g]` and `hPrev[t][j]`; `db[g]` = dot over t of `dz[t][g]` and 1 (t ascending by time index in all three). The input gradient `dx[t][i]` is the dot product over the 8H gates, forward direction's 4H first and then the reverse direction's, of `dz[t][·]` and `W_ih[·][i]`. - **Classifier** is a linear layer on the top LSTM output. ## CTC loss Definition: Graves et al. 2006 (sections 3.1–3.2, 4.1). Blank is class 0; targets use classes 1 and up. Double precision, scalar, using the double `exp` and `log` of `decoding.md` (`DetMath`) so the result is the same everywhere: 1. Log softmax per step: `m` = maximum logit; `s` = sum over classes in ascending order of `exp(z − m)`; `lp[k] = (z[k] − m) − log(s)`. 2. Forward and backward variables over the blank-extended label sequence (2L + 1 entries), log domain, `logadd(a, b) = a + log(1 + exp(b − a))` for `a ≥ b` (returning `a` when `b − a < −40` or `b` is −∞); both include the emission of their own step. Skips of two entries are allowed only between different non-blank labels. 3. `logP` = `logadd` of the last two forward entries at the last step; loss = `−logP`. When `logP` is −∞ (no alignment) the loss is +∞ and the gradient is zero; the training step then counts the line as 0 (`zero_infinity`). 4. Gradient with respect to the logits: `scale · (exp(lp[k]) − exp(acc[k] − lp[k] − logP))`, where `acc[k]` is the `logadd`, in ascending entry order, of forward + backward over the entries labelled `k`. Rounded to binary32 once. **Reduction.** The batch loss is the mean over the lines of each line's loss, **not** divided by the target length (PyTorch's `reduction='mean'` also divides by it; the parity fixtures use the plain mean). A line's logit gradient is scaled by `1/B`. ## Batch gradient and clipping Per line: forward, loss, backward into that line's own gradient buffers. Then for every parameter element, `g = g₀`, `g += g₁`, … in line order (plain additions). The global gradient norm is the double-precision sum over tensors, in parameter order, of each tensor's dot product with itself, then the square root; when clipping is on and the norm exceeds the limit every gradient is multiplied by `float(limit / (norm + 1e-6))`. Parameter order: for each convolution block `weight` (`[Out][KPadded]`, the forward layout) and `bias`; for each LSTM layer, forward direction then reverse: `W_ih`, `W_hh`, `b`; classifier `weight`, `bias`. ## AdamW Loshchilov and Hutter 2019, algorithm 2, fp32 master weights, per element, each operation rounded separately, in PyTorch's single-tensor order. For step *t* (from 1), with the scalars computed in double and rounded once to binary32: ``` c1 = 1 − β1ᵗ, c2 = 1 − β2ᵗ (running double products β1ᵗ, β2ᵗ) stepSize = float(lr / c1) sqrtC2 = float(sqrt(c2)) decay = float(1 − lr · wd) (only for tensors flagged for decay) p = p * decay m = m + (1 − β1) * (g − m) v = v * β2 + (1 − β2) * (g * g) p = p − stepSize * (m / (sqrt(v) / sqrtC2 + eps)) ``` Decay is applied to weight matrices and, by default, not to biases (a setting). Square root and division are correctly rounded IEEE operations, so the step is identical everywhere. ## Tolerances Against float64 (a straightforward double-precision implementation written from the definitions, on random-weight models of the intended size, widths with and without tails): `|ours − reference| ≤ Atol + Rtol·|reference|`. | Layer | Atol | Rtol | Measured max (abs; rel where \|ref\| > 0.05) | |---|---|---|---| | conv (dW, db, dx) | 7e-6 | 3.5e-5 | 5.1e-6; 2.5e-5 | | LSTM (dW_ih, dW_hh, db, dx) | 1e-6 | 5e-6 | 5.9e-7; 5.2e-6 | | classifier (dW, db, dx) | 1e-6 | 5e-6 | 7.2e-7; 4.5e-6 | | CTC loss and logit gradient vs brute-force sum over all alignments | 1e-6 | 1e-6 | (loss within 1e-10) | | ReLU/pool routing, Im2Col/Col2Im adjoint | exact / 1e-6 relative | | | The LSTM gradient is of the network as computed, with the approximate sigmoid and tanh of `numerics.md`; the reference uses exact functions, which is most of the LSTM difference. Against PyTorch fixtures (a trained model, fp32 references with their float64 floor): `|ours − pytorch| ≤ 2·Atol + 2·Floor + 2·Rtol·|pytorch|` with `Floor` the fixture's recorded `max_abs` fp32-versus-float64 error of that tensor. Observed worst use of that allowance: conv 0.21, LSTM 0.34, classifier 0.30 (per-layer fixtures); CTC loss and logit gradient 0.42; optimizer step on stored gradients 0.001 of the allowance. Whole-model parameter gradients on a batch of four lines are within about 0.45 of the allowance for every tensor but one: the first convolution's weights differ by up to 10 floors at the taps of one output channel, which is a single ReLU/pool decision lying within rounding of a tie going the other way in the two implementations; the test allows one such flip. Weights after three real steps are compared only where the gradient is at least 50 floors in every step (below that the sign of the first Adam step is decided by noise in either implementation). # docextract-v0: backward pass and fine-tuning The training counterpart of [docextract-v0.md](https://recursiveai.org/oneocr/docs/docextract/), written so that another implementation can reproduce the gradients and the weights. It follows [backward-numerics.md](https://recursiveai.org/oneocr/docs/backward-numerics/) (every long sum a 16-lane dot product, everything else separately rounded IEEE operations, no fused multiply-add outside a dot) and [numerics.md](https://recursiveai.org/oneocr/docs/numerics/) (the library's `exp`, `tanh`). ## Training forward pass One page is one example and one window (at most `max_words` words). The forward pass is the inference pass of docextract-v0 with every activation kept; its outputs are bit-identical to inference (tested). Dropout does not exist. Layer-normalization inputs, the softmax probabilities of every head, the GELU `tanh` terms and the pair-MLP hidden activations are saved for the backward pass. ## Backward kernels - **Layer normalization** (`y = xhat * w + b`, `xhat = (x - mean) * inv`): per row with `g = dY * w`, `dX = inv * ((g - mean(g)) - xhat * mean(g * xhat))`, both means 16-lane dot products divided by `d`. `dw`, `db` are the column sums of `dY * xhat` and `dY` (dot products with ones along the row axis). - **GELU** (tanh form, constants of the forward pass): `dX = dY * (0.5 (1 + t) + 0.5 x (1 - t²) c0 (1 + 3 c1 x²))` with `t = tanh(c0 (x + c1 x³))` saved from the forward pass. It differentiates the function as computed, with the library's `tanh`. - **Softmax with bias** (`s = raw / scale + bias`): per row `ds = p * (dP - dot(p, dP))`; the pair-bias gradient is `ds`, the query-key gradient `ds / scale`. - **Scores and weighted sum**: `dQ[i][c] = Σj dS[i][j] K[j][c]`, `dK[j][c] = Σi dS[i][j] Q[i][c]`, `dP[i][j] = dot(dA[i], V[j])`, `dV[j][c] = Σi P[i][j] dA[i][c]`, all 16-lane dots (over `j` or `i` ascending). - **Pair-bias MLP** `pair2(relu(pair1(features)))` over the `n²` ordered pairs in row-major `(i, j)` order: weight gradients are 16-lane dots over pairs in that order; the features are data and get no gradient. - **Pointer layers** (`rel_q`, `rel_k`, items and key–value): the same score adjoint, with the pointer scores' own loss. - **Embeddings**: the gradient of a table row is the sum of the row gradients with that id, added in word order with plain additions on one thread. **Row 0 of `char_emb` (the padding id) receives no gradient** (PyTorch's `padding_idx`). Position tables likewise add in word order. - **Character convolution**: taps ordered (tap, channel) as in the runtime layout; its weight gradient is mapped back to the container order `(out, channel, tap)` only when a model is written. ## Losses Cross-entropy in double precision with the library's own exponential and logarithm: `Σ w[t] · nll / Σ w[t]` over words whose target is not negative (PyTorch `reduction='mean'`, `ignore_index`, `weight`). A negative target contributes neither loss nor gradient; no counted word gives loss 0 and zero gradients. The gradient `scale · w[t] / Σ w · (softmax - onehot)` is rounded once to binary32. Terms: tag (BIO over `1 + 2 · classes` tags), role, document type (one per page), item pointer and key–value pointer (a word points at its anchor word; targets are word indices). The loss of an example is the weighted sum of its terms; the batch gradient is the sum over examples in example order, each example scaled by `1 / batch size`. Examples may run in parallel, each into its own gradient set; the sets are added in example order, so no reduction crosses threads and the result does not depend on the degree. ## Parameters and optimizer Trainable tensors have a fixed order: `char_emb`, `char_conv` (weight, bias), `char_proj`, the four position tables, `feat_proj`, `emb_ln`, per block `ln1 qkv proj ln2 fc1 fc2` (weight, bias each), `pair1`, `pair2`, `final_ln`, `tag_head`, `role_head`, `doc_head`, `rel_q`, `rel_k`. AdamW follows backward-numerics.md with one addition: a per-tensor learning-rate scale (the tensor's step uses `scale · lr`). Weight decay applies to weight matrices and tables, not to biases or layer-norm parameters. Optional global-norm clipping uses the 16-lane dot of each gradient with itself. A head-only run (`BackboneScale` 0) trains the tag head and its bias only; every other tensor and every other loss term is off. With a positive scale every tensor the active losses reach trains at `scale ×` the head's rate. The role, document and pointer heads train only when their own loss is on. ## Tolerances Against PyTorch fp32 reference gradients (fixtures from `training/parity/docextract_backward.py`, seed 31, the random-7 model), per element `|err| ≤ atol + rtol · |ref|` with the tolerance chosen against a float64 reference as in numerics.md: PyTorch's own fp32 error is measured, and the C# error must stay within 2× of it plus the absolute floor. | layer | atol | rtol | |---|---|---| | layer normalization | 2e-7 | 1e-6 | | GELU | 2e-6 | 1e-5 | | attention (scores, softmax, weighted sum) | 3e-7 | 2e-6 | | pair MLP | 1e-6 | 5e-6 | | pointer scores | 3e-7 | 2e-6 | | cross-entropy | 2e-7 | 1e-6 | | whole model, every parameter | 1e-6 | 1e-5 | Measured worst use of the allowance is at most 0.14 for every layer except GELU (0.46, from the approximate `tanh`). C# to C# the forward values, loss, gradients and updated weights are bit-identical on every kernel path and at degrees 1 and 4. ## Fine-tuning a custom schema See "Fine-tuning a custom schema" in [docextract-v0.md](https://recursiveai.org/oneocr/docs/docextract/) for the container, and the label rules below. - **Labels.** The labelling model is asked for the schema's fields (each nullable). Only values the grounding step reports as grounded (exactly or after normalization) become labels; a value computed rather than printed is not labelled; a page with a value that is not on the page is excluded from training and evaluation. The words a value was read from, if they form one run of consecutive words, are tagged `B` for the first and `I` for the rest of the field's class; every other word is `O`. A value whose words are not one run, or a word claimed by two fields, is ignored by the loss (target −1) and not scored. - **Split.** Usable pages are ordered by `SHA-256(":")` (first 8 bytes, little endian), then id; the first `max(1, round(n · fraction))` (at most `n − 1`) are held out. - **Batches.** Page order per epoch is a SplitMix64 permutation seeded with `(seed << 32) xor epoch`; step `s` takes `batch` consecutive positions of the stream. Learning rate: linear warm-up to the peak over `warmup` steps, then linear decay to 10% of the peak over the remaining steps. - **Class weights.** Tag `O` weighs 1, every `B-`/`I-` tag the field weight (default 3), against the dominance of `O`. - **Resume.** The checkpoint holds the step, the trainable weights and the AdamW moments, tied by a SHA-256 fingerprint to the starting model, the schema, the settings and every training label file; a continued run ends with the model bytes of an uninterrupted one. - **Scores.** Per field and held-out page: correct when the model's field has exactly the label's words (no value on either side also counts as correct); recall counts labelled pages only; agreement counts pages where either side has a value. The suggested `LocalFirst` threshold is the lowest field confidence from which at least the stated share (default 0.99) of the tuned model's values were exactly right, with at least 5 values. # Model container, version 1 Status: current. Version 1 is version 0 (`model-container-v0.md`) with format version `1` and one added manifest member, `calibration`. Writers produce version 1; readers read both. A version 0 file loads and has no calibration: its calibrated word confidence is its raw confidence, and the reader reports a diagnostic saying so. Everything else in version 0 stands and is repeated here so this document is complete. The container is a cross-language contract: one weights file serves every implementation. Everything below is little-endian. ## File layout | Offset | Size | Content | |---|---|---| | 0 | 8 | Magic: ASCII `ONEOCRM` followed by one zero byte | | 8 | 4 | `uint32` format version, `1` (a reader also accepts `0`; any other value is an error) | | 12 | 4 | `uint32` manifest length *M* in bytes | | 16 | *M* | Manifest: UTF-8 JSON, no BOM | | 16 + *M* | pad | Zero bytes up to the next multiple of 64 — the start of the data region *D* | | *D* | … | Tensor data, each tensor starting at a multiple of 64 from *D*, zero padding between tensors | A reader rejects the file — before allocating anything sized by it — when the magic or version is wrong, the manifest is not valid UTF-8 JSON, or any length, offset, shape or count exceeds the reader's limits or the file's size. Tensors may not overlap. All padding bytes (after the manifest and between tensors) must be zero. The file ends exactly where the last tensor ends: no trailing padding, and bytes after the last tensor are an error. Tensors need not be listed in offset order. The manifest is strict JSON: no comments, no trailing commas, no duplicate member names. Members this version does not name are ignored (`provenance` among them). In a version 0 file `calibration` is such a member and is ignored, whatever it holds. Sizes, offsets and shapes are JSON integers. ## Manifest ```json { "architecture": "crnn-v0", "name": "line-recognizer", "version": "0.1.0", "license": "MIT OR Apache-2.0", "input": { "height": 32, "channels": 1, "mean": 0.5, "std": 0.5, "invert": false }, "charset": ["", " ", "!", "..."], "config": { }, "tensors": [ { "name": "conv.0.weight", "dtype": "f16", "shape": [64, 1, 3, 3], "offset": 0, "length": 1152 } ], "calibration": { "raw": "word-mean-max-run", "scripts": { "Latin": { "x": [0.0, 0.5, 1.0], "y": [0.0, 0.4, 0.95] }, "Greek": { "x": [0.0, 1.0], "y": [0.0, 1.0] } } }, "provenance": { } } ``` - `architecture` names the layer graph (below); a reader rejects any it doesn't know. - `input`: line crops are grayscale, scaled to `height` keeping aspect ratio; a pixel `p` in 0–255 becomes `x = p / 255` (`1 - p / 255` when `invert`), then `(x - mean) / std`, all in float32 with a correctly rounded division. `channels` is 1 for `crnn-v0`; `std` is positive. - `charset`: the output classes in order. Index 0 is the CTC blank; its text is a placeholder (the exporter writes ``) and is never output. Each other entry is a non-empty NFC string, usually one code point; entries, the blank placeholder included, are unique. (Readers do not check NFC; they check emptiness, uniqueness and length.) - `tensors`: `dtype` is `f16` (IEEE 754 binary16) in versions 0 and 1; values are widened to `float32` for computation. `offset` is relative to *D*; `length` equals the element count times 2. Tensors are stored row-major in PyTorch's own dimension order. - `provenance`: free-form record of the training run (commit, config, seed, data manifest hashes). Readers ignore it. ## Architecture `crnn-v0` A convolutional stack, then a bidirectional LSTM, then a per-timestep classifier, decoded by greedy CTC. `config`: ```json { "conv": [ { "out": 64, "kernel": [3, 3], "padding": [1, 1], "pool": [2, 2] }, { "out": 128, "kernel": [3, 3], "padding": [1, 1], "pool": null } ], "lstm": { "hidden": 256, "layers": 2 }, "classes": 700 } ``` - `config.conv[i]`: `out` channels, `kernel`, `padding` and `pool` (an array of two integers `[h, w]`, or `null`, or absent for no pooling). Kernel and pool sizes are at least 1, padding at least 0. `config.classes` equals the charset length. - **Conv block *i*** (`conv.`): 2-D convolution, stride 1, the given kernel `[h, w]` and zero padding `[h, w]`, with bias; batch normalization is folded into the weight and bias at export. Then ReLU. Then, when `pool` is `[ph, pw]`, max pooling with kernel and stride `[ph, pw]` and no padding (a trailing partial window is dropped; the block's input has `in` channels, the previous block's `out`, or `input.channels` for the first). A line image too narrow to leave at least one column after every block reads as empty text. Tensors: `conv..weight` `[out, in, kh, kw]`, `conv..bias` `[out]`. - **To sequence:** the stack's output height must be exactly 1 for the configured input height (a reader checks this). Shape `[C, 1, W]` becomes a sequence of `W` vectors of size `C`, left to right. - **BiLSTM** (`lstm`): `layers` stacked bidirectional layers; each layer's input is the previous layer's forward and reverse outputs concatenated (forward first). Gate order and equations are PyTorch's `nn.LSTM` (input, forget, cell, output; `c' = f*c + i*g`, `h' = o*tanh(c')`), zero initial state. Tensors use PyTorch's names under `lstm.`: `lstm.weight_ih_l`, `lstm.weight_hh_l`, `lstm.bias_ih_l`, `lstm.bias_hh_l`, and the same with suffix `_reverse`. - **Classifier** (`fc`): linear layer `2 * hidden → classes` with bias: `fc.weight` `[classes, 2 * hidden]`, `fc.bias` `[classes]`. `classes` equals the charset length. A reader ignores tensors it does not need. - **Decoding:** greedy CTC: take the arg-max class per timestep (lowest index on ties), merge repeats, drop blanks, concatenate charset entries. Confidences are defined under *Confidence* below. ## Architecture `dbnet-lite-v0` The page detector: text lines, layout regions and page orientation from one page image. The layer graph, pre- and post-processing are in [detector-v0.md](https://recursiveai.org/oneocr/docs/detector-v0/); this section is what the container carries. - `architecture`: `dbnet-lite-v0`; `name`: `text-detector-layout`. - `input`: `{"height": 0, "channels": 1, "mean": 0.5, "std": 0.5, "invert": false}`. **Height 0 means "any multiple of 32"** and is allowed for this architecture only; every other architecture needs a positive height. `channels` is 1. - `charset`: `["", ...layout class names]`, present only to satisfy the envelope's charset rules; it is not an output alphabet and a reader ignores it for this architecture. - `calibration`: the usual object with an empty `scripts` (the detector has no word confidences). - `config`: ```json { "arch": "dbnet-lite", "stem": 16, "widths": [32, 72, 144, 224], "depths": [2, 3, 4, 3], "fpn": 56, "text": 32, "layout": 112, "layout_blocks": 3, "classes": 8, "layout_classes": ["text", "title", "list", "table", "figure", "header_footer", "form_field", "checkbox"], "layout_trained": ["text", "title", "list", "table", "figure", "header_footer"], "postprocess": {"text_threshold": 0.3, "text_expand": 1.0, "layout_threshold": 0.6, "layout_expand": 0.85} } ``` `arch` must be `dbnet-lite`; `widths` and `depths` have four entries each; every width is at least 1 and within the reader's layer-width limit, every depth at least 1; `classes` equals the number of `layout_classes`, whose names are non-empty and unique. **`layout_trained`** is optional: the names of the layout classes the model was trained on. A reader never reports a class that is not listed. When the member is absent, every class except `form_field` and `checkbox` counts as trained (those two had no training labels in the first model). A name that is not in `layout_classes` is an error. **`postprocess`** is optional, and so is each of its members: `text_threshold` and `layout_threshold` are probabilities strictly between 0 and 1, compared as float32 logits; `text_expand` and `layout_expand` are factors in (0, 4] on the growth of a text line or layout core ([detector-v0.md](https://recursiveai.org/oneocr/docs/detector-v0/), "Post-processing"). Absent members take 0.3, 1.0, 0.6 and 0.85. A value out of range is an error. - Tensors (all `f16`, PyTorch layout, batch normalization folded in; names in [detector-v0.md](https://recursiveai.org/oneocr/docs/detector-v0/)): `stem`, `stage..dw`/`.pw`, `lat`, `smooth8.dw`/`.pw`, `smooth4.dw`/`.pw`, `layout..dw`/`.pw`, `layout_out`, `text4.dw`/`.pw`, `stem_proj`, `text2.dw`/`.pw`, `prob_out` (each with `.weight` and `.bias`), and `orient.weight` `[4, widths[3]]`, `orient.bias` `[4]`. A depthwise weight is `[c, 1, 3, 3]`, a 1x1 weight `[out, in, 1, 1]`. Shapes are checked against `config`. ## Architecture `dbnet-lite-v1` The page detector with the optional stride-16 layout branch and two-scale orientation head ([detector-v1.md](https://recursiveai.org/oneocr/docs/detector/)). The envelope, `input`, `charset`, `calibration`, `layout_trained`, `postprocess` and every tensor of `dbnet-lite-v0` are as in the section above, with `architecture` `dbnet-lite-v1`. `config` gains three members, all optional: - `layout16_blocks`: integer, 0 (the default) or the number of stride-16 blocks; - `layout16`: the width of those blocks, required and positive when `layout16_blocks` is above 0; - `orient16`: boolean, false by default; when true `orient.weight` is `[4, widths[2] + widths[3]]`. With the branch the file also carries `layout16..dw`/`.pw` (`j < layout16_blocks`) and `layout_mix` (`.weight` `[layout, layout16, 1, 1]`, `.bias`). The three members in a `dbnet-lite-v0` file are an error. Height 0 in `input` is allowed for both detector architectures. ## Architecture `docextract-v0` The document-extraction model: a small layout-aware transformer over the words of a page. It reads words (text and box), not pixels. The graph, the featurization and the assembly of labels into fields are in [docextract-v0.md](https://recursiveai.org/oneocr/docs/docextract/); this section is what the container carries. - `architecture`: `docextract-v0`; `name`: `document-extraction`. - `input` has no image members. It is `{"kind": "words", "max_words": 512, "max_chars": 32, "bins": 128, "features": 8}`: the most words in one window (`MaxWordsPerWindow` bounds it), the characters kept per word (`MaxCharsPerWord`), the cells per axis of the position grid (`MaxPositionBins`) and the numeric features per word (always 8). Every other architecture needs the image members (`height`, `channels`, `mean`, `std`, `invert`) and no `kind`. - `charset`: the alphabet of the character embedding, charset-v1 (entry 0 `` is padding and never a text character). The embedding table has one more row than the charset: the last row is the unknown character. - `calibration`: `{"raw": "word-mean-max-run", "scripts": {}}`. The model has no calibrated confidences; field confidence is the mean softmax probability of the chosen tags. - `config`, all integers unless stated: `d`, `layers`, `heads`, `ffn`, `char_dim`, `char_conv`, `pair_hidden`, `rel_dim`, `max_words` (equal to `input.max_words`), `classes`, `tags` (`1 + 2 * classes`), `roles` (5), `doc_types`, `unk_char` (equal to the charset length); and the string lists `classes_list` (`classes` unique, non-empty names), `roles_list` (`O, B-KEY, I-KEY, B-VAL, I-VAL`) and `doc_types_list`. `d` is a multiple of `heads`. Widths are bounded by `MaxLayerWidth`, `layers` by `MaxTransformerLayers`, the three list lengths by `MaxExtractionClasses`, and `(layers * heads + 2) * max_words^2` by `MaxActivationElements`, all before any tensor is read. - Tensors (names and shapes exactly; a reader checks each against the configuration before reading it): `char_emb.weight [charset + 1, char_dim]`, `char_conv.weight [char_conv, char_dim, 3]`, `char_conv.bias`, `char_proj.{weight [d, char_conv], bias}`, `pos_x0 / pos_y0 / pos_x1 / pos_y1 .weight [bins, d]`, `feat_proj.{weight [d, 8], bias}`, `emb_ln.{weight, bias} [d]`, per block `blocks..{ln1, ln2}.{weight, bias} [d]`, `blocks..qkv.{weight [3d, d], bias}`, `blocks..proj.{weight [d, d], bias}`, `blocks..fc1.{weight [ffn, d], bias}`, `blocks..fc2.{weight [d, ffn], bias}`, `pair1.{weight [pair_hidden, 13], bias}`, `pair2.{weight [layers * heads + 2, pair_hidden], bias}`, `final_ln.{weight, bias}`, `tag_head.{weight [tags, d], bias}`, `role_head.{weight [5, d], bias}`, `doc_head.{weight [doc_types, d], bias}`, `rel_q.{weight [2 * rel_dim, d], bias}`, `rel_k.{weight [2 * rel_dim, d], bias}`. - A model trained for another schema (a fine-tuned head) is the same architecture with its own `classes_list` and `tag_head`. ## Calibration (version 1) A recognizer says how sure it is per character; users want the probability that a **word** is exactly right. The model file carries the map from one to the other, fitted by the trainer on a held-out calibration split. **Character probability.** The network's output at a step is softmax over the classes. A character is a *run*: a maximal sequence of consecutive steps whose arg-max is the same non-blank class (a blank step ends a run, so "ll" is two runs). Its probability is the largest softmax probability of that class over the run's steps. **Raw word confidence** (`"raw": "word-mean-max-run"`, the only value defined; any other is an error): the words of a line are the maximal sequences of runs whose characters are not whitespace. A character is whitespace when its charset entry is non-empty and consists only of whitespace code units (`char.IsWhiteSpace`-style: U+0020, U+00A0, U+2003 and the other Unicode White_Space characters). Spaces are not part of any word; a line with no non-space run has no words. The raw confidence of a word is the **mean** of its runs' probabilities, summed left to right in float32 and divided by the number of runs. A whole line's raw confidence (`RecognizedLine.Confidence`) uses the same rule over every run of the line, spaces included; it is 0 for an empty line. **Script of a word.** Count the word's letters (Unicode categories L*, by code point) in three groups: Greek (U+0370–03FF, U+1F00–1FFF), Cyrillic (U+0400–052F, U+1C80–1C8F, U+2DE0–2DFF, U+A640–A69F), Latin (every other letter). The script with the most letters wins; ties go to the first of Latin, Greek, Cyrillic. A word with no letters (digits, punctuation, symbols) is Latin. **Calibrated word confidence.** The manifest member: ```json "calibration": { "raw": "word-mean-max-run", "scripts": { "Latin": { "x": [...], "y": [...] }, "Greek": {...}, "Cyrillic": {...} } } ``` * `scripts` is an object whose names are exactly `Latin`, `Greek` or `Cyrillic` (case-sensitive, no duplicates). Any other name is an error. A script may be missing; its calibrated confidence is then its raw confidence. `scripts` may be empty. * Each script has `x` and `y`, JSON arrays of numbers of equal length, between 1 and 64 knots (more is a limit error, `MaxCalibrationKnots`). * `x` is strictly increasing and every value is in [0, 1]; `y` is non-decreasing and every value is in [0, 1]. Numbers are IEEE doubles. * A version 1 file must have the `calibration` object; the reader rejects a file without it. * The map is piecewise linear through the knots `(x[i], y[i])` and constant outside them: for `r <= x[0]` the value is `y[0]`, for `r >= x[last]` it is `y[last]`. For `x[i] < r < x[i+1]`, in IEEE double precision with each operation rounded on its own (no fused multiply-add): `t = (r - x[i]) / (x[i+1] - x[i])`, `p = (y[i+1] - y[i]) * t`, `value = y[i] + p` and the result is converted to float32. A one-knot curve is the constant `y[0]`. `r` is the float32 raw confidence converted to double. **Reporting curve (optional).** A version 1 recognizer file may also carry `report_calibration`, an object of exactly the same shape and rules as `calibration` (same `raw` definition, scripts, knot limits and evaluation). Where it is present, the confidence a finished page reports for a word is `report_calibration` applied to the word's raw confidence (script taken from the word's final text); the word's `rawConfidence` is unchanged. Every decision the reader makes while building the page (noise and speck filtering, word joining and splitting, the page-angle search, table tests, reading order) keeps using `calibration`, so the text, boxes and structure of the page are the same with or without the member. The map is applied after the last of those decisions and nothing reads its result. Line and block confidences are recomputed from the reported word confidences by the rule below. A file without the member reports `calibration` as before; a version 0 file ignores the member like any member that version does not name. A malformed member is an error exactly as a malformed `calibration` is, and JSON `null` means absent. The reason for a second curve: a curve fitted to make the reported number predict correctness on a kind of page (receipt photos, say) differs from the one the filters were tuned against, and swapping it would change the text. **Line confidence** in the result type is the calibrated confidences of the line's words averaged with each word weighted by its number of runs (double precision, converted to float32); 0 for a line with no words. This is a reader convention for reporting, not part of the file. ## Numerics Weights are stored in fp16 and computed in fp32. Implementations agree with the PyTorch reference within a documented per-layer tolerance; bit-exact agreement is only required between the C# library's own code paths. The order of floating-point operations that makes those paths bit-identical is written down in [numerics.md](https://recursiveai.org/oneocr/docs/numerics/). # Model container, version 0 Status: **superseded for new files** by [version 1](https://recursiveai.org/oneocr/docs/model-container/), which adds calibration. Readers still load version 0 files; they carry no calibration. The container is a cross-language contract: one weights file serves every implementation. Everything below is little-endian. ## File layout | Offset | Size | Content | |---|---|---| | 0 | 8 | Magic: ASCII `ONEOCRM` followed by one zero byte | | 8 | 4 | `uint32` format version, `0` | | 12 | 4 | `uint32` manifest length *M* in bytes | | 16 | *M* | Manifest: UTF-8 JSON, no BOM | | 16 + *M* | pad | Zero bytes up to the next multiple of 64 — the start of the data region *D* | | *D* | … | Tensor data, each tensor starting at a multiple of 64 from *D*, zero padding between tensors | A reader rejects the file — before allocating anything sized by it — when the magic or version is wrong, the manifest is not valid UTF-8 JSON, or any length, offset, shape or count exceeds the reader's limits or the file's size. Tensors may not overlap. All padding bytes (after the manifest and between tensors) must be zero. The file ends exactly where the last tensor ends: no trailing padding, and bytes after the last tensor are an error. Tensors need not be listed in offset order. The manifest is strict JSON: no comments, no trailing commas, no duplicate member names. Members this version does not name are ignored (`provenance` among them). Sizes, offsets and shapes are JSON integers. ## Manifest ```json { "architecture": "crnn-v0", "name": "line-recognizer", "version": "0.1.0", "license": "MIT OR Apache-2.0", "input": { "height": 32, "channels": 1, "mean": 0.5, "std": 0.5, "invert": false }, "charset": ["", " ", "!", "..."], "config": { }, "tensors": [ { "name": "conv.0.weight", "dtype": "f16", "shape": [64, 1, 3, 3], "offset": 0, "length": 1152 } ], "provenance": { } } ``` - `architecture` names the layer graph (below); a reader rejects any it doesn't know. - `input`: line crops are grayscale, scaled to `height` keeping aspect ratio; a pixel `p` in 0–255 becomes `x = p / 255` (`1 - p / 255` when `invert`), then `(x - mean) / std`, all in float32 with a correctly rounded division. `channels` is 1 for `crnn-v0`; `std` is positive. - `charset`: the output classes in order. Index 0 is the CTC blank; its text is a placeholder (the exporter writes ``) and is never output. Each other entry is a non-empty NFC string, usually one code point; entries, the blank placeholder included, are unique. (Readers do not check NFC; they check emptiness, uniqueness and length.) - `tensors`: `dtype` is `f16` (IEEE 754 binary16) in version 0; values are widened to `float32` for computation. `offset` is relative to *D*; `length` equals the element count times 2. Tensors are stored row-major in PyTorch's own dimension order. - `provenance`: free-form record of the training run (commit, config, seed, data manifest hashes). Readers ignore it. ## Architecture `crnn-v0` A convolutional stack, then a bidirectional LSTM, then a per-timestep classifier, decoded by greedy CTC. `config`: ```json { "conv": [ { "out": 64, "kernel": [3, 3], "padding": [1, 1], "pool": [2, 2] }, { "out": 128, "kernel": [3, 3], "padding": [1, 1], "pool": null } ], "lstm": { "hidden": 256, "layers": 2 }, "classes": 700 } ``` - `config.conv[i]`: `out` channels, `kernel`, `padding` and `pool` (an array of two integers `[h, w]`, or `null`, or absent for no pooling). Kernel and pool sizes are at least 1, padding at least 0. `config.classes` equals the charset length. - **Conv block *i*** (`conv.`): 2-D convolution, stride 1, the given kernel `[h, w]` and zero padding `[h, w]`, with bias; batch normalization is folded into the weight and bias at export. Then ReLU. Then, when `pool` is `[ph, pw]`, max pooling with kernel and stride `[ph, pw]` and no padding (a trailing partial window is dropped; the block's input has `in` channels, the previous block's `out`, or `input.channels` for the first). A line image too narrow to leave at least one column after every block reads as empty text. Tensors: `conv..weight` `[out, in, kh, kw]`, `conv..bias` `[out]`. - **To sequence:** the stack's output height must be exactly 1 for the configured input height (a reader checks this). Shape `[C, 1, W]` becomes a sequence of `W` vectors of size `C`, left to right. - **BiLSTM** (`lstm`): `layers` stacked bidirectional layers; each layer's input is the previous layer's forward and reverse outputs concatenated (forward first). Gate order and equations are PyTorch's `nn.LSTM` (input, forget, cell, output; `c' = f*c + i*g`, `h' = o*tanh(c')`), zero initial state. Tensors use PyTorch's names under `lstm.`: `lstm.weight_ih_l`, `lstm.weight_hh_l`, `lstm.bias_ih_l`, `lstm.bias_hh_l`, and the same with suffix `_reverse`. - **Classifier** (`fc`): linear layer `2 * hidden → classes` with bias: `fc.weight` `[classes, 2 * hidden]`, `fc.bias` `[classes]`. `classes` equals the charset length. A reader ignores tensors it does not need. - **Decoding:** greedy CTC: take the arg-max class per timestep (lowest index on ties), merge repeats, drop blanks, concatenate charset entries. The confidence of a line is computed from the per-timestep softmax probabilities; its exact definition is the implementation's until version 1. The C# reader's: for each emitted character (a run of equal non-blank arg-max steps), the largest softmax probability of its class over the run; the line confidence is the mean of those, and 0 for an empty line. ## Numerics Weights are stored in fp16 and computed in fp32. Implementations agree with the PyTorch reference within a documented per-layer tolerance; bit-exact agreement is only required between the C# library's own code paths. The order of floating-point operations that makes those paths bit-identical is written down in [numerics.md](https://recursiveai.org/oneocr/docs/numerics/). # Page detector `dbnet-lite-v0` (`dbnet-lite-v1` adds an optional stride-16 layout branch and a two-scale orientation head: [detector-v1.md](https://recursiveai.org/oneocr/docs/detector/).) One page image in; text lines, layout regions and the page's orientation out. The container section is in [model-container-v1.md](https://recursiveai.org/oneocr/docs/model-container/), the operation order in [numerics.md](https://recursiveai.org/oneocr/docs/numerics/). The text branch is the probability-map branch of Liao et al., "Real-time Scene Text Detection with Differentiable Binarization" (AAAI 2020): the threshold map and the differentiable binarization exist only for training and are not in the file. The layout head applies the same shrink-and-recover idea to region boxes. ## Outputs For a padded input of height *H* and width *W* (multiples of 32): | output | shape | meaning | |---|---|---| | `prob` | [1, H/2, W/2] | text-line core logit | | `layout` | [classes, H/8, W/8] | per-class region-core logits | | `orient` | [4] | logits for the page content being rotated 0, 90, 180, 270 degrees counter-clockwise from upright | Layout classes, in channel order, are the manifest's `layout_classes` (`text, title, list, table, figure, header_footer, form_field, checkbox` in the first model). Detection is line level: one box per text line segment, which is what the recognizer reads. ## Pre-processing 1. Convert to 8-bit gray. 2. Scale so the long side is 1024 pixels: `s = 1024 / max(w, h)`, each side `max(1, round_half_even(side * s))`, resampled with the imaging library's automatic filter in gamma-encoded values (no linear-light conversion; the model was trained on gamma-space bilinear resampling). Pages smaller than 1024 are enlarged the same way; a page already 1024 on the long side is not resampled. 3. Pad right and bottom with white (255) to a multiple of 32 in both directions. 4. Each pixel `p` becomes `(p / 255 - mean) / std` in fp32 with the manifest's `input.mean` and `input.std` (0.5 and 0.5, so `p / 127.5 - 1`); `invert` flips `p / 255` first. An A4 page gives 1024 x 736 and about 1.86 GMACs. The network is fully convolutional and runs on any multiple-of-32 size up to the reader's `MaxDetectorSide` limit (default 4096). ## Layer graph `conv(k, s, g)` is a *k* x *k* convolution with stride *s*, *g* groups, zero padding `k / 2`, bias, batch norm folded in. ReLU follows unless noted. `DW(cin, cout, s)` is `dw` = `conv(3, s, g = cin)` then `pw` = `conv(1, 1, 1)`, `cin -> cout`, each with ReLU. Tensor names are `.weight`, `.bias`; weights are `[cout, cin / g, kh, kw]`. | step | operation | tensors | |---|---|---| | `stem` | `conv(3, 2)`, 1 to `stem` channels | `stem` | | `stage.` (i = 0..3, j < depths[i]) | `DW(cin, widths[i], 2 if j = 0 else 1)`; stage outputs are at strides 4, 8, 16, 32 | `stage..dw`, `.pw` | | `lat` | `conv(1, 1)`, `widths[i]` to `fpn`, no ReLU, on the stage-*i* output | `lat` | | top-down | `p16 = lat2 + up2(lat3)`; `p8 = DW(fpn, fpn)(lat1 + up2(p16))`; `p4 = DW(fpn, fpn)(lat0 + up2(p8))`. `up2` repeats each value in a 2 x 2 block; `p16` is not smoothed | `smooth8`, `smooth4` | | layout head | `layout_blocks` blocks `DW(·, layout)` on `p8`, then `conv(1, 1)` to `classes`, no ReLU | `layout.`, `layout_out` | | text head | `t4 = DW(fpn, text)(p4)`; `t2 = DW(text, text)(up2(t4) + conv(1, 1)(stem output; stem to text, no ReLU))`; `prob = conv(1, 1)` `text` to 1, no ReLU | `text4`, `stem_proj`, `text2`, `prob_out` | | `orient` | global average of the stage-3 output, then linear `widths[3]` to 4 | `orient.weight`, `orient.bias` | The reference for this table is a float64 evaluation on plain loops (`tests/OneOcr.Tests/TestDetectorModels.cs`) and PyTorch fixtures from `training/parity/dbnet_fixtures.py`. ## Post-processing Both heads use one rule: a cell is on when its logit exceeds a threshold (compared on logits, in fp32: `ln(0.3 / 0.7) = -0.847297860387203` for text, `ln(0.6 / 0.4) = 0.405465108108164` for layout; a threshold `p` the manifest gives is converted as `float32(ln(p / (1 - p)))`); 4-connected components of on cells are labelled in the order of their first cell in row-major order; each component's bounding cell range is the shrunk core, which is grown back. A component's score is the mean sigmoid of its cells' logits (informational). **Text lines** (`prob`, stride 2): components of fewer than 2 cells are dropped. The core in network pixels is `x0 = 2 * minCol`, `x1 = 2 * (maxCol + 1)`, likewise `y`. With `c = min(x1 - x0, y1 - y0)` the box grows by `d = 0.3 / (1 - 2 * 0.3) * c * expand = 0.75 c * expand` on each side (the training shrank each line by 0.3 of its short side per side). **Layout regions** (`layout`, stride 8), for every class the manifest calls trained and no other: components of any size. Core `x0 = 8 * minCol` etc., `core = min(x1 - x0, y1 - y0)`; grown by `core / 3` when `core <= 60`, else by 20, times the layout expansion (default 0.85). Then, for every class except `table` and `figure`: the box becomes the union of the text lines (network pixels, before scaling back) whose centres lie inside it, each side limited to 8 pixels beyond the region's own box; with no line inside, or an empty union, the box stays. There is no non-maximum suppression: regions of different classes may overlap. Boxes are divided by `s` to page pixels and clipped to the page; a box that is empty after clipping is dropped. (The training-side reference does not clip.) More than `MaxLinesPerPage` lines or `MaxPageElements` regions is a limit error. **Orientation:** softmax over `orient` with the library's exp; the winner (lowest index on a tie) is the number of counter-clockwise quarter turns the content is rotated by. A page whose winner probability is below 0.9 is read as it is. Otherwise it is turned clockwise by that many quarter turns and detected again on the upright copy; the page model then describes the upright page, and a diagnostic says so. In the engine a turn the head asks for is confirmed by the classical orientation decision (sample lines read both ways up; see page-orientation-v1.md) and that decision is the one acted on: the head has only seen flat pages, and on tilted phone photos it called upright pages upside down with probability 0.99. **Values.** Defaults (used when the manifest has no `config.postprocess`): text threshold 0.3, text expansion 1.0, layout threshold 0.6, layout expansion 0.85. A model overrides any of them with `config.postprocess` (`text_threshold`, `text_expand`, `layout_threshold`, `layout_expand`; see the container spec). ## Use in the page pipeline When an engine has a detector, `Ocr` uses it instead of the built-in classical line finding: - Each detected line box is narrowed to the ink inside it (the page's Otsu threshold, as the classical finder uses), so the recognizer's crop rules, written for ink-tight lines, apply unchanged; with no ink inside, the detected box is used. - Lines inside a `text`, `title`, `list` or `header_footer` region (the smallest one holding the line's centre) form one reading-order leaf; other lines are leaves of their own, so the cells of a table row on one baseline join into a row. - Regions fill `OcrPage.Regions` with the role of their class (`header_footer` is a header in the upper half of the page and a footer in the lower half), the score as confidence and the blocks whose centres lie inside. Blocks inside a `header_footer` region get the block role Header or Footer. - Without a detector, page output is exactly what it was before. ## Evaluation order and memory Every output value of the network is an independent sum, so a port may evaluate the layers whole or in bands of rows with any band height and any number of threads and get the same bits. The C# implementation keeps its working set small by running the stem inside the first stage, the stride-4 merge with the text block, and the stride-2 text head in row bands, each band recomputing the few input rows its neighbours share (the stem rows and the stage-0 lateral rows); no stride-2 map and no stride-4 lateral exists whole. With the shipped size, an A4 page needs about 23 MB of scratch at degree 1 (the three stride-2 maps and the stride-4 lateral and smoothed maps of a whole-map evaluation take about 100 MB). Tests hold the banded and whole-map results bit-equal for band heights from one row up. # Changelog All notable changes are listed here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and versions follow [Semantic Versioning](https://semver.org/). ## [1.0.0] - 2026-10-31 First release. ### Added - **Local OCR** (`RecursiveAI.OneOcr`): a pure-managed reader for scans, born-digital renders, screenshots and phone photos of flat documents, with no dependency beyond the base class library and `RecursiveAI.OneImage`. Latin (including the EU languages, Turkish and Vietnamese), Greek and Cyrillic in one recognizer. Page orientation (0/90/180/270) and deskew are on by default. - **Deterministic output.** Text, boxes and confidences are bit-identical on AVX-512, AVX2, 128-bit vectors (SSE and NEON) and the scalar path, and at every degree of parallelism. The numeric rules are specified in `docs/spec/numerics.md`. - **The page model** (`OcrPage`, blocks, lines, words, characters, tables, key-value pairs, a `Source` and a calibrated `Confidence` on every element) and its versioned `oneocr.page` JSON, which shares member names with OnePDF's `onepdf.extraction`. - **Calibrated confidence.** A word's confidence is the probability that the word is exactly right, calibrated per script on held-out data. - **Local Markdown** from the page model: headings, lists, pipe tables, with headers and footers dropped. - **Document extraction**: `ExtractInvoice()`, `ExtractReceipt()` and `Extract()` over a small local model that labels words on the page, so a value always carries the words and boxes it was read from. Invoice fields carry their EN 16931 business-term ids. The model ships in `RecursiveAI.OneOcr.Models.Documents`. - **LLM vision** (`RecursiveAI.OneOcr.Llm`): Anthropic, OpenAI, OpenAI-compatible servers, Gemini and Azure OpenAI over a plain `HttpClient`. Every returned value is checked against the words on the page and comes back `Grounded`, `GroundedNormalized`, `Derived` or `Ungrounded`. Cost estimates and caps, routing between local models and the LLM, an `Offline` switch, a redaction hook and no-retention flags. Nothing leaves the machine unless you configure a provider. - **Fine-tuning** (`RecursiveAI.OneOcr.Training`): recognition and extraction fine-tuning on your own documents, on the CPU, deterministic and resumable, with a review queue for lines the reader and the LLM disagree on. - **PDF pages** (`RecursiveAI.OneOcr.Pdf`): text-layer words where a page has them, local OCR for scanned regions and image-only pages, one page model. - **The `oneocr` command-line tool** (`RecursiveAI.OneOcr.Tool`): `ocr`, `train`, `eval` and `review`. - **The model container**, a versioned binary format for weights, specified in `docs/spec/model-container-v1.md`; the bundled recognizer, page detector and language models are embedded in `RecursiveAI.OneOcr.Models` and checked against their SHA-256 pins on every load. - Parsers for model files, images, page JSON, PDFs, review files and LLM responses are bounds-checked, run under limits checked before anything large is allocated, and have fuzz targets. NativeAOT and trimming are supported. ### Known limits - Not supported: CJK, Arabic and Indic scripts, handwriting, curved-page dewarping and scene text. - The README's benchmark tables list every category where a competing engine reads better than OneOcr. - A NativeAOT build for x64 needs an instruction-set level of at least `x86-64-v3`; the package sets it by default. Below it the reader runs its exact scalar path, about a hundred times slower, with the same text. # Invoices as fields Read an invoice image into typed fields with their EN 16931 business-term ids, line items included, and see which words and box each value came from. Extraction needs the extraction model, which ships in its own package: ```shell dotnet add package RecursiveAI.OneOcr --version 1.0.0 dotnet add package RecursiveAI.OneOcr.Models.Documents --version 1.0.0 ``` A small layout-aware model tags the words of the page, and deterministic code assembles the fields from the tagged words, so a value is always text that is on the page. The image is `invoice.png` from [the sample images](https://recursiveai.org/oneocr/docs/getting-started/#the-sample-images): ```csharp using OneImage; using OneOcr; using OneOcr.Documents; using OneOcr.Models.Documents; OcrEngine engine = OcrEngine.LoadBundled(); DocumentModels.Configure(); // the extraction model from RecursiveAI.OneOcr.Models.Documents, once Invoice invoice = Image.Load("invoice.png").ExtractInvoice(engine); Console.WriteLine($"{invoice.Number?.Value}, issued {invoice.IssueDate?.Value}, total {invoice.Total?.Value}"); Console.WriteLine($"{invoice.SellerName?.Value} -> {invoice.BuyerName?.Value}, IBAN {invoice.Iban?.Value}"); foreach (InvoiceLine line in invoice.Lines) Console.WriteLine($" {line.Description?.Text}: {line.Quantity?.Value} {line.Unit?.Text} x {line.UnitPrice?.Value} = {line.Amount?.Value}"); ExtractedField total = invoice.Extraction.Get("BT-112")!; Console.WriteLine($"BT-112 \"{total.Text}\" -> {total.Number}, words [{string.Join(", ", total.WordIndices)}], " + $"box ({total.Box.Left:F0}, {total.Box.Top:F0}), confidence {total.Confidence:F2}"); Console.WriteLine($"Due date: {invoice.DueDate?.Value ?? "null"}, delivery date: {invoice.DeliveryDate?.Value ?? "null"}"); ``` ```text INV-2041, issued 2026-10-14, total 215.78 Northwind Supply Ltd -> Blue Heron Bakery, IBAN GB33 BUKB 2020 1555 5555 55 Flour, 25 kg bag: 4 pcs x 31.50 = 126 Cane sugar, 10 kg: 2 pcs x 18.25 = 36.50 Baking parchment roll: 6 pcs x 7.40 = 44.40 BT-112 "£215.78" -> 215.78, words [83], box (2072, 1474), confidence 1.00 Due date: null, delivery date: 2026-11-13 ``` ## What came back - Each `ExtractedField` has the exact `Text`, a normalized `Value` (an ISO date, a plain decimal amount), typed `Number` or `Date` where it applies, the indices of its words in the page and their box. - `invoice.Extraction.Get("BT-112")` looks any field up by its business term: `BT-1` number, `BT-2` issue date, `BT-27` seller, `BT-44` buyer, `BT-84` IBAN, `BT-106` subtotal, `BT-110` tax, `BT-112` total, and line items from `BT-126` to `BT-153`. - A field the model didn’t find is `null`, never a guess. Here the model tagged the second date as the delivery date (`BT-72`) rather than the due date (`BT-9`), so `DueDate` is `null`: read the fields you depend on, and look at `Confidence`, which is the mean probability of the field’s word tags (not a calibrated probability like a word’s). - Amounts are normalized to plain decimals: `126.00` came back as `126`. ## Good to know - `ExtractInvoice` and `ExtractReceipt` take a `DocumentExtractionOptions` (`MaxDegreeOfParallelism`, `Strict`) and have `Async` twins. - For your own document type, `Extract` takes a source-generated `JsonTypeInfo` and a model you fine-tuned; the [extraction model spec](https://recursiveai.org/oneocr/docs/docextract/) describes the model and the [overview](https://recursiveai.org/oneocr/docs/overview/#invoices-receipts-and-your-own-document-types) the API. - Receipts work the same way: see [Receipts](https://recursiveai.org/oneocr/cookbook/receipts/). # Receipts Read a till receipt into merchant, date, time, totals, payment method and items, each with the words it was read from. Install `RecursiveAI.OneOcr.Models.Documents` next to `RecursiveAI.OneOcr`, as for [invoices](https://recursiveai.org/oneocr/cookbook/invoices/). The image is `receipt.png` from [the sample images](https://recursiveai.org/oneocr/docs/getting-started/#the-sample-images). This program lists every field and item the model found: ```csharp using OneImage; using OneOcr; using OneOcr.Documents; using OneOcr.Models.Documents; OcrEngine engine = OcrEngine.LoadBundled(); DocumentModels.Configure(); Receipt receipt = Image.Load("receipt.png").ExtractReceipt(engine); foreach (ExtractedField field in receipt.Extraction.Fields) Console.WriteLine($"{field.Id,-24} {field.Text,-16} -> {field.Value,-12} {field.Confidence:F2}"); foreach (ExtractedGroup item in receipt.Extraction.LineItems) Console.WriteLine("item: " + string.Join(", ", item.Fields.Select(f => $"{f.Id["receipt.".Length..]}={f.Text}"))); ``` ```text receipt.merchant Corner Market -> Corner Market 0.95 receipt.date 24/10/2026 -> 2026-10-24 1.00 receipt.time 14:32 -> 14:32 1.00 receipt.subtotal 27.22 -> 27.22 1.00 receipt.tax 2.07 -> 2.07 1.00 receipt.total $ 29.29 -> 29.29 0.81 receipt.paymentMethod Card **** 4821 -> Card **** 4821 1.00 item: itemName=Coffee beans, amount=12.99 item: quantity=2x, itemName=Oat milk, amount=8.98 item: itemName=Bagels, amount=5.25 ``` `Receipt` also has typed shortcuts: `receipt.Merchant`, `Date`, `Time`, `Subtotal`, `Tax`, `Tip`, `Total` and `PaymentMethod`, and `receipt.Items`. The fields are the same `ExtractedField` records as an invoice’s, with `receipt.` ids instead of business terms. `Value` is normalized (`2026-10-24` from `24/10/2026`, `29.29` from `$ 29.29`) and `Text` is what is printed. The total’s confidence is lower than the rest here (0.81): it is the mean probability of the tags on its words, and a figure to threshold on per field, not a calibrated probability. # Save and load the page as JSON Write a page to `oneocr.page` JSON, the documented and versioned format, and read it back under limits. ```csharp using OneImage; using OneOcr; OcrEngine engine = OcrEngine.LoadBundled(); OcrPage page = Image.Load("receipt.png").Ocr(engine, new OcrOptions { Dpi = 192 }); var options = new OcrPageJsonOptions { Indented = true, IncludeChars = false }; string json = OcrPageJson.ToJson([page], options); File.WriteAllText("receipt.page.json", json); IReadOnlyList again = OcrPageJson.Parse(File.ReadAllText("receipt.page.json")); Console.WriteLine($"{json.Length} characters; read back: {again.Count} page, {again[0].Words.Count} words"); Console.WriteLine($"equal records: {again[0].Words.SequenceEqual(page.Words)}; same JSON: {OcrPageJson.ToJson(again, options) == json}"); string[] lines = json.Split('\n'); Console.WriteLine(string.Join('\n', lines.Take(9))); int w = Array.FindIndex(lines, l => l.Contains("\"words\"")); Console.WriteLine(string.Join('\n', lines.Skip(w).Take(12))); ``` ```text 27949 characters; read back: 1 page, 40 words equal records: False; same JSON: True { "format": "oneocr.page", "version": 1, "pageCount": 1, "pages": [ { "index": 0, "width": 920, "height": 940, "words": [ { "text": "Corner", "bounds": [314,69,445,100 ], "visible": true, "source": "ocr", "confidence": 0.9839, "rawConfidence": 0.9999 }, { "text": "Market", ``` - The JSON is deterministic: the same pages give the same bytes, and writing a page read back from JSON gives the same JSON again. - It is not a lossless copy of the objects in memory. Coordinates are rounded to three decimals and confidences to four, which is why the records read back don’t compare equal to the originals. Store the JSON, not the in-memory page, when you need to compare later. - `bounds` are `[left, top, right, bottom]` with y running down; every element has a `source` (`ocr`, `textLayer` or `llm`). The member names match OnePDF’s `onepdf.extraction`. The whole format is in [oneocr.page JSON](https://recursiveai.org/oneocr/docs/page-json/). - `OcrPageJson.Parse` checks sizes and depth against `OcrLimits` and throws `OcrFormatException` on bad input, so it is safe on files you didn’t write. - `IncludeChars = false` leaves out the per-character boxes, which make up most of the file. # Words to check Use calibrated confidence to find the words a person should look at, on a grainy, tilted copy of the receipt. A word’s `Confidence` is the calibrated probability that the word is exactly right, fitted per script on held-out data, so one threshold means the same thing on every page. `receipt-photo.jpg` is the sample receipt turned 3°, scaled down, blurred, given grain and saved as a low-quality JPEG: ```csharp using OneImage; using OneOcr; OcrEngine engine = OcrEngine.LoadBundled(); OcrPage page = Image.Load("receipt-photo.jpg").Ocr(engine); Console.WriteLine($"straightened {page.SkewDegrees:F1}°, {page.Words.Count} words"); foreach (OcrLine line in page.Blocks.SelectMany(b => b.Lines)) Console.WriteLine($"{line.Words.Min(w => w.Confidence),5:P0} {line.Text}"); IEnumerable check = page.Words.Where(w => w.Confidence < 0.9f); Console.WriteLine($"Below 90%: {string.Join(" ", check.Select(w => $"\"{w.Text}\" {w.Confidence:P0}"))}"); ``` ```text straightened -3.0°, 48 words 4% 1 3% e 3% 3 99% Corner Market 95% 1402 Oak Street 99% 78704 Austin 95% +1 512 555 0142 98% Date: 24/10/2026 13% : Time: 14:32 98% Coffee beans 96% 2x Oat milk 100% Bagels 3% 15 98% 12.99 98% 8.98 99% 5.25 10% g Subtotal 27.22 7% U2 Tax 8.25 % 2.07 98% TOTAL $ 29.29 65% d**** 4821 98% Paid by Card 100% Thank you! 3% 9N Below 90%: "1" 4% "e" 3% "3" 3% ":" 13% "15" 3% "g" 10% "U2" 7% "d****" 65% "9N" 3% ``` - The reader straightened the 3° tilt, and every word of the receipt’s own text scores 90% or more with one exception: the payment line came back as two lines, and `****` was misread as `d****`, at 65%. - The grain was read as a few short “words” (`1`, `e`, `15`, `9N` and others). Each scores 13% or less, so the threshold separates them from the receipt’s text. - `RawConfidence` is the uncalibrated mean of the characters’ best probabilities; use `Confidence` for thresholds. Lines and blocks average their words’. - Coming from Tesseract? Its `conf` is a 0–100 score; divide an existing threshold by 100 as a starting point, then look at the words just below it ([Moving from Tesseract](https://recursiveai.org/oneocr/docs/migrating-from-tesseract/#confidence)). # Same bytes on every machine Check that the text, boxes and confidences don’t depend on the thread count or the CPU’s vector instructions. The program reads the invoice at four degrees of parallelism and hashes the page JSON each time: ```csharp using System.Security.Cryptography; using System.Text; using OneImage; using OneOcr; OcrEngine engine = OcrEngine.LoadBundled(); Image image = Image.Load("invoice.png"); foreach (int threads in new[] { 1, 2, 4, 16 }) { OcrPage page = image.Ocr(engine, new OcrOptions { MaxDegreeOfParallelism = threads }); string json = OcrPageJson.ToJson([page]); string hash = Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(json)))[..16]; Console.WriteLine($"{engine.KernelPath,-9} threads={threads,-2} {page.Words.Count} words sha256={hash}"); } if (engine.PerformanceNotice is { } notice) Console.WriteLine(notice); ``` ```text Avx512 threads=1 98 words sha256=DC165D8B4ADFF440 Avx512 threads=2 98 words sha256=DC165D8B4ADFF440 Avx512 threads=4 98 words sha256=DC165D8B4ADFF440 Avx512 threads=16 98 words sha256=DC165D8B4ADFF440 ``` The same program with AVX-512 switched off, and then with AVX2 off, which leaves the scalar path: ```shell $ DOTNET_EnableAVX512=0 dotnet run Avx2 threads=1 98 words sha256=DC165D8B4ADFF440 Avx2 threads=2 98 words sha256=DC165D8B4ADFF440 Avx2 threads=4 98 words sha256=DC165D8B4ADFF440 Avx2 threads=16 98 words sha256=DC165D8B4ADFF440 ``` ```shell $ DOTNET_EnableAVX2=0 dotnet run Scalar threads=1 98 words sha256=DC165D8B4ADFF440 Scalar threads=2 98 words sha256=DC165D8B4ADFF440 Scalar threads=4 98 words sha256=DC165D8B4ADFF440 Scalar threads=16 98 words sha256=DC165D8B4ADFF440 The scalar kernel path is running although this CPU supports AVX2 and FMA; OCR is far slower (about 100 times for a NativeAOT binary) but gives the same text. A NativeAOT app published below x86-64-v3 does this: set x86-64-v3 (or x86-64-v4 for AVX-512) in the app project. The RecursiveAI.OneOcr package sets v3 for x64 publishes unless you set your own. Turning the instruction sets off with DOTNET_EnableAVX2=0 or DOTNET_EnableHWIntrinsic=0 has the same effect. ``` - One hash on every kernel path and thread count. Dot products use fused multiply-add with a fixed 16-lane accumulation order whatever the vector width, `exp`, `tanh`, `log` and softmax are OneOcr’s own, and no sum crosses threads. The rules are in [Numeric rules](https://recursiveai.org/oneocr/docs/numerics/). - `PerformanceNotice` is set when the scalar path runs on a CPU that could do better. In a NativeAOT app the usual cause is an instruction-set level below `x86-64-v3`; the `RecursiveAI.OneOcr` package sets v3 for x64 publishes unless your project chooses its own. - The scalar path gives the same text but takes far longer: keep the switches above for checks like this one. # PDF pages Read a PDF whose page has both real text and a scanned image: the text layer is used where it exists, the image is read with OCR, and every word says which. ```shell dotnet add package RecursiveAI.OneOcr --version 1.0.0 dotnet add package RecursiveAI.OneOcr.Pdf --version 1.0.0 ``` `expense.pdf` is a one-page US Letter PDF printed from HTML: a heading and a paragraph as text, and the [grainy receipt](https://recursiveai.org/oneocr/cookbook/words-to-check/) as an embedded image. ```csharp using OneOcr; using OneOcr.Pdf; OcrEngine engine = OcrEngine.LoadBundled(); IReadOnlyList pages = PdfOcr.Read("expense.pdf", engine); foreach (OcrPage page in pages) { Console.WriteLine($"page {page.Index + 1}: {page.Width:F0} x {page.Height:F0} pt, {page.Words.Count} words"); foreach (var group in page.Words.GroupBy(w => w.Source)) Console.WriteLine($" {group.Key}: {string.Join(" ", group.Take(8).Select(w => w.Text))} ..."); foreach (OcrDiagnostic diagnostic in page.Diagnostics) Console.WriteLine($" {diagnostic.Code}: {diagnostic.Message}"); } ``` ```text page 1: 612 x 792 pt, 64 words TextLayer: Expense report EX-118 Submitted by Dana Whitfield on ... Ocr: Corner Market 1402 Oak Street 78704 Austin +1 ... OCR301: 1 regions were read from pixels at 300 DPI (47 words). ``` - Each PDF page is one `OcrPage` in PDF points, y down; `ToPdf` maps coordinates back to PDF user space. - Image regions the text doesn’t cover are rendered at `PdfOcrOptions.RenderDpi` (300 by default) through OneImage.Pdf and read with the local engine; `OCR301` says how many. - A text layer isn’t trusted blindly: an invisible layer over a scan (an earlier OCR pass) is read again unless you set `TrustExistingOcrLayer`, and garbled lines are read again from a render. See [PDF pages](https://recursiveai.org/oneocr/docs/pdf-bridge/). - For a `PdfPage` or `PdfDocument` you already hold from [OnePDF](https://recursiveai.org/onepdf/), the extension methods `Ocr`, `ToText`, `ToMarkdown`, `ExtractInvoice` and `ExtractReceipt` do the same. Text drawn as outlined glyphs and annotations aren’t read yet. # The oneocr command line Read images from a shell or a script with the `oneocr` tool: text to stdout, or a folder of pages to `.txt` and `.page.json` files. ```shell dotnet tool install --global RecursiveAI.OneOcr.Tool --version 1.0.0 ``` One image prints its text: ```shell $ oneocr ocr --in scans/receipt.png Corner Market 1402 Oak Street 78704 Austin +1 512 555 0142 Date: 24/10/2026 Time: 14:32 Coffee beans 12.99 2x Oat milk 8.98 Bagels 5.25 Subtotal 27.22 Tax 8.25 % 2.07 TOTAL $ 29.29 Paid by Card **** 4821 Thank you! ``` A folder writes one `.txt` per image, and `--json` adds the page model as `oneocr.page` JSON. `--degree` caps the threads, and the output doesn’t depend on it: ```shell $ oneocr ocr --in scans/ --out text/ --json $ oneocr ocr --in scans/ --out text-1/ --json --degree 1 $ ls text/ invoice.page.json invoice.txt receipt-photo.page.json receipt-photo.txt receipt.page.json receipt.txt $ diff -r text/ text-1/ && echo identical identical ``` - `ocr` reads PNG and JPEG with the bundled models and never touches the network. `--model` and `--detector` read your own model files instead, and `--stats ` writes load time, time per page and peak memory as JSON. - The same tool fine-tunes models on your own documents (`train`, `eval`, `review export`, `review import`); `oneocr help` lists every option, and the [review queue](https://recursiveai.org/oneocr/docs/review-queue/) describes the work directory. - The tool is NativeAOT-clean; publishing it ahead of time gives fast starts in scripts (see the [overview](https://recursiveai.org/oneocr/docs/overview/#nativeaot)).