Results

The measured results of the 1.0 packages (RecursiveAI.OneDiff, .Documents, .Pdf, .Ocr and the onediff tool): the text core's exact gates, fuzzing and speed against the managed comparators; the PDF and scan engine's gates and gold-set scores; change semantics and combine; budgets; platforms. Losing rows and gaps are listed, not left out. Nothing is claimed beyond this page. Method: measurement.md.

Status words: pass meets the bar; FAIL a gate with failures (listed with its count); floor a score recorded as the floor later releases may only raise; loses a row where a comparator scores better; information measured, no bar.

No commercial comparison product is measured or compared against. The adapter packages for Word, spreadsheets, presentations, rendering, OneNLP recognizers and the model ship later; their rows are not part of this page, except the DOCX comparator scores kept at the end as information.

What was run

Code under test: the release candidate plus test-and-tooling changes only (no library code changed). Sister libraries: the rows were measured on the OneFont 1.4.0 and OnePDF 1.18.0 release candidates; the release builds against the published OneFont 1.4.0, OnePDF 1.18.0, OneImage 1.10.5, OneRender 1.3.3 and oneocr 1.0.0, and its test suites passed on those on both machines. Comparators and reference tools run as released binaries and packages: DiffPlex 1.9.0, DiffMatchPatch 5.0.0, git 2.51.2, GNU diffutils 3.12, GNU patch 2.8, the diff-match-patch Python package 20241021, W3C Nu Html Checker 26.11.4, BenchmarkDotNet 0.15.8.

Data sets (all evaluation splits, never used for tuning): text corpus manifest sha256 087bd93a… (9,483 code and prose pairs, 2,599 merge triples; pinned fetch in tools/corpora-pins.json); synth-v1 e11b5614… (1,050 pairs, 7 languages); pdf-v1 eb10c6b3… (1,110 pairs rendered to PDF, 60 of them real); scan-v1 6fe36af3… (600 scanned pairs); combine-v1 5c55ebb5… (630 cases; regenerating it from the seed gives the same hash).

Exact gates (text core)

Oracles run as black-box processes. Zero failures in every gate.

The skipped cases are listed in the gate output; none is a failure. Cases skipped by the oracle (timeouts, over-size) are counted as skips, never passes.

Fuzzing

Seeded mutation runners (fuzz/OneDiff.Fuzz, no coverage feedback): inputs are mutations of patches the library writes plus hostile cases; for the change-list reader, mutations of valid comparisons; for the PDF source, mutations of valid PDFs (flips, cuts, splices, repeated chunks, extreme numbers). A crash is an exception other than the documented typed failure, a broken invariant, or an input slower than 3 s. Seed 1, release build, one process per target, all seven run side by side on the x64 machine (8 cores reserved; load average 11–16).

Document exact gates

Every pair is compared with the Legal and the Strict settings.

The rendered pdf-v1 failure, fixed. A table cell whose text wraps over two lines in the source PDF (for example "6" and "months") kept the line break in its node text; the inline page drew the cell as a stack of short lines, and the re-extraction read the neighbouring cell's text between the cell's lines. The grid reading now joins the lines of a cell with a space (a word broken by a hyphen keeps its break so the hyphenation fold still joins it). The smallest failing pair is a test (RenderedReadbackTests). Re-run on the fixed code: pdf-v1 Verify, fallback 0, Check, marks re-read and determinism on 1,110 pairs × Legal/Strict, the rendered read-back inline and side by side on 1,110 pairs, and the text corpus (638 pairs and 24 over 300 kB): 0 failures. The PDF digest slice (20 pairs) is identical on x64 and arm64.

After the cell-text pairing rule (a paired table cell with one unmatched text block on each side pairs them), the rows it can touch were scored again on the final code: synth-v1 eval (above: edited, unrelated and changes per gold unit improved, every other row identical), change semantics (every row identical), combine-v1 (every row identical, digest unchanged), scan-v1 (every row identical, 600 pairs), the scan-v1 exact gates on 100 pairs (every 6th; 5,322 changes, 8,921 marks, 20 determinism re-checks, 0 failures), and the alignment, invertibility and grouping sweeps at 60,000 cases each (seeds 1–60,000 in two chunks), 0 failures.

Same bytes on both machines

A fixed slice of pdf-v1 (20 pairs, every 56th) and of scan-v1 (20 pairs, every 30th), plus generated and corpus inputs. Each file below has the same SHA-256 on x64 and arm64.

Gold sets

Paired bootstrap over documents, 10,000 resamples, seed 20261104, 95% percentile intervals. Every row is scored once on the evaluation split. All rows equal or beat their floors except one, marked.

Alignment, boundaries, fragmentation (synth-v1 eval)

Gold documents as input; 210 documents, 1,050 pairs.

Fragmentation counts changes, not display regions, and is read beside boundary precision (definition in measurement.md). These are gold-set rows with floors, not comparator rows.

Change semantics (synth-v1 eval)

Known limit: the remaining cascade misses are Japanese reference edits merged with a neighbouring adverb change into one unit.

Combine and chains (combine-v1)

630 cases from the 210 evaluation documents with 2, 3 and 4 reviewers.

Reading a PDF (pdf-v1, every fifth pair)

The PDF source run on 210 synthetic pairs (222 compared, 0 failures), its nodes mapped onto the gold blocks (20,689 of 22,191 content nodes matched a gold block; the rest are page furniture and extraction artefacts, which count as unrelated).

Scanned pairs (scan-v1)

120 evaluation documents (30 per language: en, de, fr, es), 600 pairs. "Digital" is the same pairs read from the digital PDFs by the PDF source, for comparison; the digital column is information.

The scan boundary and noise rows are low because recognition errors make most changes "noise" (86%), a limit of the recognition engine; the flags mark such changes (precision .93, recall .62 at threshold .95). The digital column is lower than at the previous measurement (boundary F1 .851 against .879, edited .929 against .944, unrelated .466 against .438; intervals overlap); it is the PDF source reading the same pairs after later changes to clause numbers and tables, and it is not a floor row.

Speed: text core against the managed comparators

Comparators: DiffPlex 1.9.0 and DiffMatchPatch 5.0.0 (the diff-match-patch authors' C# package), restored in the benchmark project only. Ratio = ours ÷ comparator, so lower is better. Bar: geometric mean per comparator and area at most 1.0, no row above 1.25.

Settings. Single-threaded, workstation non-concurrent GC; DiffMatchPatch Diff_Timeout = 0; no cleanup in the diff rows; DiffPlex ignoreWhitespace: false with an explicit word separator set; DiffMatchPatch's line mode maps lines to characters inside the timed work (its helper is not public); cleanup rows time the copy of the input diff on both sides. BenchmarkDotNet 0.15.8, one launch per row with its default warm-up and iteration counts, Release. Inputs are corpus pairs of about 2 KB (Small), 40 KB (Medium) and 300–400 KB (Large; lines and words only), named in the raw tables. Each area ran as its own group of under 20 minutes. Raw BenchmarkDotNet output, the machine record and the input and edit-distance listing are kept with the run; peak working set is measured once per row in a cold child process.

Three-way merge has no managed comparator and is covered by the exact gate above, not a speed row.

• Words are not equal work. DiffPlex's word differ cuts words at the separators it is given, so it does not see our token stream: it reports fewer edits on the same input (Small 48 against our 57–61, Medium 60 against 69, Large 840 against 1,114). The ratios compare the time to produce each library's own script. Lines and characters agree on edit distance between all implementations except where a heuristic algorithm (patience, histogram) finds a longer, equally valid script than Myers.
• Not covered. No word rows against diff-match-patch (it has no word mode), no large-size character rows, no large-size cleanup or patch rows, no multi-threaded rows (the text core is single-threaded). Native tools (git, GNU diff) are not compared.

Allocations (bytes per call, x64; arm64 agrees within 1%)

Every measured row allocates less than its comparator (allocation ratio at most 0.67).

Memory

Peak working set above the inputs, x64, cold child process:

At the two largest rows where we are higher the process baseline dominates (within about 8–16% of DiffPlex's); no bar is set on peak memory for the text core.

Linear growth (lines, Myers, bytes allocated per diff on inputs of 2 KB, 40 KB and 317 KB): 9,144, 230,184 and 1,514,568, that is 4.6, 5.8 and 4.8 bytes per input byte. The time grows the same way (1.7 µs, 68 µs and 792 µs).

Budgets

A 100-page pair with about 300 changes, 4 cores (DOTNET_PROCESSOR_COUNT=4), one untimed warm-up and three timed repetitions per part, median reported, each part in its own process; method in measurement.md. The peak is the operating system's maximum resident set size. Budgets are targets of this release.

The scan OCR time is the recognition engine's, not ours; on the Mac it was measured on a loaded machine.

Platforms

The test run and the pack dry run were repeated after the adapter packages moved to the renamed document library: the same counts (864 tests, 0 failed; 5 packages) on both machines. Skipped tests are those that need a native host or a file set that is not present (including the pinned failing read-back test above). Browser WebAssembly (headless Chrome 155, x64): Core 361 tests, 353 passed, 0 failed, 8 skipped; .Documents 200 (197 passed, 3 skipped); .Pdf 135 (119 passed, 16 skipped); .Ocr 18 (8 passed, 10 skipped); skips are tests that need files, threads, processes or AES, marked native. The .Ocr skips include every test that reads a scan: under WebAssembly the image library (OneImage 1.10.5) throws a TypeLoadException while rasterizing, so scans are not supported in the browser until a OneImage fix (expected in OneImage 1.11).

DOCX comparators on docx-v1 (information; not part of this release)

Set: docx-v1, sha256 62b43875b9746cf41e59fafefa643e3ddf02c3301c490a6cf1eae7ce6697f91e: the 1,050 evaluation pairs of the synthetic set rendered to Word files in six looks. The free comparison tools were run as released binaries through their public interfaces, one run each, with default settings except where noted in results/docx-v1-comparators.md (Docxodus 15.0.0 package, OfficeIMO.Word 3.5.3, WmlComparer through python-redlines 2.0.0, LibreOffice 26.8.2.2 headless). Indicative: one run each, on a shared machine. Bootstrap 95% intervals over documents (1,000 resamples).

OneDiff's own rows on this set: pending: the DOCX package is not in this release. The engine's tracked-changes DOCX output is not scored here.

The comparators' zeros for moved, split and merged are not misreads: only Docxodus writes move markers, and none of the others can express a link between paragraphs. OfficeIMO's fragmentation of 1.000 is a paragraph per change (its boundary F1 is .337). Speed of the comparators on joined pairs of about 5, 40 and 120 pages is in the linked file (indicative); the DOCX speed rows of OneDiff are pending with the package.

Rows where OneDiff's DOCX output and the free tools are compared (strictly better, the ceiling clause, the blind-preference rows) are not published in this release.

Failures, rows below their floor, and gaps

• PDF source, changes per gold unit: floor re-based to 1.174 [1.133, 1.219] from 1.135, because the earlier value counted wrong matches of short equal cells across table rows. An inserted or deleted table row now costs one change per cell; making it one change is planned. Every other pdf-v1 row is at or above its floor.
• At the floor within rounding: PDF/PDF moved align F1 (.8429 against .843).
• Scan limits: scan boundary F1 (.237), split and merge recovery (F1 .055 and .109) and moved precision are limited by recognition errors and by paragraphs the recognition engine merges; the flags (precision .93, recall .62) mark rather than hide them. Latin-script scans only.
• Known limit: one 29.7 MB pair exceeds the renderer's display-list limit; the rendered redline gives a clear error for it. It is a limit, not a failure of the gates above.
• Text-core rows that lose: none. Patch make on the medium size is slower than diff-match-patch's by 1.13× on x64 and 1.02× on arm64, inside the bar (geomean .85 and .83); peak memory on the largest line and word inputs is above DiffPlex's (5.7 against 5.3 MB and 14.7 against 12.7 MB), and no bar is set on it.
• Not run in this release: DOCX engine rows and DOCX speed rows against the free DOCX tools, blind-preference rows, grid and slide rows, the model rows, and a WebAssembly timing row; the packages that produce them ship later. There is no managed comparator for PDF, scan, merge or document speed, so those rows are budgets only.
• Platform coverage: the browser suites ran on x64 only; native AOT, the whole-solution tests and the pack dry run ran on both machines. The scan-time and OCR rows were taken on a 20-pair slice.