---
title: "Measure OCR character and word error rates in C#"
url: https://recursiveai.org/oneocr/cookbook/evaluate-local-ocr-model/
description: "Use OneOcr’s local evaluator to measure character error rate and word error rate against hand-checked line transcriptions."
version: "1.0.0"
---

Score an OCR model on lines it did not train on by counting character and word edits. The zip’s program exports `recognizer.onm` from the bundled models and runs `oneocr eval`; the snippet scores the same file directly in C#.

```shell
dotnet add package RecursiveAI.OneOcr.Training
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using OneOcr.Training;

const string model = "recognizer.onm";
var report = OcrTrainer.Evaluate(model, new() {
    ModelPath = model,
    WorkDirectory = "eval-data"
});
Console.WriteLine(OcrTrainer.ToJson(report));
```

Unzip [the example](https://recursiveai.org/oneocr/cookbook/evaluate-local-ocr-model/evaluate-local-ocr-model.zip), then `dotnet run`.

Prints character and word error rates as JSON. The zip includes `eval-data/` with the line crops for `Rack 24`, `Bin 51`, and `Shelf 82`, plus their hand-checked references. Each `labels/*.json` stores its line’s `cropWidth` and base64 `crop` bytes; `reviewed.json` stores the verified text. One review entry:

```json
{ "id": "4e1c39d61a01518a-0", "text": "Rack 24" }
```

The part before `-0` is the label file’s `id`. The default held-out split selects one of the three fixture documents; each has one line, so only one line is scored. The snippet prints:

```json
{
  "rates": {
    "cer": 0,
    "wer": 0,
    "lines": 1,
    "exactLines": 1
  },
  "suggestedLocalFirstThreshold": null,
  "thresholdSupport": 0
}
```

`suggestedLocalFirstThreshold` is a suggested confidence cutoff for accepting a local reading, and `thresholdSupport` counts the scored lines at or above it; one held-out line isn’t enough to suggest a cutoff.

## Full program

`dotnet oneocr eval` runs the tool pinned in the zip’s `dotnet-tools.json`, so `dotnet run` does not need a global tool installation. To build the runner in your own console project, also install the package it uses to get the bundled recognizer:

```shell
dotnet add package RecursiveAI.OneOcr.Models
```

```csharp
using System.Diagnostics;
using OneOcr.Models;

File.WriteAllBytes("recognizer.onm", BundledModels.ReadRecognizer());
Run("tool", "restore");
Run("oneocr", "eval", "--model", "recognizer.onm", "--work", "eval-data");

static void Run(params string[] arguments)
{
    var command = new ProcessStartInfo("dotnet");
    foreach (string argument in arguments)
        command.ArgumentList.Add(argument);
    using var process = Process.Start(command)!;
    process.WaitForExit();
    if (process.ExitCode != 0)
        throw new InvalidOperationException($"dotnet {arguments[0]} failed.");
}
```

After restoring the tool, the program prints the same JSON report as the C# snippet. Restore can fetch packages from NuGet; evaluation itself does not use the network. If no held-out line has reference text, `eval` fails instead of printing a zero-error report.

## Variations

In the unzipped example, after `dotnet run`, evaluate two of the three fixture documents by setting `--held-out`:

```shell
dotnet oneocr eval --model recognizer.onm --work eval-data --held-out 0.67
```

The report then shows `"lines": 2` and `"exactLines": 2` for these crops.

## Related recipes

- [Fine-tune an OCR recognizer offline](https://recursiveai.org/oneocr/cookbook/fine-tune-local-ocr-recognizer/)
- [Load a custom OCR recognizer](https://recursiveai.org/oneocr/cookbook/load-custom-ocr-recognition-model/)

## The short answer

Call `OcrTrainer.Evaluate` in C#, or run `oneocr eval`, with a recognizer file and a local work directory of transcribed line crops. Either compares the recognizer’s readings with the references in the held-out split. Read `rates.cer` and `rates.wer` as ratios, and check `rates.lines` to see how many lines the report actually covers. This measures line recognition, not whether a full-page pipeline finds all the lines or puts them in the right order.

To build the files from your own line images without a provider, use the crop and transcription fields in the [work-directory spec](https://recursiveai.org/oneocr/docs/review-queue/#work-directory); the [CLI labeling and review flow](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/README.md#L513-L526) writes those files from page images if you configure a labeling provider.

## How do you calculate character error rate and word error rate?

Sum the minimum number of edits needed for each line: a substitution, deletion, or insertion counts as one. Then divide by the total number of reference units across the scored lines. For CER the units are Unicode code points, including normalized spaces; for WER they are space-separated words. OneOcr normalizes both sides to NFC, trims and collapses whitespace, and keeps case and punctuation. See the [scoring implementation](https://github.com/recursive-ai-oss/oneocr-csharp/blob/8db430fb77c4d86d51686f15dfd9b5ccc20c99a7/src/OneOcr.Training/TextMetrics.cs#L95-L129).

For example, reading `Rack 24` as `Rack 2A` costs one character edit out of seven, so CER is about 14.3%. It also costs one word edit out of two, so WER is 50%. The CLI prints ratios and line counts, not separate counts for insertions, substitutions, and deletions. Enough insertions can make the ratio greater than 1, since the denominator is the reference length, not the length of the OCR output.

## What does a zero OCR error rate tell you?

It tells you that the scored text matched the supplied references after normalization, not that every new page will be read correctly. Here it covers only one synthetic line. Keep verified references for documents not used for training, including the kinds of print and scan defects you expect. When comparing two recognizer files against the same work directory, use the same `--seed` and `--held-out` for both. Otherwise the models may be scored on different documents.

The threshold in the report is separate from the error rates. OneOcr only suggests a confidence cutoff if at least five held-out lines support it and the requested share of lines at or above it are exact matches. This fixture is too small to suggest one, even with zero measured errors.
