---
title: "Serialize annotated NLP documents to binary in C#"
url: https://recursiveai.org/onenlp/cookbook/store-annotated-documents-in-binary-bin/
description: "Store text and its annotations together in a binary document bin. Documents in a bin share one string table."
version: "1.0.0"
---

# Serialize annotated NLP documents to binary in C#

**OneNLP** is a free, open-source natural language processing library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneNlp) · [GitHub](https://github.com/recursive-ai-oss/onenlp-csharp) · [Docs](https://recursiveai.org/onenlp/docs/) · [Download example (.zip)](https://recursiveai.org/onenlp/cookbook/store-annotated-documents-in-binary-bin/store-annotated-documents-in-binary-bin.zip)

A document bin stores annotated text with a shared string table.

```shell
dotnet add package RecursiveAI.OneNlp
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
// annotated.conllu: zip input shown below.
using RecursiveAI.OneNlp;
using RecursiveAI.OneNlp.Data;
using RecursiveAI.OneNlp.Serialization;

var bin = new DocBin();
foreach (var sentence in ConllUReader.Read("annotated.conllu"))
{
    sentence.Doc.SetCats([new("ANIMAL", 1f)]);
    bin.Add(sentence.Doc);
}
File.WriteAllBytes("docs.bin", bin.ToBytes());
var restored = DocBin.FromBytes(
    File.ReadAllBytes("docs.bin"));
Console.WriteLine($"Restored {restored.Count} documents");
```

```text
Restored 2 documents
```

Unzip [the example](https://recursiveai.org/onenlp/cookbook/store-annotated-documents-in-binary-bin/store-annotated-documents-in-binary-bin.zip), then `dotnet run`.

## Full program

The full program adds an `ANIMAL` category to each document, writes `docs.bin` and checks the restored documents against the original JSON. It then tries a truncated copy. The offsets printed below are half-open UTF-16 ranges: the start is included and the end is excluded.

`annotated.conllu` (tab-separated):

```text
1	Cats	cats	NOUN	NNS	_	2	nsubj	_	_
2	chase	chase	VERB	VBP	_	0	root	_	_
3	birds	birds	NOUN	NNS	_	2	obj	_	_

1	Dogs	dogs	NOUN	NNS	_	2	nsubj	_	_
2	chase	chase	VERB	VBP	_	0	root	_	_
3	cats	cats	NOUN	NNS	_	2	obj	_	_
```

```csharp
using RecursiveAI.OneNlp;
using RecursiveAI.OneNlp.Data;
using RecursiveAI.OneNlp.Serialization;

var bin = new DocBin();
foreach (var sentence in ConllUReader.Read("annotated.conllu"))
{
    sentence.Doc.SetCats([new("ANIMAL", 1f)]);
    bin.Add(sentence.Doc);
}
File.WriteAllBytes("docs.bin", bin.ToBytes());
var restored = DocBin.FromBytes(
    File.ReadAllBytes("docs.bin"));
for (int i = 0; i < restored.Count; i++)
{
    var doc = restored.Docs[i];
    if (doc.ToJson() != bin.Docs[i].ToJson())
        throw new InvalidDataException("Round-trip changed annotations.");
    Console.WriteLine($"{doc.Text} | ANIMAL={doc.Cats["ANIMAL"]}");
    foreach (var token in doc)
        Console.WriteLine($"  {token.Text} [{token.Idx},{token.EndIdx}) head={token.HeadIndex} {token.Dep}");
}
Console.WriteLine("All document JSON matches after binary round-trip");
byte[] bytes = File.ReadAllBytes("docs.bin");
try
{
    DocBin.FromBytes(bytes.AsSpan(0, bytes.Length - 1));
    throw new Exception("Truncated bin was accepted.");
}
catch (InvalidDataException)
{
    Console.WriteLine("Truncated bin: InvalidDataException");
}
```

`dotnet run` prints:

```text
Cats chase birds | ANIMAL=1
  Cats [0,4) head=1 nsubj
  chase [5,10) head=1 root
  birds [11,16) head=1 obj
Dogs chase cats | ANIMAL=1
  Dogs [0,4) head=1 nsubj
  chase [5,10) head=1 root
  cats [11,15) head=1 obj
All document JSON matches after binary round-trip
Truncated bin: InvalidDataException
```

Truncated or invalid binary input throws `InvalidDataException`; a bin reader checks the checksum after the last document.

## Variations

### Read documents one at a time

After writing `docs.bin`, enumerate its records instead of materializing a second whole bin:

```csharp
using var stream = File.OpenRead("docs.bin");
foreach (var doc in new DocBinReader(stream))
    Console.WriteLine(doc.Text);
```

```text
Cats chase birds
Dogs chase cats
```

## Related recipes

- [Use pretrained part-of-speech tagging](https://recursiveai.org/onenlp/cookbook/part-of-speech-tagging/)
- [Use a pretrained dependency parser](https://recursiveai.org/onenlp/cookbook/dependency-parsing/)
- [Read CoNLL-U training data](https://recursiveai.org/onenlp/cookbook/read-conllu-training-data/)
- [Save NLP results as JSON](https://recursiveai.org/onenlp/cookbook/save-nlp-results-as-json/)

---

## The short answer

Use OneNLP’s `DocBin` to serialize several annotated documents into one binary container. Add each `Doc`, write `ToBytes()` to a file, then reconstruct it with `DocBin.FromBytes`. Use `DocBinReader` when you want to consume records one at a time. This is a document format, not general-purpose object serialization or a trained-model checkpoint.

## How do you serialize an NLP document to binary in C#?

Create or load annotated documents first, then add them to a bin. A bin keeps references rather than copies, so changes made to documents before serialization affect what is written. Documents may come from different `StringStore` instances; the writer constructs the shared table for the resulting container. The [binary format](https://recursiveai.org/onenlp/docs/binary-format/) names the many-document container `ONDB` and the single-document container `ONDC`.

The example starts from CoNLL-U so that the inputs visibly contain token boundaries, POS labels, heads and dependency relations. It adds a document category in code. The full program compares the original and restored JSON representations, then prints text, offsets, head indices and the category score. Those checks exercise this particular fixture; they are not a substitute for checking whichever annotations your own application relies on.

The shared string table stores repeated annotation values once across the bin. It is not a table of trained weights. If you need to keep an encoder and its prediction heads, save a pipeline file instead. If you need human-readable annotated text for inspection or interchange, [document JSON](https://recursiveai.org/onenlp/cookbook/save-nlp-results-as-json/) is the corresponding alternative.

## How do you deserialize binary documents in C#?

`DocBin.FromBytes` materializes the complete bin. `DocBinReader` takes a stream and yields records in order, allowing you to process each document without holding the full restored collection. Its constructor reads the header and string table; each read then decodes a record. The checksum is verified after the last record. Finish enumeration if you need that final checksum check.

The stream variation opens the file with `using`, which disposes the file handle after enumeration. Streaming the read does not make the write incremental: `WriteBinary` uses the same bytes as `ToBytes`, and the implementation builds them before writing. Plan memory use around the actual API rather than assuming that every stream overload avoids buffering.

## How does binary serialization differ from JSON serialization?

Both can preserve document annotations, but their representations serve different needs. JSON is readable text; the binary containers use section lengths, table indices, token offsets and an optional CRC-32 trailer. The binary specification is language-neutral and versioned. Unknown sections can be skipped by length, so readers can retain the parts they understand.

Keep readers and writers aligned with the annotations you need. The pinned specification notes that categories and span groups were added to version 1 after its first release, and older readers skip those sections. The released package used here round-trips the category in the example bin; the JSON equality check makes that visible rather than assuming every historical reader supports it.

Readers reject invalid input with `InvalidDataException`, including bad magic or version, truncation, invalid indices and failed checksums. The documented token-count cap is 33,554,432. Stream readers accept at most a declared length of 1 GiB and read it in 1 MiB steps, allocating no more than the stream supplies. These are format-reader limits, not a recommended document size or a guarantee that a process has enough memory for the maximum. The truncated-input case in the program shows how to catch the same exception your application should handle when a stored file is incomplete.
