---
title: "Compress small messages with a Zstandard dictionary in C#"
url: https://recursiveai.org/onearchive/cookbook/zstd-dictionaries/
description: "Zstandard with a trained dictionary in C#: train it on samples of small, similar messages and compress each new message to a fraction of plain zstd. A complete program with OneCompress."
version: "1.0.0"
---

# Compress small messages with a Zstandard dictionary in C#

Train a dictionary from sample messages with `ZstdCompressionDictionary.Train`, then pass it in the options to compress and to decompress. The program below trains on part of a stream of small messages and compares the sizes of new ones with and without it.

Small payloads, such as game state updates, API messages or log lines, compress badly on their own: each one is too short for the compressor to learn its structure. A dictionary carries that structure in advance.

```shell
dotnet new console -o ZstdDict && cd ZstdDict
dotnet add package RecursiveAI.OneCompress --version 1.0.0
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using System.Text;
using RecursiveAI.OneCompress.Zstd;

// Many small, similar payloads: player state messages of 100-200 bytes.
var rng = new Random(1);
string Message(int i) =>
    $$"""{"type":"player_state","tick":{{1000 + i}},"player":"p{{rng.Next(1, 64)}}","pos":[{{rng.Next(0, 4096)}},{{rng.Next(0, 4096)}}],"hp":{{rng.Next(0, 101)}},"weapon":"{{(i % 3 == 0 ? "bow" : "sword")}}","status":["{{(i % 2 == 0 ? "running" : "idle")}}"]}""";
var samples = Enumerable.Range(0, 2000).Select(i => (ReadOnlyMemory<byte>)Encoding.UTF8.GetBytes(Message(i))).ToList();

// Train on part of the traffic; compress messages it has not seen.
var dictionary = ZstdCompressionDictionary.Train(samples[..1500], new ZstdDictionaryTrainingOptions { DictionarySize = 4096 });
Console.WriteLine($"dictionary id {dictionary.Id}, {dictionary.ToArray().Length:N0} bytes");

long raw = 0, plain = 0, withDict = 0;
foreach (var m in samples[1500..])
{
    raw += m.Length;
    plain += Zstd.Compress(m.Span, new ZstdCompressionOptions { Level = 3 }).Length;
    byte[] packed = Zstd.Compress(m.Span, new ZstdCompressionOptions { Level = 3, Dictionary = dictionary });
    withDict += packed.Length;
    byte[] back = Zstd.Decompress(packed, new ZstdDecompressionOptions { Dictionary = dictionary.ForDecompression });
    if (!back.AsSpan().SequenceEqual(m.Span)) throw new Exception("round trip failed");
}
Console.WriteLine($"500 messages: {raw:N0} bytes raw, {plain:N0} with zstd, {withDict:N0} with zstd and the dictionary");
```

```text
dictionary id 243566071, 4,096 bytes
500 messages: 55,508 bytes raw, 52,488 with zstd, 19,032 with zstd and the dictionary
```

## Limits

- **Both sides need the same dictionary.** Store it with your data or ship it with your app; a message compressed with it can’t be read without it.
- **It pays off for small messages.** On large files plain zstd already finds the repetition, and a dictionary adds little.

## Good to know

- Training is deterministic: the same samples and options give the same dictionary bytes on every platform and with any thread count, so peers can train their own copy. `ToArray()` gives the bytes to store or ship; `ZstdCompressionDictionary.Parse` loads them, and dictionaries in the standard Zstandard format.
- `ForDecompression` is the same dictionary prepared for decoding, made once and kept.
- The frame records the dictionary ID, and a decoder can hold several dictionaries (`ZstdDecompressionOptions.Dictionaries`) and picks the one the frame names.
- For a stream of tiny messages, reuse one encoder and decoder to avoid per-message allocations: see [streams of tiny messages](https://recursiveai.org/onearchive/docs/message-mode/).

Training options, the parameter search and how it compares with `zstd --train` are in [dictionary training](https://recursiveai.org/onearchive/docs/zstd-dictionary-training/).