---
title: "Store ONNX model weights as float16 in C#"
url: https://recursiveai.org/onemodel/cookbook/half-precision-weights/
description: "Halve the memory of an ONNX model’s weights in C# by storing them as float16 (binary16) or bfloat16, and measure the accuracy cost. Runnable code, with OneModel."
version: "1.0.0"
---

# Store ONNX model weights as float16 in C#

Set `ModelOptions.WeightStorage` to `Binary16` or `BFloat16`, and the large weights of a float32 model are kept in 16 bits, half the memory. The program runs `wide.onnx` three ways and prints how far the outputs move.

```shell
dotnet new console -o HalfWeights && cd HalfWeights
dotnet add package RecursiveAI.OneModel --version 1.0.0
curl -O https://recursiveai.org/onemodel/samples/wide.onnx
```

Put this in `Program.cs` and run `dotnet run`:

```csharp
using RecursiveAI.OneModel;
using RecursiveAI.OneModel.Graph;

float[] input = Enumerable.Range(0, 256).Select(i => MathF.Cos(i * 0.1f)).ToArray();

using var full = Model.Load("wide.onnx");
using var half = Model.Load("wide.onnx", new ModelOptions { WeightStorage = WeightStorage.Binary16 });
using var bf16 = Model.Load("wide.onnx", new ModelOptions { WeightStorage = WeightStorage.BFloat16 });

float[] a = full.Run(input), b = half.Run(input), c = bf16.Run(input);
Console.WriteLine($"float32  y[0..3] = {Show(a)}");
Console.WriteLine($"Binary16 y[0..3] = {Show(b)}  max |diff| = {MaxDiff(a, b):E2}");
Console.WriteLine($"BFloat16 y[0..3] = {Show(c)}  max |diff| = {MaxDiff(a, c):E2}");

static string Show(float[] v) => string.Join(", ", v[..3].Select(x => x.ToString("F5")));
static float MaxDiff(float[] x, float[] y) => x.Zip(y).Max(p => MathF.Abs(p.First - p.Second));
```

```text
float32  y[0..3] = -0.52048, -0.56874, 0.46823
Binary16 y[0..3] = -0.52028, -0.56858, 0.46820  max |diff| = 2.71E-004
BFloat16 y[0..3] = -0.52041, -0.57004, 0.46990  max |diff| = 2.35E-003
```

## What changes

- `WeightStorage.Binary16` and `WeightStorage.BFloat16` round the constant weights of `MatMul`, `Gemm` and `Conv` (1024 values or more) to 16 bits when the session is created and free the float32 copy. Each run widens them into scratch memory. Activations stay float32, so the only change is the rounding of the weights.
- Embedding tables, biases and normalization parameters stay as they are.
- binary16 keeps more significand bits, so it is closer to float32 here. bfloat16 keeps float32’s range; binary16 ends at 65504, and a weight tensor with a larger value is left in float32.
- Models whose weights are already float16 or bfloat16 run as they are, with no option needed.

## Limits

- **Outputs move.** Rounding the weights changes results in the last digits, as above; measure on your own model before you ship it.
- **It saves memory, not compute.** Each run widens the 16-bit weights back to float32 in scratch memory.

Conversions and the widening rule are in [half-precision storage](https://recursiveai.org/onemodel/docs/fp16/). For int8 models (QDQ, `MatMulInteger`, `QLinearConv` and the rest), see [int8 inference](https://recursiveai.org/onemodel/docs/int8/).