---
title: "Keep the BatchNorm running mean fixed in eval mode in C#"
url: https://recursiveai.org/onemodel/cookbook/preserve-batchnorm-running-statistics/
description: "Switch BatchNorm2d to evaluation mode in C# to keep its running mean and variance fixed. Train on a tiny batch, then inspect both buffers in StateDict."
version: "1.3.0"
---

# Keep the BatchNorm running mean fixed in eval mode in C#

**OneModel** is a free, open-source neural-network library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneModel.Training) · [GitHub](https://github.com/recursive-ai-oss/onemodel-csharp) · [Docs](https://recursiveai.org/onemodel/docs/) · [Download example (.zip)](https://recursiveai.org/onemodel/cookbook/preserve-batchnorm-running-statistics/preserve-batchnorm-running-statistics.zip)

Call `Eval()` to keep batch normalization’s running mean and variance fixed.

```shell
dotnet add package RecursiveAI.OneModel.Training
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneModel.Tensors;
using RecursiveAI.OneModel.Training;

using var norm = new BatchNorm2d(1);
using var batch =
    Tensor.FromSpan<float>([2, 6], 2, 1, 1, 1);
using var noGrad = GradMode.NoGrad();
norm.Forward(batch).Dispose();
Console.WriteLine(
    norm.RunningMean.Item<float>());
norm.Eval();
norm.Forward(batch).Dispose();
Console.WriteLine(
    norm.RunningMean.Item<float>());
```

Unzip [the example](https://recursiveai.org/onemodel/cookbook/preserve-batchnorm-running-statistics/preserve-batchnorm-running-statistics.zip), then `dotnet run`.

A new layer starts in training mode, so the first pass moves the running mean from `0` to `0.4` (default momentum `0.1` times batch mean `4`); after `Eval()`, it stays at `0.4`.

```text
0.4
0.4
```

## The full program

The example trains on a single batch with values `2` and `6`, both with target `0`. The batch has shape `[2, 1, 1, 1]`: two samples, one channel, one row, one column. After one optimizer step, it switches to evaluation mode and feeds in `8` as a single sample. `StateDict()` lets you read the running mean and variance before and after both passes.

```csharp
using System.Globalization;
using RecursiveAI.OneModel.Tensors;
using RecursiveAI.OneModel.Training;

CultureInfo.CurrentCulture = CultureInfo.InvariantCulture;
using var norm = new BatchNorm2d(1, momentum: 0.5f);
using var batch = Tensor.FromSpan<float>([2, 6], 2, 1, 1, 1);
PrintStats("Initial");

norm.Train(true);
using (var activations = new ActivationScope())
using (var optimizer = new Sgd(norm.Parameters(), learningRate: 0.1f))
{
    using var target = Tensor.FromSpan<float>([0, 0], 2, 1, 1, 1);
    var loss = Losses.Mse(norm.Forward(batch), target);
    loss.Backward();
    optimizer.Step();
}
PrintStats("After training");

norm.Eval();
using (GradMode.NoGrad())
{
    using var probe = Tensor.FromSpan<float>([8], 1, 1, 1, 1);
    using var output = norm.Forward(probe);
    Console.WriteLine($"Eval output: {output.Item<float>():F3}");
}
PrintStats("After eval");

void PrintStats(string label)
{
    var state = norm.StateDict();
    var mean = state.Single(entry => entry.Name == "running_mean").Tensor;
    var variance = state.Single(entry => entry.Name == "running_var").Tensor;
    Console.WriteLine($"{label}: mean={mean.Item<float>()}, var={variance.Item<float>()}");
}
```

```text
Initial: mean=0, var=1
After training: mean=2, var=4.5
Eval output: 2.263
After eval: mean=2, var=4.5
```

If you forget `Eval()`, a forward pass in training mode uses the new batch’s statistics and updates both buffers, even when gradient recording is off.

## Variations

**Use evaluation mode for validation, then resume training.** Replace the lines from `norm.Eval()` onward in the snippet at the top:

```csharp
norm.Eval();
using (GradMode.NoGrad())
{
    using var validation = norm.Forward(batch);
    Console.WriteLine(norm.IsTraining);
}
norm.Train(true);
Console.WriteLine(norm.IsTraining);
```

```text
0.4
False
True
```

## Related recipes

- [Disable gradient recording for inference](https://recursiveai.org/onemodel/cookbook/disable-autograd-recording-for-inference/)
- [Freeze model parameters during training](https://recursiveai.org/onemodel/cookbook/freeze-model-parameters-during-training/)

## The short answer

After training, call `Eval()` on the layer or its parent model. `BatchNorm2d` then uses `running_mean` and `running_var` without updating either. Read them from `StateDict()` to check what training collected. Use `GradMode.NoGrad()` separately if you don’t need a backward pass.

## What does batch normalization normalize?

For an input shaped `[N, C, H, W]`, `BatchNorm2d` normalizes each channel separately. In training mode it measures that channel across the samples and spatial positions in the current batch, subtracts the mean, and divides by the square root of the variance plus a small epsilon. It then applies the channel’s learned scale (`weight`) and shift (`bias`).

The layer also blends the new statistics into its running buffers. `momentum` is the weight of the new statistic in that blend. In the full program the batch mean is `4`, so with momentum `0.5` the running mean moves from `0` to `2`. The training forward pass uses the batch variance, `4`, to normalize. The running buffer instead uses the unbiased estimate, `8` for these two values. Blending it halfway with the initial `1` gives `4.5`. See [OneModel’s batch normalization implementation](https://github.com/recursive-ai-oss/onemodel-csharp/blob/f453bf255cf2de843f4de083754d09356d53962a/src/OneModel/Tensors/Ops.Layers.cs#L143-L150).

## What does evaluation mode do to BatchNorm?

`Eval()` makes the layer use the statistics collected from training, so the sample’s normalization doesn’t depend on what else is in the prediction batch. It doesn’t recompute the running statistics from validation data. The layer still applies the learned scale and shift, which is why the printed prediction includes the effect of the optimizer step.

The running mean and variance are buffers, not optimizer parameters. `StateDict()` includes them alongside `weight` and `bias`; reading them as numbers at each stage lets you compare the values. `Eval()` also sets the mode on child modules, so with a model made of several layers you normally call it once on the parent. If you’re validating between training epochs, switch back with `Train(true)` before the next training pass. See [the OneModel training guide](https://recursiveai.org/onemodel/docs/training/).
