---
title: "Train a neural network in C#"
url: https://recursiveai.org/onemodel/cookbook/train-a-model/
description: "Train a neural network in C# on MNIST with no Python, no native libraries and no GPU: load the data, train, save the model and run it. Complete Program.cs and real output, with OneModel."
version: "1.0.0"
---

# Train a neural network in C#

`RecursiveAI.OneModel.Training` trains models in plain C# on the CPU, then saves them as an ordinary model file that `Model.Load` runs. The program below trains a small network on MNIST, the 70,000 handwritten digits every machine-learning course starts with. It reaches 96% on the test set in three passes over the data, saves the model and runs it again from the file. No Python, no native libraries.

```shell
dotnet new console -o TrainMnist && cd TrainMnist
dotnet add package RecursiveAI.OneModel.Training --version 1.0.0
curl -O https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz
curl -O https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz
curl -O https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz
curl -O https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz
```

The training package brings the runtime, `RecursiveAI.OneModel`, with it. The four files are MNIST from the PyTorch project’s mirror, 11 MB in all. Put this in `Program.cs` and run `dotnet run`:

```csharp
using System.IO.Compression;
using RecursiveAI.OneModel;
using RecursiveAI.OneModel.Tensors;
using RecursiveAI.OneModel.Training;

// MNIST: 60,000 training and 10,000 test images of handwritten digits,
// 28 x 28 gray pixels each.
float[] trainX = Images("train-images-idx3-ubyte.gz");
long[] trainY = Labels("train-labels-idx1-ubyte.gz");
float[] testX = Images("t10k-images-idx3-ubyte.gz");
long[] testY = Labels("t10k-labels-idx1-ubyte.gz");
Console.WriteLine($"Train: {trainY.Length:N0} images, test: {testY.Length:N0}");

// Seeded: the same seed gives the same starting weights.
var generator = new Generator(42);
using var model = new Mlp(generator);
using var optimizer = new AdamW(model.Parameters(), learningRate: 1e-3f);

const int Batch = 128;
var shuffle = new Random(42);
int[] order = Enumerable.Range(0, trainY.Length).ToArray();
for (int epoch = 1; epoch <= 3; epoch++)
{
    shuffle.Shuffle(order);
    float lossSum = 0;
    int steps = 0;
    for (int start = 0; start + Batch <= order.Length; start += Batch)
    {
        var xs = new float[Batch * 784];
        var ys = new long[Batch];
        for (int i = 0; i < Batch; i++)
        {
            int n = order[start + i];
            trainX.AsSpan(n * 784, 784).CopyTo(xs.AsSpan(i * 784));
            ys[i] = trainY[n];
        }

        using var scope = new ActivationScope();   // frees this step's tensors
        using var x = Tensor.FromSpan<float>(xs, Batch, 784);
        using var y = Tensor.FromSpan<long>(ys, Batch);
        optimizer.ZeroGrad();
        Tensor loss = Losses.CrossEntropy(model.Forward(x), y);
        loss.Backward();
        optimizer.Step();
        lossSum += loss.Item<float>();
        steps++;
    }
    Console.WriteLine($"Epoch {epoch}: mean loss {lossSum / steps:F4}");
}

// Save the trained network as an ordinary model file. "batch" stays
// variable; 784 is fixed. traced.SaveOnnx("mnist-mlp.onnx") writes ONNX.
using (Tensor example = Tensor.Zeros(DType.F32, 2, 784))
using (var traced = model.Trace(example, "batch", null))
    traced.Save("mnist-mlp.onemodel");

// Run it like any other model: this part needs only RecursiveAI.OneModel.
using var trained = Model.Load("mnist-mlp.onemodel");
float[] scores = trained.Run(testX);   // 10,000 test images, 10 scores each
int correct = 0;
for (int i = 0; i < testY.Length; i++)
    if (ArgMax(scores.AsSpan(i * 10, 10)) == testY[i]) correct++;
Console.WriteLine($"Test accuracy: {correct * 100.0 / testY.Length:F2}%");

int predicted = ArgMax(trained.Run(testX.AsSpan(0, 784).ToArray()));
Console.WriteLine($"First test image: label {testY[0]}, predicted {predicted}");

static int ArgMax(ReadOnlySpan<float> v)
{
    int best = 0;
    for (int i = 1; i < v.Length; i++) if (v[i] > v[best]) best = i;
    return best;
}

static byte[] Read(string path)
{
    using var file = File.OpenRead(path);
    using var gz = new GZipStream(file, CompressionMode.Decompress);
    using var ms = new MemoryStream();
    gz.CopyTo(ms);
    return ms.ToArray();
}

// IDX files: a 16-byte header for images, 8 for labels,
// then one byte per pixel or label.
static float[] Images(string path) =>
    Read(path)[16..].Select(b => b / 255f).ToArray();
static long[] Labels(string path) =>
    Read(path)[8..].Select(b => (long)b).ToArray();

sealed class Mlp : NnModule
{
    private readonly Linear _hidden, _output;

    public Mlp(Generator g)
    {
        _hidden = RegisterModule("hidden", new Linear(784, 128, g));
        _output = RegisterModule("output", new Linear(128, 10, g));
    }

    public override Tensor Forward(Tensor input) =>
        _output.Forward(Ops.Relu(_hidden.Forward(input)));
}
```

```text
Train: 60,000 images, test: 10,000
Epoch 1: mean loss 0.4144
Epoch 2: mean loss 0.1961
Epoch 3: mean loss 0.1413
Test accuracy: 96.21%
First test image: label 7, predicted 7
```

## How it works

- **The model** is a subclass of `NnModule`: 784 pixels in, a hidden layer of 128 with ReLU, and 10 scores out, one per digit. `RegisterModule` gives each layer a name, so its parameters have stable names for checkpoints. Nothing is found by reflection, so it works under NativeAOT and trimming.
- **The loop** is the usual one: clear the gradients, run the batch forward, compute the cross-entropy loss, `Backward()`, `optimizer.Step()`. `Sgd` and `Adam` swap in for `AdamW`, and a schedule such as `CosineLr` sets the learning rate per step.
- **`ActivationScope`** returns everything a step computed to the pool when it ends, so memory stays flat however long you train.
- **The same seed gives the same run.** `Generator(42)` fixes the starting weights and `Random(42)` the batch order, so this program prints the same numbers every time it runs on one machine.
- **Tracing** records one forward pass as a graph. The saved `.onemodel` file needs only `RecursiveAI.OneModel` to run, the same way as an imported ONNX model, and gets the same graph optimizations. `SaveOnnx` writes ONNX instead, for other runtimes.

To stop and resume a long run, `Checkpoint.Save(path, model, optimizer)` stores the weights and the optimizer’s state, and a run resumed from it continues bit for bit: see [training and checkpoints](https://recursiveai.org/onemodel/docs/guide-training/).

## Limits

- **CPU only.** Training runs on OneModel’s SIMD CPU kernels; there is no GPU backend. That suits small and medium models like this one, not large ones.
- **Other machines can differ in the last digits.** In the default fast mode, another CPU may pick other kernels and round differently, so the losses can move slightly. For the same bits everywhere, add this line before the training loop:

  ```csharp
  using var compute = TensorCompute.Use(ArithmeticMode.Exact);
  ```
- **A small network.** 96% is what a two-layer network reaches here. `Conv2d`, `BatchNorm2d` and pooling are in the package for a convolutional one, which does better on images.
- **The data is all in memory.** This program loads the 60,000 images as floats (about 190 MB). For bigger datasets, the `Data` namespace has datasets, samplers and a prefetching `DataLoader` ([data loading](https://recursiveai.org/onemodel/docs/data-loading/)).
- **What can’t be traced.** Losses, dropout in training mode, and code that branches on a tensor’s values throw `TraceException` naming the operation. Trace the forward pass only, as here.

Running a model someone else trained is in [Run an ONNX model in C#](https://recursiveai.org/onemodel/cookbook/run-onnx-model/), and the training reference is [training](https://recursiveai.org/onemodel/docs/training/).