---
title: "Run GPU layer normalization in C#"
url: https://recursiveai.org/onegpu/cookbook/gpu-layer-normalization/
description: "Run layer normalization over a float tensor on the GPU in C#. Read back the result with OneGPU and compare it with the documented CPU error tolerance."
version: "1.0.0"
---

# Run GPU layer normalization in C#

**OneGpu** is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneGpu.Backends) · [GitHub](https://github.com/recursive-ai-oss/onegpu-csharp) · [Docs](https://recursiveai.org/onegpu/docs/) · [Download example (.zip)](https://recursiveai.org/onegpu/cookbook/gpu-layer-normalization/gpu-layer-normalization.zip)

Normalize tensor rows on a GPU; a working backend and driver are required.

```shell
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var gpu = GpuBackends.CreateDevice();
using var x = gpu.CreateBuffer<float>(
    [1, 2, 3, 4]);
using var y = gpu.CreateBuffer<float>(4);
using var norm = new NormKernels(gpu);
var commands = gpu.BeginCommands();
norm.Encode(commands, false, x, 0, 1, 4,
    null, 0, null, 0, y, 0);
gpu.Submit(commands);
var values = new float[4];
await y.ReadAsync(values);
Console.WriteLine(string.Join(", ",
    values.Select(v => $"{v:F6}")));
```

`false` selects layer normalization, `1` and `4` mean one row of four values, `null` means no scale or bias buffer, and each `0` is an element offset.

```text
-1.341635, -0.447212, 0.447212, 1.341635
```

Unzip [the example](https://recursiveai.org/onegpu/cookbook/gpu-layer-normalization/gpu-layer-normalization.zip), then `dotnet run`.

## The full program

The full program normalizes the row `[1, 2, 3, 4]`, reads the result and compares it with a double-precision CPU formula using the documented normalized-error tolerance. With no `ONEGPU_ADAPTER` override, `Adapter = "discrete"` throws `GpuUnsupportedException` if only an integrated GPU is available; omit the adapter option to use the default device ([adapter selection](https://recursiveai.org/onegpu/docs/environment/)).

```csharp
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var device = GpuBackends.CreateDevice(
    new GpuDeviceOptions { Adapter = "discrete" });
using var input = device.CreateBuffer<float>([1, 2, 3, 4]);
using var output = device.CreateBuffer<float>(4);
using var norm = new NormKernels(device);
var commands = device.BeginCommands();
norm.Encode(commands, false, input, 0, 1, 4,
    null, 0, null, 0, output, 0);
device.Submit(commands);
var result = new float[4];
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result.Select(x => $"{x:F6}")));
for (int i = 0; i < result.Length; i++)
{
    double expected = (i + 1 - 2.5) / Math.Sqrt(1.25 + 1e-5f);
    if (Math.Abs(result[i] - expected) / (1.5 / Math.Sqrt(1.25 + 1e-5f)) > 1e-5)
        throw new Exception("Layer normalization mismatch.");
}
Console.WriteLine("CPU normalized error <= 1e-5: passed");
Console.WriteLine($"{device.Backend}: {device.Adapter.Name}");
```

`dotnet run` prints:

```text
-1.341635, -0.447212, 0.447212, 1.341635
CPU normalized error <= 1e-5: passed
Vulkan: NVIDIA GeForce RTX 4070
```

Invalid dimensions or out-of-buffer ranges throw `GpuKernelShapeException`; device creation also requires an available backend and compatible driver.

## Variations

**Use RMS normalization instead.** Replace the `norm.Encode` call in the full program with this call and omit the layer-normalization CPU comparison loop and its success message:

```csharp
norm.Encode(commands, true, input, 0, 1, 4,
    null, 0, null, 0, output, 0);
```

```text
0.365148, 0.730296, 1.095444, 1.460593
Vulkan: NVIDIA GeForce RTX 4070
```

## Related recipes

- [Compute softmax on the GPU](https://recursiveai.org/onegpu/cookbook/compute-softmax-on-gpu/)
- [Read back a GPU buffer](https://recursiveai.org/onegpu/cookbook/read-back-a-gpu-buffer/)
- [Get deterministic GPU results](https://recursiveai.org/onegpu/cookbook/deterministic-gpu-results/)

## The short answer

Use `NormKernels.Encode` with `rms: false` to perform layer normalization. Supply the row count and row length, encode into a command buffer, submit it, and read the output buffer asynchronously. Optional scale and bias buffers apply per column. Passing `null` for them means a scale of one and a bias of zero.

This is an actual GPU dispatch, not a CPU fallback. The printed result above came from Vulkan on an NVIDIA GeForce RTX 4070. OneGPU also documents Metal and browser WebGPU backends, but those were not exercised for this recipe. Check that your deployment has its required backend and driver rather than treating a successful NuGet restore as evidence that a GPU is available.

## How do I run layer normalization on the GPU from C#?

Create device buffers for your input and output, construct `NormKernels` on the same device, and encode the normalization. The shape is rows of consecutive values, with normalization over the last axis. For a larger tensor, `rows` is the product of the preceding dimensions and `length` is the size of that last dimension. Offsets are element offsets, not byte offsets.

For each row, layer normalization subtracts the row mean and divides by the square root of the row variance plus `epsilon`. The default epsilon is `1e-5f`. The sample row has mean `2.5` and variance `1.25`, which is why its output is symmetric around zero. Scale and bias, if supplied, follow that normalization rather than changing the mean and variance calculation.

Encoding records work; `Submit` dispatches it. `ReadAsync` copies the finished result into the C# array. Keep the device, kernel family and buffers alive until the readback completes. The `using` declarations in the complete program dispose them afterward. When integrating the operation into a larger GPU pipeline, leave the result in its GPU buffer until another CPU operation genuinely needs it.

The [kernel documentation](https://recursiveai.org/onegpu/docs/kernels/) defines the comparison measure as the maximum absolute difference divided by the maximum absolute reference value over the output. For float layer normalization, its test tolerance is `1e-5`; for half with float accumulation it is `2e-3`. The complete program uses the float tolerance against a double-precision reference, not bitwise equality. These are documented test tolerances, not a promise about every tensor’s conditioning.

### How do I switch between layer normalization and RMS normalization?

The boolean immediately after the encoder selects the operation: `false` for layer normalization and `true` for RMS normalization. RMS normalization scales by the root mean square without subtracting the mean. Consequently, the positive sample row stays positive in the variation, rather than producing negative and positive values around zero. Bias is ignored for RMS normalization; scale still applies.

Both operations offer fast and deterministic modes. The default is fast. Deterministic mode fixes the accumulation order as described in the kernel docs; choosing it does not change RMS normalization into layer normalization or remove the need for a numerical error check. Compare against the formula for the operation you actually requested.

Float inputs are the simplest place to start. Half tensors have additional packing requirements: normalization rows must have even lengths and even element offsets. Buffer ranges must cover all rows and each optional scale or bias must cover a row’s columns. The public API checks shapes and ranges, but cannot decide whether your chosen axis matches the model you intended. Write down the row layout before flattening a tensor into buffers.
