---
title: "Add a bias and apply ReLU on the GPU in C#"
url: https://recursiveai.org/onegpu/cookbook/broadcast-bias-and-activation-on-gpu/
description: "Add a bias vector to each matrix row and apply ReLU in one OneGpu compute pass, with GPU output and a CPU reference."
version: "1.0.0"
---

# Add a bias and apply ReLU on the GPU in C#

**OneGpu** is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneGpu.Kernels) · [GitHub](https://github.com/recursive-ai-oss/onegpu-csharp) · [Docs](https://recursiveai.org/onegpu/docs/) · [Download example (.zip)](https://recursiveai.org/onegpu/cookbook/broadcast-bias-and-activation-on-gpu/broadcast-bias-and-activation-on-gpu.zip)

Broadcast a `[3]` bias over a `[2, 3]` matrix and apply ReLU in one fused compute pass. This example uses Vulkan 1.3.

```shell
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
float[] matrix = [-3, 1, 2, 4, -2, 0];
float[] bias = [1, -2, 3];
using var x = gpu.CreateBuffer<float>(matrix);
using var b = gpu.CreateBuffer<float>(bias);
using var dst = gpu.CreateBuffer<float>(6);
using var ops = new ElementwiseKernels(gpu);
var commands = gpu.BeginCommands();
ops.EncodeBinary(commands, GpuBinaryOp.Add,
    x, 0, [2, 3], b, 0, [3], dst, 0,
    new GpuActivation(GpuUnaryOp.Relu));
gpu.Submit(commands);
var values = new float[6];
await dst.ReadAsync(values);
Console.WriteLine(string.Join(", ", values));
```

Unzip [the example](https://recursiveai.org/onegpu/cookbook/broadcast-bias-and-activation-on-gpu/broadcast-bias-and-activation-on-gpu.zip), then `dotnet run`.

Prints:

```text
0, 0, 5, 5, 0, 3
```

## The full program

The full program calculates the same result on the CPU and checks all six values. If the shapes don’t broadcast or a buffer is too small, `GpuKernelShapeException` names the bad argument.

On Vulkan, `Adapter = "discrete"` picks the first adapter of that kind or with “discrete” in its name and throws `GpuUnsupportedException` if none matches; for integrated graphics, use `Adapter = "integrated"` or omit `Adapter` to take the default, but neither will help if there’s no usable Vulkan 1.3 adapter.

```csharp
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
float[] matrix = [-3, 1, 2, 4, -2, 0];
float[] bias = [1, -2, 3];
using var x = gpu.CreateBuffer<float>(matrix);
using var b = gpu.CreateBuffer<float>(bias);
using var dst = gpu.CreateBuffer<float>(6);
using var ops = new ElementwiseKernels(gpu);
var commands = gpu.BeginCommands();
ops.EncodeBinary(commands, GpuBinaryOp.Add,
    x, 0, [2, 3], b, 0, [3], dst, 0,
    new GpuActivation(GpuUnaryOp.Relu));
gpu.Submit(commands);
var values = new float[6];
await dst.ReadAsync(values);
var expected = new float[matrix.Length];
for (var row = 0; row < 2; row++)
    for (var col = 0; col < 3; col++)
    {
        var index = row * 3 + col;
        expected[index] = MathF.Max(0, matrix[index] + bias[col]);
    }
if (!values.AsSpan().SequenceEqual(expected))
    throw new InvalidOperationException("GPU and CPU results differ.");
Console.WriteLine("GPU: " + string.Join(", ", values));
Console.WriteLine("CPU: " + string.Join(", ", expected));
Console.WriteLine($"{gpu.Backend}: {gpu.Adapter.Name}");
```

Output:

```text
GPU: 0, 0, 5, 5, 0, 3
CPU: 0, 0, 5, 5, 0, 3
Vulkan: NVIDIA GeForce RTX 4070
```

## Variations

### Keep a fraction of each negative sum

Replace the activation argument in the snippet at the top with:

```csharp
new GpuActivation(GpuUnaryOp.LeakyRelu, P0: 0.25f)
```

Output:

```text
-0.5, -0.25, 5, 5, -1, 3
```

## Related recipes

- [Multiply matrices on the GPU](https://recursiveai.org/onegpu/cookbook/multiply-matrices-on-the-gpu/)
- [Compute softmax on the GPU](https://recursiveai.org/onegpu/cookbook/compute-softmax-on-gpu/)

## The short answer

Call `ElementwiseKernels.EncodeBinary` with `GpuBinaryOp.Add`, the matrix shape `[2, 3]`, the bias shape `[3]`, and `new GpuActivation(GpuUnaryOp.Relu)`. OneGpu reads the same three bias values for both rows and applies ReLU to each sum before writing the result. You need only three elements for the bias buffer, not six.

## How does tensor broadcasting work for a bias vector?

OneGpu aligns the shapes from the right: in this example `[3]` behaves like `[1, 3]`. The last dimensions match, and the leading size of one lets the same bias row apply to both matrix rows. The result still has shape `[2, 3]`. The first input row `[-3, 1, 2]` becomes `[-2, -1, 5]` after adding `[1, -2, 3]`, then `[0, 0, 5]` after ReLU. The second row starts with `4`, so its first result is `5`, using the same first bias value `1`.

Shape matters even if the number of elements looks right. A `[2]` vector doesn’t match `[2, 3]` because its trailing dimension would line up with the three columns. To use one scalar per row, describe two values as `[2, 1]` instead. The [kernel docs](https://recursiveai.org/onegpu/docs/kernels/) describe the ONNX broadcasting rules the binary operations use.

## What is kernel fusion?

Here it means the add and the activation run in one GPU compute pass. The call writes the activated sums, not an intermediate tensor containing the plain sums. Leave out the optional activation argument if you need the plain sums. In the Leaky ReLU variation, `P0` sets the scale for negative sums: `-2` becomes `-0.5` at a scale of `0.25`. Fusion here applies to the binary operation and its unary activation; it doesn’t fold an earlier matrix multiplication into the pass.
