Recursive AI

Add a bias and apply ReLU on the GPU in C#

OneGpu is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

Broadcast a [3] bias over a [2, 3] matrix and apply ReLU in one fused compute pass. This example uses Vulkan 1.3.

Terminal
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
float[] matrix = [-3, 1, 2, 4, -2, 0];
float[] bias = [1, -2, 3];
using var x = gpu.CreateBuffer<float>(matrix);
using var b = gpu.CreateBuffer<float>(bias);
using var dst = gpu.CreateBuffer<float>(6);
using var ops = new ElementwiseKernels(gpu);
var commands = gpu.BeginCommands();
ops.EncodeBinary(commands, GpuBinaryOp.Add,
    x, 0, [2, 3], b, 0, [3], dst, 0,
    new GpuActivation(GpuUnaryOp.Relu));
gpu.Submit(commands);
var values = new float[6];
await dst.ReadAsync(values);
Console.WriteLine(string.Join(", ", values));

Unzip the example, then dotnet run.

Prints:

Text
0, 0, 5, 5, 0, 3

The full program

The full program calculates the same result on the CPU and checks all six values. If the shapes don’t broadcast or a buffer is too small, GpuKernelShapeException names the bad argument.

On Vulkan, Adapter = "discrete" picks the first adapter of that kind or with “discrete” in its name and throws GpuUnsupportedException if none matches; for integrated graphics, use Adapter = "integrated" or omit Adapter to take the default, but neither will help if there’s no usable Vulkan 1.3 adapter.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
float[] matrix = [-3, 1, 2, 4, -2, 0];
float[] bias = [1, -2, 3];
using var x = gpu.CreateBuffer<float>(matrix);
using var b = gpu.CreateBuffer<float>(bias);
using var dst = gpu.CreateBuffer<float>(6);
using var ops = new ElementwiseKernels(gpu);
var commands = gpu.BeginCommands();
ops.EncodeBinary(commands, GpuBinaryOp.Add,
    x, 0, [2, 3], b, 0, [3], dst, 0,
    new GpuActivation(GpuUnaryOp.Relu));
gpu.Submit(commands);
var values = new float[6];
await dst.ReadAsync(values);
var expected = new float[matrix.Length];
for (var row = 0; row < 2; row++)
    for (var col = 0; col < 3; col++)
    {
        var index = row * 3 + col;
        expected[index] = MathF.Max(0, matrix[index] + bias[col]);
    }
if (!values.AsSpan().SequenceEqual(expected))
    throw new InvalidOperationException("GPU and CPU results differ.");
Console.WriteLine("GPU: " + string.Join(", ", values));
Console.WriteLine("CPU: " + string.Join(", ", expected));
Console.WriteLine($"{gpu.Backend}: {gpu.Adapter.Name}");

Output:

Text
GPU: 0, 0, 5, 5, 0, 3
CPU: 0, 0, 5, 5, 0, 3
Vulkan: NVIDIA GeForce RTX 4070

Variations

Keep a fraction of each negative sum

Replace the activation argument in the snippet at the top with:

C#
new GpuActivation(GpuUnaryOp.LeakyRelu, P0: 0.25f)

Output:

Text
-0.5, -0.25, 5, 5, -1, 3

The short answer

Call ElementwiseKernels.EncodeBinary with GpuBinaryOp.Add, the matrix shape [2, 3], the bias shape [3], and new GpuActivation(GpuUnaryOp.Relu). OneGpu reads the same three bias values for both rows and applies ReLU to each sum before writing the result. You need only three elements for the bias buffer, not six.

How does tensor broadcasting work for a bias vector?

OneGpu aligns the shapes from the right: in this example [3] behaves like [1, 3]. The last dimensions match, and the leading size of one lets the same bias row apply to both matrix rows. The result still has shape [2, 3]. The first input row [-3, 1, 2] becomes [-2, -1, 5] after adding [1, -2, 3], then [0, 0, 5] after ReLU. The second row starts with 4, so its first result is 5, using the same first bias value 1.

Shape matters even if the number of elements looks right. A [2] vector doesn’t match [2, 3] because its trailing dimension would line up with the three columns. To use one scalar per row, describe two values as [2, 1] instead. The kernel docs describe the ONNX broadcasting rules the binary operations use.

What is kernel fusion?

Here it means the add and the activation run in one GPU compute pass. The call writes the activated sums, not an intermediate tensor containing the plain sums. Leave out the optional activation argument if you need the plain sums. In the Leaky ReLU variation, P0 sets the scale for negative sums: -2 becomes -0.5 at a scale of 0.25. Fusion here applies to the binary operation and its unary activation; it doesn’t fold an earlier matrix multiplication into the pass.

All onegpu recipes