Recursive AI

GEMM explained

GEMM is general matrix multiplication: it scales a matrix product and adds a scaled copy of the old output. Either input can be transposed.

OneGpu is an open-source GPU compute library for .NET from Recursive AI, licensed MIT or Apache-2.0. NuGet: Backends and Kernels · GitHub · Docs

Terminal.NET 10
dotnet add package RecursiveAI.OneGpu.Backends --version 1.0.0
dotnet add package RecursiveAI.OneGpu.Kernels --version 1.0.0

See the checked GEMM example and its output.

What GEMM computes

Read C = α × op(A) × op(B) + β × C as an update: the C on the right is the old matrix, and C on the left receives the new values. op(X) is either X or its transpose. OneGpu’s GEMM contract uses row-major storage (each row is stored together, one row after the next) and float32 accumulation and output.

After applying the transpose flags, op(A) is M × K and op(B) is K × N. The result is M × N. Each new element starts with a dot product between a row of op(A) and a column of op(B), with K terms. Alpha scales that dot product; beta scales the old element at the same position.

A new GemmProblem(m, n, k) defaults to alpha = 1 and beta = 0, which gives plain matrix multiplication. With beta = 0, OneGpu doesn’t read the old values in C. Set beta = 1 to add the scaled product to C, or set another nonzero value to keep a fraction or multiple of the old C. Initialize C before using a nonzero beta.

Transpose changes indexing, not storage

With TransposeB = true, you supply B stored as N × K. GEMM reads each stored row as a column of op(B); you don’t need to rearrange the buffer. Likewise, TransposeA = true means A is stored as K × M.

The leading dimensions Lda, Ldb, and Ldc give the distance from one stored row’s start to the next, in elements, not bytes. A value of zero selects the tightly packed row width. For transposed B that default is K, not N. A padded row still holds the same number of matrix values; only the next row’s starting index changes.

A checked example with padded rows

OneGpu runs on Vulkan 1.3 on Linux and Windows, Metal on Apple silicon Macs, and WebGPU in Chrome; on Linux and Windows, the docs also list Mesa’s software Vulkan driver as an option without a suitable hardware GPU.

Here M = 2, N = 2, and K = 3. A holds the rows (1, 2, 3) and (4, 5, 6). B stores two rows of three values with a fourth slot of padding (999). Thus Ldb = 4, and the second row starts at index 4. C stores two values per row with a stride of three (Ldc = 3); its padding starts at -1 so the program can check that it wasn’t overwritten.

For the top-left result, the dot product is 1 × 1 + 2 × 0 + 3 × 2 = 7. Alpha is 2 and beta is 0.5, and the old top-left value is 10, so the new value is 2 × 7 + 0.5 × 10 = 19.

In a .NET 10 console project, use version 1.0.0 of RecursiveAI.OneGpu.Backends and RecursiveAI.OneGpu.Kernels. This run used Vulkan on Linux with a GeForce RTX 4070. The program checks all four results and both output padding slots before printing them.

Terminal.NET 10
dotnet new console
dotnet add package RecursiveAI.OneGpu.Backends --version 1.0.0
dotnet add package RecursiveAI.OneGpu.Kernels --version 1.0.0
Program.cs.NET 10
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var device = GpuBackends.CreateDevice();
Console.WriteLine($"{GpuBackends.Selected}: {device.Adapter.Name}");
 
float[] a = [1, 2, 3, 4, 5, 6];
float[] b = [1, 0, 2, 999, 0, 1, 3, 999];
float[] c = [10, 20, -1, 30, 40, -1];
var problem = new GemmProblem(m: 2, n: 2, k: 3)
{
    TransposeB = true, Ldb = 4, Ldc = 3,
    Alpha = 2, Beta = 0.5f
};
 
using var bufferA = device.CreateBuffer<float>(a);
using var bufferB = device.CreateBuffer<float>(b);
using var bufferC = device.CreateBuffer<float>(c);
using var gemm = new Gemm(device, GemmElementType.F32);
var encoder = device.BeginCommands();
gemm.Encode(encoder, problem, bufferA, bufferB, bufferC);
device.Submit(encoder);
await bufferC.ReadAsync(c);
 
float[] expected = [19, 32, -1, 47, 66, -1];
if (!c.AsSpan().SequenceEqual(expected))
    throw new InvalidOperationException("Unexpected GEMM result");
Console.WriteLine($"C: [{c[0]}, {c[1]}]; [{c[3]}, {c[4]}]");
Console.WriteLine($"Padding: {c[2]}, {c[5]}");
OutputOneGpu 1.0.0
Vulkan: NVIDIA GeForce RTX 4070
C: [19, 32]; [47, 66]
Padding: -1, -1

What this check tells you

The exact check is appropriate here because the small integer products, sums, and half scaling are all exactly representable in float32. For general floating-point inputs, use a reference and a tolerance suited to your data. OneGpu’s deterministic mode promises repeatability on one device and driver, not identical last bits across devices and drivers.

For supported element types, batching, and kernel options, see the OneGpu GEMM reference. For package setup, see getting started with OneGpu.