GPU matrix multiplication in C#

Multiply two row-major matrices on the GPU and copy the product into a C# array. This example uses Vulkan on Windows or Linux, with a Vulkan 1.3 driver.

On Apple silicon Macs, GpuBackends selects Metal automatically; the Gemm API is unchanged.

dotnet new console
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var device = GpuBackends.CreateDevice();
using var a = device.CreateBuffer<float>([1, 2, 3, 4]);
using var b = device.CreateBuffer<float>([5, 6, 7, 8]);
using var c = device.CreateBuffer<float>(4);
using var gemm = new Gemm(device, GemmElementType.F32);
var commands = device.BeginCommands();
gemm.Encode(commands, new GemmProblem(2, 2, 2), a, b, c);
device.Submit(commands);
var result = new float[4];
await c.ReadAsync(result);
Console.WriteLine(string.Join(", ", result));

Prints the row-major product:

19, 22, 43, 50

The inputs are A = [[1, 2], [3, 4]] and B = [[5, 6], [7, 8]]. An invalid shape or range throws GpuKernelShapeException before recording GPU commands.

Unzip the example, then dotnet run.

Variations

For repeatable sums on the same GPU and driver, replace the Gemm construction:

using var gemm = new Gemm(device, GemmElementType.F32,
    GpuKernelMode.Deterministic);

19, 22, 43, 50

To multiply A-transpose by B, replace the problem passed to Encode:

gemm.Encode(commands,
    new GemmProblem(2, 2, 2) { TransposeA = true }, a, b, c);

26, 30, 38, 44

For two products sharing A and B, replace C allocation, the encode call and the result allocation respectively:

using var c = device.CreateBuffer<float>(8);
// After creating commands:
gemm.Encode(commands, new GemmProblem(2, 2, 2)
{ Batch = 2, StrideA = 0, StrideB = 0, StrideC = 4 }, a, b, c);
// After submission:
var result = new float[8];

The existing readback and print then produce:

19, 22, 43, 50, 19, 22, 43, 50

Related recipes

• Run a compute shader in C#
• Read a GPU buffer back to the CPU in C#
• Choose a Vulkan physical device in C#

The short answer

Use OneGpu’s Gemm kernel family with float32 buffers for A, B and C. Describe the product with GemmProblem(M, N, K), record it into a command encoder, submit, and await C.ReadAsync. You don’t write a GPU shader yourself; the released kernels package supplies it.

The example computes a two-by-two product and prints the flattened result in row-major order. The same API accepts rectangular shapes; M is the number of output rows, N the number of output columns, and K the shared inner dimension.

What is the GPU matrix multiplication formula?

The GEMM documentation defines the operation as C = alpha * op(A) * op(B) + beta * C. op is either the stored matrix or its transpose. With the constructor defaults, alpha is one, beta is zero and the batch contains one product. The old contents of C are not read when beta is zero.

For the sample, A’s rows are [1, 2] and [3, 4]; B’s rows are [5, 6] and [7, 8]. The first output is 1 * 5 + 2 * 7 = 19, and the next is 1 * 6 + 2 * 8 = 22. The second row is forty-three and fifty. Reading the flat output as a column-major array would display the wrong matrix even though the product itself is correct.

Dimensions, leading dimensions, offsets and batch strides count elements, not bytes. A leading dimension is the distance between stored rows. Zero selects a dense leading dimension. Transpose flags change how the stored operands are interpreted, without requiring a separate CPU transpose; the variation produces A-transpose times B from the same stored input arrays.

The output is always float32. Inputs can also be float16 or bfloat16 through the corresponding GemmElementType; that is a change to both the selected element type and the operand storage, not just a cast of the output. Keep F32 for the float arrays used here.

What is a GPU matrix multiplication example in C#?

The runnable program above covers allocation, upload, dispatch and readback rather than just the call to Encode. Creation from an input span queues its upload before submission. Encode records work; Submit starts the submitted command stream. Only after the awaited readback is the managed result ready to inspect.

Create a Gemm instance once per element type and mode and reuse it for repeated products. It keeps its pipelines. For batching, set Batch and separate outputs with StrideC. A zero input stride shares that operand across the batch. The variation repeats the same product twice by sharing both A and B while reserving separate four-element output regions.

The CPU loop below checks this small integer-valued example exactly. Do not assume all floating-point products are bit-identical to a CPU loop: accumulation order and rounding matter. GpuKernelMode.Deterministic fixes the reduction and accumulation order for repeated runs on the same GPU and driver; the docs explicitly do not promise bit-identical results across vendors or drivers.

Shapes and buffer ranges are checked before recording. A bad shape, stride or offset throws GpuKernelShapeException, while a buffer exceeding the device’s storage-binding limit throws GpuUnsupportedException. Each operand must fit one storage-buffer binding.

After the example’s readback, replace its print statement with this CPU comparison:

float[] inputA = [1, 2, 3, 4], inputB = [5, 6, 7, 8];
var expected = new float[4];
for (int row = 0; row < 2; row++)
    for (int col = 0; col < 2; col++)
        for (int k = 0; k < 2; k++)
            expected[row * 2 + col] +=
                inputA[row * 2 + k] * inputB[k * 2 + col];
Console.WriteLine(result.SequenceEqual(expected));

True