Recursive AI

Run NCHW convolution on the GPU in C#

OneGpu is a free, open-source GPU compute and rendering library for .NET from Recursive AI, on Vulkan, Metal and WebGPU. MIT or Apache-2.0, no bundled native binaries.

NuGet GitHub Docs Download example (.zip)

Apply a convolution filter to an NCHW tensor: batch, channels, height, width. This float32 example uses Vulkan; Linux and Windows need a Vulkan 1.3 driver.

Terminal
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var device = GpuBackends.CreateDevice(
    new GpuDeviceOptions { Adapter = "discrete" });
using var input = device.CreateBuffer<float>([1, 2, 3, 4, 5, 6, 7, 8, 9]);
using var weights = device.CreateBuffer<float>([1, 0, 0, -1]);
using var output = device.CreateBuffer<float>(4);
using var conv = new ConvKernels(device, GpuKernelMode.Deterministic);
var problem = new ConvProblem(batch: 1, inChannels: 1, height: 3, width: 3,
    outChannels: 1, kernelHeight: 2, kernelWidth: 2)
{
    StrideHeight = 1, StrideWidth = 1,
    PadTop = 0, PadLeft = 0, PadBottom = 0, PadRight = 0
};
var scratch = conv.Scratch(problem, half: false);
using var columns = device.CreateBuffer<float>(
    checked((int)Math.Max(1, scratch.ColumnElements)));
var commands = device.BeginCommands();
conv.Encode(commands, problem, input, weights, bias: null, residual: null, y: output,
    columns: columns);
device.Submit(commands);
var result = new float[4];
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result));

Unzip the example, then dotnet run.

Prints:

Text
-4, -4, -4, -4

The full program

The snippet above, plus the CPU check and device name below, is the complete Program.cs in the zip:

C#
// Hand calculation: top-left minus bottom-right in each 2 x 2 window.
float[] pixels = [1, 2, 3, 4, 5, 6, 7, 8, 9];
var expected = new float[4];
for (int row = 0; row < 2; row++)
    for (int col = 0; col < 2; col++)
        expected[row * 2 + col] = pixels[row * 3 + col] - pixels[(row + 1) * 3 + col + 1];
Console.WriteLine($"CPU match: {result.SequenceEqual(expected)}");
Console.WriteLine($"{device.Backend}: {device.Adapter.Name}");

The input is one channel of one image, with rows [1, 2, 3], [4, 5, 6] and [7, 8, 9]. The OIHW weights are one output channel, one input channel and a 2 × 2 filter with rows [1, 0], [0, −1]. Stride is one and padding is zero. The output shape is [1, 1, 2, 2].

Each output subtracts the bottom-right pixel from the top-left pixel of a window: 1 − 5, 2 − 6, 4 − 8, 5 − 9. The full program checks these values on the CPU. This output was recorded on an NVIDIA GeForce RTX 4070 through Vulkan.

Invalid shapes or overlapping scratch ranges throw GpuKernelShapeException before recording; unsupported device binding sizes throw GpuUnsupportedException.

Text
-4, -4, -4, -4
CPU match: True
Vulkan: NVIDIA GeForce RTX 4070

Without an adapter override, Adapter = "discrete" throws GpuUnsupportedException on an integrated-only machine; macOS defaults to Metal (Apple arm64 only), so omit the adapter filter or use "integrated" on Apple silicon.

Variations

Use stride two

Replace the problem’s strides, output allocation and readback array with these lines. Keep the other pads at zero:

C#
// In the ConvProblem initializer:
StrideHeight = 2, StrideWidth = 2,
// Replace the corresponding allocations:
using var output = device.CreateBuffer<float>(1);
var result = new float[1];
Text
-4

The short answer

Use ConvKernels with a ConvProblem describing NCHW input and OIHW weights. Ask the instance for scratch requirements on the selected device, allocate that range, encode, submit and await the output. You supply tensors and layer parameters; the family chooses its convolution implementation. See the convolution docs for supported element types and fused operations.

This example is a neural-network convolution over float arrays, not an image-file blur. There is no file decoder, color conversion or automatic channel rearrangement. The filter is applied in its stored order; it is not reversed before sliding across the input.

What do NCHW and OIHW mean?

NCHW means batch first, then input channel, height and width, with width contiguous. For a dense tensor, the flat index is ((n * C + c) * H + row) * W + col. An interleaved RGB image does not have that arrangement. Convert its channel layout before using it as an NCHW tensor.

OIHW weights use output channel, input channel within a group, kernel height and kernel width. The single-channel example removes channel summation from the hand calculation but still exercises sliding windows. With multiple channels, each output adds contributions from the relevant input channels. Groups changes which input channels connect to each output group; do not just enlarge the weight array without updating the problem.

For each spatial axis, output size is floor((input + padBegin + padEnd - effectiveKernel) / stride) + 1, where effectiveKernel = dilation * (kernel - 1) + 1. Here (3 - 2) / 1 + 1 gives two. Stride two gives one, which is why the variation also changes the output allocation and readback size. Changing stride alone leaves the result array’s interpretation wrong.

What are im2col and implicit GEMM in convolution?

Im2col unfolds input windows into a column matrix, then uses matrix multiplication. Implicit GEMM gathers windows while multiplying instead of materializing that entire matrix. OneGpu chooses among documented variants according to the problem, element type, mode and device capabilities. Neither term changes the NCHW/OIHW contract your application supplies.

Call conv.Scratch(problem, half: false) on the instance that will encode the operation. Do not copy scratch sizes from another GPU. Some variants need no column elements; the sample allocates at least one float to keep buffer creation valid even in that case. It passes its own buffer so ownership is explicit. Dispose it only after submitted work and readback complete.

Scratch sizes and offsets count elements, not bytes. A scratch range may share an arena allocation with operands only when the ranges do not overlap. Separate buffers, as used here, avoid that bookkeeping. If you omit caller scratch, the instance can retain scratch internally; that is convenient for repeated shapes but is a different allocation policy.

The family validates sizes and offsets before recording. That cannot identify an array you packed in the wrong logical order: a sufficiently large NHWC array still fits a buffer. Check both the numeric shape and the meaning of each dimension when results look plausible but incorrect.

All onegpu recipes