Recursive AI

Reuse GPU bindings across compute dispatches in C#

OneGpu is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

A bind group is the set of buffer references a dispatch uses. Caching works on Vulkan 1.3 (discrete or integrated), Metal, and browser WebGPU.

Terminal
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
using var ops = new ElementwiseKernels(gpu);
using var input =
    gpu.CreateBuffer<float>([-2, 5]);
using var output = gpu.CreateBuffer<float>(1);
for (int offset = 0; offset < 2; offset++)
{
    var cmd = gpu.BeginCommands();
    ops.EncodeUnary(cmd, GpuUnaryOp.Relu,
        input, offset, output, 0, 1);
    gpu.Submit(cmd);
    Console.WriteLine(ops.BindGroupsCreated);
}

Unzip the example, then dotnet run.

Prints:

Text
1
1

The full program

The zip also reads back the output after each submission. The first pass reads -2 and writes 0; the second reads 5 from the next element and writes 5. The count staying at 1 means the second pass didn’t build another bind group.

An element range that extends past its buffer throws GpuKernelShapeException when you record the pass.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
using var ops = new ElementwiseKernels(gpu);
using var input = gpu.CreateBuffer<float>([-2, 5]);
using var output = gpu.CreateBuffer<float>(1);
var result = new float[1];
for (int offset = 0; offset < 2; offset++)
{
    var cmd = gpu.BeginCommands();
    ops.EncodeUnary(cmd, GpuUnaryOp.Relu,
        input, offset, output, 0, 1);
    gpu.Submit(cmd);
    await output.ReadAsync(result);
    Console.WriteLine($"Offset {offset}: {result[0]}; bind groups built: {ops.BindGroupsCreated}");
}
Console.WriteLine($"{gpu.Backend}: {gpu.Adapter.Name}");

Output:

Text
Offset 0: 0; bind groups built: 1
Offset 1: 5; bind groups built: 1
Vulkan: NVIDIA GeForce RTX 4070

Variations

Clear the cache before the second dispatch

Add this to the top of the loop in the snippet at the top. The next dispatch rebuilds the group without resetting the counter.

C#
if (offset == 1)
    ops.ReleaseBindings();

Prints:

Text
1
2

The short answer

Create ElementwiseKernels once and pass the same input and output buffers to each EncodeUnary call. Compare BindGroupsCreated after each call: the first builds the group, and the second should leave the count unchanged. You can change the element offset without changing the bindings.

What is a descriptor set in Vulkan?

A Vulkan descriptor set groups references to resources a shader uses. OneGpu calls its version of that grouping a bind group. The group points to the input and output buffers; it doesn’t contain the values the shader reads from those buffers. Reusing the bindings therefore doesn’t mean reusing the first calculation’s result.

OneGpu passes the offset and count separately in push constants, small values recorded with the dispatch. That’s why going from offset 0 to offset 1 still uses the original group. If you allocate a new input buffer with the same two numbers, that’s a different buffer, so the old group doesn’t match it. Keep the same kernel family (the ElementwiseKernels instance) and pipeline variant (the compiled shader for that configuration, such as float rather than Half): variants can have separate caches.

When do you need to rebuild a bind group?

OneGpu handles the rebuild. The first call with a new set of buffers creates a group. If the cache exceeds its maxCachedBindGroups limit, it drops the saved groups and starts over. Set the limit when you create the family: using var ops = new ElementwiseKernels(gpu, maxCachedBindGroups: 64);. The kernel docs give the default as 1,024. Choose a bound that covers the distinct buffer sets you expect to reuse, not the number of dispatches you record. A loop that dispatches 10,000 times with one set still needs only one group for that variant.

When you retire or replace buffers, call ReleaseBindings() so the cached entries don’t keep the old buffers reachable. That drops the groups but keeps the pipelines. The counter is cumulative: a count of 2 after the variation means two groups have been built over the family’s lifetime, not that it has two groups in its cache.

All onegpu recipes