Recursive AI

Select GPU tensor values with a boolean mask in C#

OneGpu is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

A boolean mask chooses which value to keep at each position. This example uses Vulkan 1.3.

Terminal
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
int[] shape = [5];
using var src = gpu.CreateBuffer<float>(
    [-2, 1, 0, 3, -4]);
using var zero = gpu.CreateBuffer<float>([0]);
using var mask = gpu.CreateBuffer<byte>(
    GpuKernelBuffers.PaddedBoolLength(5));
using var dst = gpu.CreateBuffer<float>(5);
using var ops = new ElementwiseKernels(gpu);
var commands = gpu.BeginCommands();
ops.EncodeCompare(commands,
    GpuCompareOp.Greater,
    src, 0, shape, zero, 0, [], mask, 0);
ops.EncodeWhere(commands, mask, 0, shape,
    src, 0, shape, zero, 0, [], dst, 0);
gpu.Submit(commands);
var values = new float[5];
await dst.ReadAsync(values);
Console.WriteLine(string.Join(", ", values));

Each input operand is passed as a buffer, an element offset, and a shape; [] means a scalar.

Unzip the example, then dotnet run.

Prints:

Text
0, 1, 0, 3, 0

The full program

The five input values are [-2, 1, 0, 3, -4]. The comparison writes five flags into an eight-byte buffer; the full program prints only the five logical flags. A mask buffer that ends partway through a four-byte word throws GpuKernelShapeException.

On Vulkan, Adapter = "discrete" picks the first adapter of that kind or with “discrete” in its name and throws GpuUnsupportedException if none matches; for integrated graphics, use Adapter = "integrated" or omit Adapter to take the default, but neither will help if there’s no usable Vulkan 1.3 adapter.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
int[] shape = [5];
using var src = gpu.CreateBuffer<float>(
    [-2, 1, 0, 3, -4]);
using var zero = gpu.CreateBuffer<float>([0]);
using var mask = gpu.CreateBuffer<byte>(
    GpuKernelBuffers.PaddedBoolLength(5));
using var dst = gpu.CreateBuffer<float>(5);
using var ops = new ElementwiseKernels(gpu);
var commands = gpu.BeginCommands();
ops.EncodeCompare(commands, GpuCompareOp.Greater,
    src, 0, shape, zero, 0, [], mask, 0);
ops.EncodeWhere(commands, mask, 0, shape,
    src, 0, shape, zero, 0, [], dst, 0);
gpu.Submit(commands);
var values = new float[5];
await dst.ReadAsync(values);
var flags = new byte[GpuKernelBuffers.PaddedBoolLength(5)];
await mask.ReadAsync(flags);
var logicalMask = flags.AsSpan(0, 5);
byte[] expectedMask = [0, 1, 0, 1, 0];
if (!logicalMask.SequenceEqual(expectedMask) ||
    !values.AsSpan().SequenceEqual([0, 1, 0, 3, 0]))
    throw new InvalidOperationException("Unexpected mask or selected values.");
Console.WriteLine("Mask: " + string.Join(", ", logicalMask.ToArray()));
Console.WriteLine("Where: " + string.Join(", ", values));
Console.WriteLine($"{gpu.Backend}: {gpu.Adapter.Name}");

Output:

Text
Mask: 0, 1, 0, 1, 0
Where: 0, 1, 0, 3, 0
Vulkan: NVIDIA GeForce RTX 4070

Variations

Keep the negative values instead

Replace the comparison call in the snippet at the top:

C#
ops.EncodeCompare(commands, GpuCompareOp.Less,
    src, 0, shape, zero, 0, [], mask, 0);

Output:

Text
-2, 0, 0, 0, -4

The short answer

Record ElementwiseKernels.EncodeCompare to build a mask and EncodeWhere to choose between two tensors. This example compares each element with zero. Where picks the original value if it is positive and the scalar zero otherwise. The empty shape [] describes a scalar; it doesn’t mean an empty tensor.

What is a boolean mask?

A mask is a tensor of true or false flags. Here, comparing [-2, 1, 0, 3, -4] with zero produces [false, true, false, true, false]. OneGpu stores each flag in a GpuBuffer<byte>, with 1 for true and 0 for false in a comparison result. Where treats any nonzero condition byte as true.

Keep the buffer length separate from the tensor shape. GpuKernelBuffers.PaddedBoolLength(5) returns eight because the buffer must end on a four-byte boundary. The shape passed to both kernels stays [5]; the padding doesn’t add conditions. Read the padded buffer back if you want to inspect it, but interpret only the five logical bytes. The bool tensor docs explain why a buffer ending inside a word is rejected.

Does Where filter out masked values or keep the tensor shape?

Where picks one value at each position in the broadcast result. It doesn’t compact the matches into a shorter array. In the example there are two true flags, but the output still has five values because the false positions each receive zero. Both value inputs can be tensors; the scalar zero is just a convenient default. OneGpu’s Where supports float, Half, int, uint and long values. The comparison and selection can use the same command encoder, with the mask left on the GPU until selection is done.

All onegpu recipes