Recursive AI

Slice and concatenate tensors on the GPU in C#

OneGpu is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

Take every other value backward, then join the slice with another range. This example uses Vulkan 1.3.

Terminal
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
using var arena = gpu.CreateBuffer<int>(
    [10, 20, 30, 40, 0, 0, 0, 0]);
using var slice = gpu.CreateBuffer<int>(2);
using var layout = new LayoutKernels(gpu);
var commands = gpu.BeginCommands();
layout.EncodeSlice(encoder: commands,
    input: arena, offsetInput: 0,
    shape: [4], starts: [3], ends: [0],
    axes: [0], steps: [-2],
    output: slice, offsetOutput: 0);
layout.EncodeConcat<int>(commands,
    [new(slice, 0, 2), new(arena, 0, 2)],
    [4], 0, arena, 4);
gpu.Submit(commands);
var values = new int[4];
await arena.ReadAsync(values, 4);
Console.WriteLine(string.Join(", ", values));

Unzip the example, then dotnet run.

Prints:

Text
40, 20, 10, 20

The full program

The input is [10, 20, 30, 40]; the four trailing zeroes reserve room for the result in the same buffer. Slice needs a separate output buffer. Concat throws GpuKernelShapeException if an input range overlaps the destination in the same buffer.

On Vulkan, Adapter = "discrete" picks the first adapter of that kind or with “discrete” in its name and throws GpuUnsupportedException if none matches; for integrated graphics, use Adapter = "integrated" or omit Adapter to take the default, but neither will help if there’s no usable Vulkan 1.3 adapter.

C#
using System;
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var gpu = GpuBackends.CreateDevice(
    new() { Adapter = "discrete" });
using var arena = gpu.CreateBuffer<int>(
    [10, 20, 30, 40, 0, 0, 0, 0]);
using var slice = gpu.CreateBuffer<int>(2);
using var layout = new LayoutKernels(gpu);
var commands = gpu.BeginCommands();
layout.EncodeSlice(commands, arena, 0,
    [4], [3], [0], [0], [-2], slice, 0);
layout.EncodeConcat<int>(commands,
    [new(slice, 0, 2), new(arena, 0, 2)],
    [4], 0, arena, 4);
gpu.Submit(commands);
var values = new int[4];
await arena.ReadAsync(values, 4);
var reversed = new int[2];
await slice.ReadAsync(reversed);
if (!reversed.AsSpan().SequenceEqual([40, 20]) ||
    !values.AsSpan().SequenceEqual([40, 20, 10, 20]))
    throw new InvalidOperationException("Unexpected slice or concatenation.");
Console.WriteLine("Slice: " + string.Join(", ", reversed));
Console.WriteLine("Concat: " + string.Join(", ", values));
Console.WriteLine($"{gpu.Backend}: {gpu.Adapter.Name}");

Output:

Text
Slice: 40, 20
Concat: 40, 20, 10, 20
Vulkan: NVIDIA GeForce RTX 4070

Variations

Put the original range first

Replace the EncodeConcat call in the snippet at the top:

C#
layout.EncodeConcat<int>(commands,
    [new(arena, 0, 2), new(slice, 0, 2)],
    [4], 0, arena, 4);

Output:

Text
10, 20, 40, 20

The short answer

Use LayoutKernels.EncodeSlice to copy a region in reverse order, then EncodeConcat to join two ranges. Record them in that order in one command encoder. The second operation consumes the first operation’s GPU output, so you don’t need to read it back between the calls.

How do you slice a tensor with a negative step?

Pass the starting index, the exclusive ending index, the axis, and the negative step. The example starts at index 3 and steps by -2 toward index 0, so it reads indices 3 and 1, giving [40, 20]. The end isn’t included. A negative end counts from the far end of the axis; -1 is the last element, not a position before index zero. For a reverse slice that includes index zero, use an end before the negative extent, such as -5 on an axis of length four. SliceGeometry can compute the output shape before you allocate the slice buffer.

How do you concatenate tensors along an axis?

Each GpuConcatInput<T> names a buffer, an element offset and a length along the join axis. For the one-dimensional example, the axis lengths are 2 and 2, so the output shape is [4]. For a higher-rank tensor, the other dimensions must match; the output’s axis length is the sum of the inputs’ lengths on that axis. Concatenating joins an existing axis rather than adding a new one.

The example reads two elements at offset 0 from arena and puts the four-element result at offset 4. Those ranges don’t overlap. On Vulkan and Metal, Concat reads a shared buffer directly when its ranges are disjoint; on browser WebGPU it first copies the inputs that share the output buffer into scratch space. Don’t apply that rule to Slice: it rejects using the same underlying buffer for input and output even at disjoint offsets. See the pinned layout implementation for the overlap and scratch-copy rules.

All onegpu recipes