Recursive AI

Run an LSTM layer on the GPU in C#

OneGpu is a free, open-source GPU compute and rendering library for .NET from Recursive AI, on Vulkan, Metal and WebGPU. MIT or Apache-2.0, no bundled native binaries.

NuGet GitHub Docs Download example (.zip)

An LSTM carries hidden and cell state through a sequence. This example runs one layer with supplied weights on Vulkan; Linux and Windows need a Vulkan 1.3 driver.

Terminal
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;
 
using var device = GpuBackends.CreateDevice(
    new GpuDeviceOptions { Adapter = "discrete" });
using var input = device.CreateBuffer<float>([1, 2]);
// ONNX LSTM gate order: input, output, forget, cell.
using var weights = device.CreateBuffer<float>([0, 0, 0, 1]);
using var recurrent = device.CreateBuffer<float>([0, 0, 0, 0]);
using var initialH = device.CreateBuffer<float>([0]);
using var initialC = device.CreateBuffer<float>([0]);
using var y = device.CreateBuffer<float>(2);
using var yh = device.CreateBuffer<float>(1);
using var yc = device.CreateBuffer<float>(1);
using var lstm = new RecurrentKernels(device, GpuKernelMode.Deterministic);
var problem = new RecurrentProblem(GpuRecurrentCell.Lstm, seq: 2, batch: 1, input: 1, hidden: 1);
var operands = new RecurrentOperands<float>
{
    X = new(input), W = new(weights), R = new(recurrent),
    InitialH = new(initialH), InitialC = new(initialC),
    Y = new(y), YH = new(yh), YC = new(yc)
};
var commands = device.BeginCommands();
lstm.Encode(commands, problem, operands);
device.Submit(commands);
var sequence = new float[2];
var hidden = new float[1];
var cell = new float[1];
await y.ReadAsync(sequence);
await yh.ReadAsync(hidden);
await yc.ReadAsync(cell);
Console.WriteLine($"Y: {string.Join(", ", sequence.Select(x => x.ToString("F6")))}");
Console.WriteLine($"YH: {hidden[0]:F6}; YC: {cell[0]:F6}");

Unzip the example, then dotnet run.

Prints:

Text
Y: 0.181700, 0.293282
YH: 0.293282; YC: 0.672412

The full program

The snippet above, plus the CPU check and device name below, is the complete Program.cs in the zip:

C#
// With zero gate weights/biases, i = o = f = sigmoid(0) = 0.5.
double cpuCell = 0;
var expected = new double[2];
for (int step = 0; step < 2; step++)
{
    cpuCell = 0.5 * cpuCell + 0.5 * Math.Tanh(step + 1);
    expected[step] = 0.5 * Math.Tanh(cpuCell);
}
bool matches = sequence.Zip(expected).All(p => Math.Abs(p.First - p.Second) < 1e-6)
    && Math.Abs(hidden[0] - expected[1]) < 1e-6
    && Math.Abs(cell[0] - cpuCell) < 1e-6;
Console.WriteLine($"CPU match: {matches}");
Console.WriteLine($"{device.Backend}: {device.Adapter.Name}");

The sequence has two steps, batch 1, input size 1 and hidden size 1. X is [1, 2]. W is [0, 0, 0, 1] in ONNX order: input, output, forget, cell. R is all zeros; initial hidden and cell states are explicitly zero. Biases and peepholes are omitted.

The default direction is forward and Layout1 is false (ONNX layout 0). Y has shape [sequence, directions, batch, hidden], or [2, 1, 1, 1]; YH and YC each have shape [1, 1, 1]. The full program compares all three outputs with a CPU recurrence. It ran on an NVIDIA GeForce RTX 4070 through Vulkan.

Invalid shapes, offsets or ranges throw GpuKernelShapeException; scratch exceeding a storage-binding limit throws GpuUnsupportedException.

Text
Y: 0.181700, 0.293282
YH: 0.293282; YC: 0.672412
CPU match: True
Vulkan: NVIDIA GeForce RTX 4070

Without an adapter override, Adapter = "discrete" throws GpuUnsupportedException on an integrated-only machine; macOS defaults to Metal (Apple arm64 only), so omit the adapter filter or use "integrated" on Apple silicon.

Variations

Start with a nonzero cell state

Replace the initial cell allocation; the hidden state remains zero:

C#
using var initialC = device.CreateBuffer<float>([1]);
Text
Y: 0.353409, 0.363518
YH: 0.363518; YC: 0.922412

The short answer

Create a RecurrentProblem for GpuRecurrentCell.Lstm, supply RecurrentOperands<float>, encode it with RecurrentKernels and submit. Request Y for the sequence and YH/YC for the final hidden and cell states. This runs a layer with weights you already have; it does not train a network or load a model file. The recurrent-layer docs specify the tensor and state semantics.

The example’s weights make the recurrence inspectable. The input, output and forget gates all receive zero, so each sigmoid gate is one-half. Only the cell candidate depends on X. The first step already changes the cell state, and the second step depends on that state even though the recurrent weight matrix is zero.

How can I run an LSTM on the GPU?

Select a device with a working driver, allocate X, W and R, then create output buffers matching the problem’s dimensions. Initial states are optional, but supplying them explicitly makes sequence boundaries visible. The snippet at the top includes every array it needs, including the ONNX-ordered weights. No trained model download is required.

Encoding alone does not complete the calculation. Submit the command encoder, then await each requested output’s readback. Keep inputs, outputs and the kernel instance alive until the work has finished. Reuse the kernel instance for repeated layers rather than recreating its pipelines for every call.

The family projects input steps with a matrix product, initializes state, and records recurrent steps with ordering between passes. A single matrix multiplication is therefore not a replacement for this operation: it lacks the state updates and gates. You need the sequence and state dimensions even when batch and hidden size are one.

What weights and states does an LSTM layer need?

W is stored as [directions, 4 * hidden, input] and R as [directions, 4 * hidden, hidden]. The gate order is i, o, f, c, not an arbitrary four-way split. The API uses the stored arrays without repacking. When importing weights, check the originating layer’s gate order before uploading them. Each operand is a buffer plus an element offset through GpuTensorRef<T>.

With the example’s settings, the CPU recurrence is c = 0.5 * c + 0.5 * tanh(x) and h = 0.5 * tanh(c). The full program evaluates it in double precision and compares within 1e-6 absolute error. The six printed decimal places are formatting, not a precision guarantee. General weights use recurrent contributions and biases too; this simplified equation applies only to the supplied arrays.

Y contains each step’s hidden output. YH is the last hidden state, and YC is the last cell state; the latter is not the last Y value. To continue a forward sequence in another call, carry both states, preserving the expected layout. The variation changes only InitialC and consequently changes both steps and final outputs.

A null output is not written. For variable-length batches, SequenceLengths uses device int32 values; validate those lengths on the host. The documented zero-length-row behavior sets final hidden and cell outputs to zero. An entirely empty sequence instead leaves final outputs as the initial states or zeros. Those cases are different from an ordinary sequence whose numeric inputs happen to be zero.

All onegpu recipes