---
title: "Compute softmax on the GPU in C#"
url: https://recursiveai.org/onegpu/cookbook/compute-softmax-on-gpu/
description: "Compute softmax on the GPU in C#. Includes buffer setup, real Vulkan output and a runnable .NET 10 example."
version: "1.0.0"
---

# Compute softmax on the GPU in C#

**OneGpu** is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneGpu.Backends) · [GitHub](https://github.com/recursive-ai-oss/onegpu-csharp) · [Docs](https://recursiveai.org/onegpu/docs/) · [Download example (.zip)](https://recursiveai.org/onegpu/cookbook/compute-softmax-on-gpu/compute-softmax-on-gpu.zip)

Softmax turns scores into probabilities. This example requires Vulkan 1.3.

```shell
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var device = GpuBackends.CreateDevice(
    new GpuDeviceOptions { Adapter = "discrete" });
using var input = device.CreateBuffer<float>([1, 2, 3, 1, 2, 3]);
using var output = device.CreateBuffer<float>(6);
using var softmax = new SoftmaxKernels(device);
var commands = device.BeginCommands();
softmax.Encode(commands, input, 0, [2, 3], 1, output, 0,
    mode: GpuKernelMode.Deterministic);
device.Submit(commands);
var result = new float[6];
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result.Select(x => $"{x:F6}")));
```

Unzip [the example](https://recursiveai.org/onegpu/cookbook/compute-softmax-on-gpu/compute-softmax-on-gpu.zip), then `dotnet run`.

Prints:

```text
0.090031, 0.244728, 0.665241, 0.090031, 0.244728, 0.665241
```

## The full program

The input is a `[2, 3]` matrix with both rows `[1, 2, 3]`. The program checks row sums and a CPU reference. Invalid shapes or axes throw `GpuKernelShapeException` before recording.

```csharp
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var device = GpuBackends.CreateDevice(
    new GpuDeviceOptions { Adapter = "discrete" });
using var input = device.CreateBuffer<float>([1, 2, 3, 1, 2, 3]);
using var output = device.CreateBuffer<float>(6);
using var softmax = new SoftmaxKernels(device);
var commands = device.BeginCommands();
softmax.Encode(commands, input, 0, [2, 3], 1, output, 0,
    mode: GpuKernelMode.Deterministic);
device.Submit(commands);
var result = new float[6];
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result.Select(x => $"{x:F6}")));
for (int row = 0; row < 2; row++)
{
    var sum = result.Skip(row * 3).Take(3).Sum();
    Console.WriteLine($"Row {row} sum: {sum:F6}");
    for (int col = 0; col < 3; col++)
    {
        double expected = Math.Exp(col - 2) / (Math.Exp(-2) + Math.Exp(-1) + 1);
        if (Math.Abs(result[row * 3 + col] - expected) > 1e-5)
            throw new Exception("Softmax mismatch");
    }
}
Console.WriteLine("CPU tolerance 1e-5: passed");
Console.WriteLine($"{device.Backend}: {device.Adapter.Name}");
```

Output from an NVIDIA GeForce RTX 4070 using Vulkan:

```text
0.090031, 0.244728, 0.665241, 0.090031, 0.244728, 0.665241
Row 0 sum: 1.000000
Row 1 sum: 1.000000
CPU tolerance 1e-5: passed
Vulkan: NVIDIA GeForce RTX 4070
```

## Variations

### Log-softmax along the last axis

Append this to the full program:

```csharp
var logCommands = device.BeginCommands();
softmax.Encode(logCommands, input, 0, [2, 3], -1, output, 0,
    logarithm: true, mode: GpuKernelMode.Deterministic);
device.Submit(logCommands);
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result.Select(x => $"{x:F6}")));
```

Additional output:

```text
-2.407606, -1.407606, -0.407606, -2.407606, -1.407606, -0.407606
```

## Related recipes

- [Max and average pooling on the GPU in C#](https://recursiveai.org/onegpu/cookbook/pool-tensors-on-gpu/)
- [Gather embedding rows on the GPU in C#](https://recursiveai.org/onegpu/cookbook/gather-embedding-rows-on-gpu/)
- [Dynamic tensor quantization on the GPU in C#](https://recursiveai.org/onegpu/cookbook/quantize-tensors-on-gpu/)

## The short answer

These console examples use Vulkan on Linux or Windows with a Vulkan 1.3 driver.

Use `SoftmaxKernels.Encode` with an explicit shape and axis, submit the command encoder, then read the output buffer. The float32 tensor stays the same shape. For `[2, 3]` and axis `1`, each group of three scores becomes a probability distribution independently. Axis `-1` means the last dimension and selects the same operation here.

This example uses deterministic mode and an absolute CPU comparison tolerance of `1e-5`. It checks every output against a double-precision reference, not just the row sums. A sum near one alone would not detect a row whose probabilities were permuted. The tolerance is this example’s check; the library’s [kernel tolerance table](https://recursiveai.org/onegpu/docs/kernels/#tolerances) defines its own scaled-error tests.

## How do I implement softmax in C# on the GPU?

Upload dense row-major scores into a `GpuBuffer<float>`. Create a second buffer with the same element count, and create one `SoftmaxKernels` family for the device. The snippet at the top records the operation; `Submit` sends it to the device. `ReadAsync` waits for the result and copies it into the C# array.

The first row in this example is `[1, 2, 3]`, and the second is identical. Both produce approximately `[0.090031, 0.244728, 0.665241]`. No shader source file or model file is needed: the Kernels package provides the compiled operation. Keep the family and buffers alive until the work and readback finish. Reuse them for repeated calls rather than recreating the family for every tensor; families cache pipelines and buffer bindings and are not thread-safe.

Changing the axis changes which values share a normalizing sum. On this matrix, axis zero would normalize pairs down the columns instead of three values across each row. Do not flatten a batch and normalize all its scores together unless that is actually the operation your model requires.

## How do I prevent overflow in a C# softmax implementation?

The library uses a numerically stable form that subtracts the row maximum and computes statistics with an online normalizer. This avoids directly exponentiating large positive logits. Its [softmax implementation](https://github.com/recursive-ai-oss/onegpu-csharp/blob/5d3317caecb82fedd96642e54b0ab33bcb9759c5/src/RecursiveAI.OneGpu.Kernels/SoftmaxKernels.cs) documents the exceptional cases: a row containing NaN, positive infinity, or only negative infinities produces NaN throughout. A negative infinity beside finite entries produces probability zero.

Deterministic mode fixes the reduction tree for the workgroup form. It does not turn float32 into exact real arithmetic, and it does not remove the need for a numerical tolerance when comparing with a CPU formula. Short rows use a single thread; other row lengths select grouped or full-workgroup paths automatically.

For `Half` buffers, accumulation is single precision and the result is rounded on output. The softmax axis must be last, the row length must be even, and element offsets must be even. Float32 is the simpler starting point when the axis is not last.

## What is log-softmax?

Log-softmax returns log probabilities directly. Set `logarithm: true` as in the variation instead of applying a separate logarithm to already rounded probabilities. On the example rows its outputs are approximately `[-2.407606, -1.407606, -0.407606]`. Exponentiating these values recovers the probabilities, within floating-point rounding. Choose this output when the next operation expects log probabilities; choose ordinary softmax when it expects probabilities.
