---
title: "Reproducible GPU inference in C#"
url: https://recursiveai.org/onemodel/cookbook/reproducible-gpu-inference/
description: "Run exact-mode GPU inference twice in C#, compare float32 output bits, and understand the same-device, same-driver reproducibility boundary."
version: "1.3.0"
---

# Reproducible GPU inference in C#

**OneModel** is a free, open-source neural-network runtime for .NET from Recursive AI that runs ONNX models and trains models. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneModel.Gpu) · [GitHub](https://github.com/recursive-ai-oss/onemodel-csharp) · [Docs](https://recursiveai.org/onemodel/docs/) · [Download example (.zip)](https://recursiveai.org/onemodel/cookbook/reproducible-gpu-inference/reproducible-gpu-inference.zip)

Exact GPU mode repeats the same output bits on the same device and driver, not across different GPUs or against the CPU.

```shell
dotnet add package RecursiveAI.OneModel.Gpu
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneModel;
using RecursiveAI.OneModel.Graph;
using RecursiveAI.OneModel.Gpu;

// Input file is in the example zip.
OneModelGpu.Register(new()
    { Adapter = "discrete" });
using var model = Model.Load("square.onnx", new()
{
    Device = ComputeDevice.Gpu,
    Mode = ArithmeticMode.Exact,
    RequireGpu = true
});
float[] input = [0.1f, 0.2f, 0.3f, 0.4f];
var a = model.Run(input, shape: [2, 2]);
var b = model.Run(input, shape: [2, 2]);
var aBits = a.Select(
    BitConverter.SingleToInt32Bits);
var bBits = b.Select(
    BitConverter.SingleToInt32Bits);
Console.WriteLine(aBits.SequenceEqual(bBits));
```

Prints:

```text
True
```

Unzip [the example](https://recursiveai.org/onemodel/cookbook/reproducible-gpu-inference/reproducible-gpu-inference.zip), then `dotnet run`.

`square.onnx` is in the zip. Its whole graph is `y = MatMul(x, x)`, with float32 input `x` and output `y`, both shaped `[2, 2]`. The test input is `[[0.1, 0.2], [0.3, 0.4]]`.

`Adapter = "discrete"` requests a discrete graphics card, not an integrated GPU. On an integrated-only system, that explicit selection cannot find a matching adapter and session creation fails with `DeviceUnsupported`; use `Adapter = "integrated"` or omit the selection to use the platform’s default.

## The full program

`Model` is the array-based convenience wrapper; its underlying `Session` (also available as `model.Session`) accepts named tensors and provides `GetDiagnostics` and the session argument for `OneModelGpu.GetInfo`.

The full program prints each float32 value’s hexadecimal bit pattern, not rounded decimal text. This run used an NVIDIA GeForce RTX 4070 through Vulkan.

```csharp
using RecursiveAI.OneModel;
using RecursiveAI.OneModel.Graph;
using RecursiveAI.OneModel.Gpu;
using RecursiveAI.OneModel.Tensors;

OneModelGpu.Register(new GpuOptions
{
    Adapter = "discrete"
});
using var session = Session.Load(
    "square.onnx", new ModelOptions
{
    Device = ComputeDevice.Gpu,
    Mode = ArithmeticMode.Exact,
    RequireGpu = true
});
using var input = Tensor.FromSpan<float>(
    [0.1f, 0.2f, 0.3f, 0.4f], 2, 2);
var inputs = new Dictionary<string, Tensor>
    { ["x"] = input };
using var first = session.Run(inputs);
using var second = session.Run(inputs);
int[] firstBits = first["y"].ToArray<float>()
    .Select(BitConverter.SingleToInt32Bits).ToArray();
int[] secondBits = second["y"].ToArray<float>()
    .Select(BitConverter.SingleToInt32Bits).ToArray();
Console.WriteLine("Run 1: " + string.Join(" ",
    firstBits.Select(x => $"{x:X8}")));
Console.WriteLine("Run 2: " + string.Join(" ",
    secondBits.Select(x => $"{x:X8}")));
Console.WriteLine("Same bits: "
    + firstBits.SequenceEqual(secondBits));
var info = OneModelGpu.GetInfo(session);
Console.WriteLine($"Adapter: {info?.Adapter}; "
    + $"API: {info?.Api}");
var placement = session.GetDiagnostics(inputs);
Console.WriteLine(
    $"GPU nodes: {placement.NodesOnDevice}");
```

`dotnet run` prints:

```text
Run 1: 3D8F5C29 3DCCCCCE 3E19999A 3E6147AF
Run 2: 3D8F5C29 3DCCCCCE 3E19999A 3E6147AF
Same bits: True
Adapter: NVIDIA GeForce RTX 4070; API: Vulkan
GPU nodes: 1
```

With `RequireGpu`, an operator lacking a deterministic GPU kernel is rejected with `DeviceUnsupported` rather than silently kept on the CPU.

## Variations

Use the same arithmetic mode on an explicitly selected CPU session by replacing the full program’s options with the following. Also remove the `GetInfo` call and adapter-print line, which apply only to GPU sessions:

```csharp
new ModelOptions
{
    Device = ComputeDevice.Cpu,
    Mode = ArithmeticMode.Exact
}
```

This variant also prints `Same bits: True`; its diagnostics report `GPU nodes: 0`. CPU and GPU exact mode have different reproducibility scopes.

## Related recipes

- [Choose CPU or GPU](https://recursiveai.org/onemodel/cookbook/choose-cpu-or-gpu/)
- [Use exact arithmetic mode](https://recursiveai.org/onemodel/cookbook/exact-mode/)

## The short answer

Set `Mode = ArithmeticMode.Exact` together with `Device = ComputeDevice.Gpu`, and keep the model, inputs, device and driver fixed. OneModel documents identical bits for repeated runs on that device and driver. This is not a promise of matching bits on another GPU, another driver, or the CPU.

The snippet at the top compares the integer bit representations of the output floats. The full program prints those representations in hexadecimal so you can see the comparison. It also uses `RequireGpu = true` and checks placement: the matrix product must actually run on the GPU for this to demonstrate GPU repeatability.

## How do I make GPU inference deterministic?

In OneModel, use exact-mode GPU kernels rather than relying on the default fast mode. According to the [GPU arithmetic contract](https://recursiveai.org/onemodel/docs/gpu/#fast-and-exact-mode), exact kernels use a fixed summation order, no atomics and no scheduling dependence. Matrix products, reductions, normalization and softmax have deterministic variants. Elementwise and layout operators have no reduction order, so their device kernel serves both modes.

Keep the inputs identical as well as the execution settings. A deterministic kernel does not make two different input tensors equivalent. This example spells out all four float32 inputs and ships a model with no external weights. For a deployed model, retain the exact model file and the tensors used for the comparison, not just a description of how they were prepared.

Compare bits rather than formatted numbers. Printing a few decimal places can hide differences in the last bits. Here `BitConverter.SingleToInt32Bits` preserves the float32 representation, and `SequenceEqual` compares the resulting integer arrays element by element. The displayed hexadecimal values are those integers written with eight hexadecimal digits. A matching printout is meaningful for these outputs because no decimal rounding is involved.

An operator without a deterministic GPU kernel stays on the CPU unless GPU execution is required. Add `RequireGpu` when mixed placement would make your check misleading, and inspect diagnostics with the actual input shapes. An error can appear at session creation or when the first plan for a shape is made. The full program’s strict requirement avoids silently presenting a CPU result as a GPU result.

## Why can GPU inference be non-deterministic across devices?

OneModel’s GPU exact contract is narrower than its CPU exact contract. GPU kernels are repeatable on one device and driver, but another adapter or driver can differ in the last bits. Device transcendental functions such as `Exp`, `Tanh` and `Erf` are not the CPU exact mode’s fixed polynomials. Consequently, the docs describe GPU-to-CPU agreement in float32 tolerances, not bit identity.

Do not turn a successful two-run comparison into a cross-hardware guarantee. It shows that these inputs and this model repeated on the selected setup. If you change the GPU or driver, record that as a different execution environment and apply the numerical comparison your application requires. Preserve the adapter and API information with your result; the program prints them for this reason.

Likewise, exact inference is not a training seed setting or a text-generation sampling policy. This recipe executes an existing graph twice. It does not initialize weights, shuffle a dataset or choose output tokens. Keep those separate sources of variation out of a kernel repeatability check. For a CPU deployment requiring OneModel’s cross-machine exact behavior, choose the CPU device explicitly and follow the [exact-mode documentation](https://recursiveai.org/onemodel/docs/exact-mode/) for supported operators.