---
title: "Run an ONNX model on the GPU in C#"
url: https://recursiveai.org/onemodel/cookbook/run-onnx-model-on-gpu/
description: "Run a small ONNX model on a discrete GPU in C#, print the selected adapter and API, and check placement with a runnable .NET 10 example."
version: "1.3.0"
---

# Run an ONNX model on the GPU in C#

**OneModel** is a free, open-source neural-network runtime for .NET from Recursive AI that runs ONNX models and trains models. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneModel.Gpu) · [GitHub](https://github.com/recursive-ai-oss/onemodel-csharp) · [Docs](https://recursiveai.org/onemodel/docs/) · [Download example (.zip)](https://recursiveai.org/onemodel/cookbook/run-onnx-model-on-gpu/run-onnx-model-on-gpu.zip)

Run ONNX inference on a discrete GPU. This desktop example uses Vulkan, not CUDA; no CUDA toolkit needed. It needs a working Vulkan graphics driver.

```shell
dotnet add package RecursiveAI.OneModel.Gpu
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneModel;
using RecursiveAI.OneModel.Graph;
using RecursiveAI.OneModel.Gpu;

// Input file is in the example zip.
OneModelGpu.Register(new()
    { Adapter = "discrete" });
using var model = Model.Load("square.onnx",
    new() { Device = ComputeDevice.Gpu });
Console.WriteLine(string.Join(", ",
    model.Run([1f, 2f, 3f, 4f],
        shape: [2, 2])));
```

Prints:

```text
7, 10, 15, 22
```

Unzip [the example](https://recursiveai.org/onemodel/cookbook/run-onnx-model-on-gpu/run-onnx-model-on-gpu.zip), then `dotnet run`.

`square.onnx` is in the zip. Its whole graph is `y = MatMul(x, x)`, with float32 input `x` and output `y`, both shaped `[2, 2]`. The snippet’s input is the matrix `[[1, 2], [3, 4]]`.

## The full program

`Model` is the array-based convenience wrapper; its underlying `Session` (also available as `model.Session`) accepts named tensors and provides `GetDiagnostics` and the session argument for `OneModelGpu.GetInfo`.

The full program also prints the actual adapter, API and GPU node count. This run used an NVIDIA GeForce RTX 4070 through Vulkan.

```csharp
using RecursiveAI.OneModel;
using RecursiveAI.OneModel.Graph;
using RecursiveAI.OneModel.Gpu;
using RecursiveAI.OneModel.Tensors;

OneModelGpu.Register(new GpuOptions
{
    Adapter = "discrete"
});
using var session = Session.Load("square.onnx",
    new ModelOptions
    {
        Device = ComputeDevice.Gpu
    });
using var input = Tensor.FromSpan<float>(
    [1, 2, 3, 4], 2, 2);
var inputs = new Dictionary<string, Tensor>
    { ["x"] = input };
using var outputs = session.Run(inputs);
Console.WriteLine(string.Join(", ",
    outputs["y"].ToArray<float>()));
var info = OneModelGpu.GetInfo(session);
Console.WriteLine($"Adapter: {info?.Adapter}");
Console.WriteLine($"API: {info?.Api}");
var placement = session.GetDiagnostics(inputs);
Console.WriteLine(
    $"GPU nodes: {placement.NodesOnDevice}");
```

`dotnet run` prints:

```text
7, 10, 15, 22
Adapter: NVIDIA GeForce RTX 4070
API: Vulkan
GPU nodes: 1
```

If no usable GPU exists, session creation throws `SessionException` with `DeviceUnsupported`; requesting GPU execution does not silently create a CPU-only session.

## Variations

To require every remaining runtime operator to use the GPU, replace the options in the full program with:

```csharp
new ModelOptions
{
    Device = ComputeDevice.Gpu,
    RequireGpu = true
}
```

For `square.onnx`, the output stays the same and the GPU node count remains `1`.

## Related recipes

- [Choose CPU or GPU](https://recursiveai.org/onemodel/cookbook/choose-cpu-or-gpu/)
- [Reproducible GPU inference](https://recursiveai.org/onemodel/cookbook/reproducible-gpu-inference/)
- [Run ONNX in the browser](https://recursiveai.org/onemodel/cookbook/onnx-in-the-browser-with-webgpu/)

## The short answer

Register `RecursiveAI.OneModel.Gpu` once, then load the model with `Device = ComputeDevice.Gpu`. Registration makes the GPU backend available; it does not change sessions that have already been created. Set `Adapter = "discrete"` when you want the discrete graphics card rather than an integrated adapter.

The example deliberately uses a tiny ONNX graph with no external weights, image decoding or preprocessing. You can check its arithmetic by hand: the first row of the matrix product is `[1 × 1 + 2 × 3, 1 × 2 + 2 × 4]`, or `[7, 10]`. Replace both the model and input preparation for your real application. ONNX describes the graph, but its input names, element types and dimensions still have to match the tensors you supply.

## Can ONNX run on GPU?

Yes. OneModel’s GPU package runs supported ONNX operators through Vulkan on this desktop. The package also has Metal and browser WebGPU paths. It is an optional package: the core runtime does not discover or enable it by itself. The [GPU documentation](https://recursiveai.org/onemodel/docs/gpu/) lists the supported operator families and the attribute, shape and type restrictions that decide placement.

GPU selection is not a promise that every operator runs there. OneModel offers operators to the backend individually and groups accepted consecutive operators into device regions. Operators the GPU cannot handle remain on the CPU unless you request `RequireGpu`. Constant calculations may be evaluated while making the plan rather than becoming runtime work.

Use `session.GetDiagnostics(inputs)` after supplying representative inputs. It reports the operators left after graph optimization, their placement and any fallback reason. `NodesOnDevice` is therefore a count for the resulting execution plan, not necessarily the original ONNX node count. If your model has symbolic dimensions, inspect the shapes you will actually run rather than drawing conclusions from one example batch.

`OneModelGpu.GetInfo(session)` supplies the adapter name and graphics API. The full program prints these separately from placement: successfully opening a graphics device and actually assigning operators to it are different facts. The output above establishes both for this small matrix-product graph.

## Does ONNX use CUDA?

ONNX is the model format; the runtime and backend decide how to execute it. This OneModel recipe uses Vulkan, not CUDA. Installing a CUDA toolkit is not the setup step for this path. You need a usable Vulkan device and its graphics driver. The browser recipe instead uses WebGPU and asynchronous calls, so do not copy the desktop blocking run into a browser app.

If no usable device exists, requesting `Device = Gpu` fails rather than silently replacing the session with a CPU session. To offer an explicit CPU option in your application, create a separate session with `Device = ComputeDevice.Cpu` and prepare the same named tensors. Decide whether that is acceptable for your application instead of treating a missing device as the same condition as an unsupported operator.

Keep the session alive for repeated inference and dispose it when finished. Each GPU session owns its device, compiled kernels and buffers. Runs of one session take turns on the GPU. Dispose each `SessionOutputs` after reading its tensors; the result owns them. See [tensors and ownership](https://recursiveai.org/onemodel/docs/guide-tensors/) for the distinction between tensor values and the objects that own their storage.