---
title: "Gather embedding rows on the GPU in C#"
url: https://recursiveai.org/onegpu/cookbook/gather-embedding-rows-on-gpu/
description: "Gather embedding rows on the GPU in C#. Includes buffer setup, real Vulkan output and a runnable .NET 10 example."
version: "1.0.0"
---

# Gather embedding rows on the GPU in C#

**OneGpu** is a free, open-source GPU compute library for .NET from Recursive AI. MIT or Apache-2.0, no bundled native binaries.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneGpu.Backends) · [GitHub](https://github.com/recursive-ai-oss/onegpu-csharp) · [Docs](https://recursiveai.org/onegpu/docs/) · [Download example (.zip)](https://recursiveai.org/onegpu/cookbook/gather-embedding-rows-on-gpu/gather-embedding-rows-on-gpu.zip)

An embedding lookup selects table rows by integer ID. This example requires Vulkan 1.3.

```shell
dotnet add package RecursiveAI.OneGpu.Backends
dotnet add package RecursiveAI.OneGpu.Kernels
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var device = GpuBackends.CreateDevice(
    new GpuDeviceOptions { Adapter = "discrete" });
using var table = device.CreateBuffer<float>([10, 11, 20, 21, 30, 31]);
using var indices = device.CreateBuffer<int>([2, 0, -1]);
using var output = device.CreateBuffer<float>(6);
using var gather = new GatherKernels(device);
using var status = new GpuIndexStatus(device);
var commands = device.BeginCommands();
gather.EncodeGather(commands, table, 0, [3, 2], 0,
    indices, 0, 3, output, 0, status);
device.Submit(commands);
await status.ThrowIfFailedAsync();
var result = new float[6];
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result));
```

Unzip [the example](https://recursiveai.org/onegpu/cookbook/gather-embedding-rows-on-gpu/gather-embedding-rows-on-gpu.zip), then `dotnet run`.

Prints:

```text
30, 31, 10, 11, 30, 31
```

## The full program

The table is `[3, 2]` and the indices are `[2, 0, -1]`. Invalid IDs write zeros for affected elements; checking the supplied status throws `GpuKernelIndexException` after execution.

```csharp
using RecursiveAI.OneGpu;
using RecursiveAI.OneGpu.Kernels;

using var device = GpuBackends.CreateDevice(
    new GpuDeviceOptions { Adapter = "discrete" });
using var table = device.CreateBuffer<float>([10, 11, 20, 21, 30, 31]);
using var indices = device.CreateBuffer<int>([2, 0, -1]);
using var output = device.CreateBuffer<float>(6);
using var gather = new GatherKernels(device);
using var status = new GpuIndexStatus(device);
var commands = device.BeginCommands();
gather.EncodeGather(commands, table, 0, [3, 2], 0,
    indices, 0, 3, output, 0, status);
device.Submit(commands);
await status.ThrowIfFailedAsync();
var result = new float[6];
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result));
if (!result.SequenceEqual(new float[] { 30, 31, 10, 11, 30, 31 }))
    throw new Exception("Gather mismatch");
Console.WriteLine("Expected rows: passed");
Console.WriteLine($"{device.Backend}: {device.Adapter.Name}");
```

Output from an NVIDIA GeForce RTX 4070 using Vulkan:

```text
30, 31, 10, 11, 30, 31
Expected rows: passed
Vulkan: NVIDIA GeForce RTX 4070
```

## Variations

### Int64 indices and an invalid row ID

Append this to the full program:

```csharp
using var longIndices = device.CreateBuffer<long>([-1, 1, 0]);
status.Reset();
var pass = device.BeginCommands();
gather.EncodeGather(pass, table, 0, [3, 2], 0,
    longIndices, 0, 3, output, 0, status);
device.Submit(pass);
await status.ThrowIfFailedAsync();
await output.ReadAsync(result);
Console.WriteLine(string.Join(", ", result));
// Now deliberately ask for row 3 in a table with rows 0..2.
indices.Write([3, 0, 1]);
status.Reset();
var invalid = device.BeginCommands();
gather.EncodeGather(invalid, table, 0, [3, 2], 0,
    indices, 0, 3, output, 0, status);
device.Submit(invalid);
try { await status.ThrowIfFailedAsync(); }
catch (GpuKernelIndexException error) { Console.WriteLine(error.Message); }
```

Additional output:

```text
30, 31, 20, 21, 10, 11
Gather: the index at position 0 of the indices is outside its axis
```

## Related recipes

- [Compute softmax on the GPU in C#](https://recursiveai.org/onegpu/cookbook/compute-softmax-on-gpu/)
- [Max and average pooling on the GPU in C#](https://recursiveai.org/onegpu/cookbook/pool-tensors-on-gpu/)
- [Dynamic tensor quantization on the GPU in C#](https://recursiveai.org/onegpu/cookbook/quantize-tensors-on-gpu/)

## The short answer

These console examples use Vulkan on Linux or Windows with a Vulkan 1.3 driver.

Use `GatherKernels.EncodeGather` with a table shape `[rowCount, embeddingWidth]` and axis zero. Upload integer row IDs, allocate `indexCount * embeddingWidth` output elements, and pass a `GpuIndexStatus` so invalid IDs are reported after submission. This selects existing vectors; it does not generate embeddings from text.

The table in this recipe has rows `[10, 11]`, `[20, 21]` and `[30, 31]`. IDs `[2, 0, -1]` return the third row, first row and third row again. The full program checks every selected value exactly. There is no floating-point arithmetic in the selection itself, so no tolerance is needed for this fixture.

## How does gather implement an embedding lookup?

A row ID chooses a vector from a dense table. The GPU copies that row’s payload to the next output row. In this example, each vector contains two float32 elements, so each ID produces two adjacent output values. Repeated IDs intentionally repeat their vectors. Their order in the output is the order of the IDs, not a sorted order of table rows.

More generally, Gather replaces the selected data axis with the shape of the indices. Around that axis, the library treats the data as outer dimensions, the axis length and inner dimensions. For an embedding table gathered along axis zero, the inner extent is the embedding width. The API takes the number of indices; your application tracks their shape when interpreting the flat result.

All tensors are dense and row-major. Element offsets are measured in elements of the buffer’s type, not bytes. The output must not overlap the data. Keep the table resident when selecting several batches of IDs, and reuse the family so its cached pipelines and bindings can be reused. Families are not thread-safe; arrange their use accordingly. The [kernel guide](https://recursiveai.org/onegpu/docs/kernels/) describes this buffer-based model.

## Can a GPU gather use negative or int64 indices?

Yes. An index of `-1` means the final entry on the selected axis; `-rowCount` means its first entry. For a three-row table, valid IDs are from `-3` through `2`. A negative index is not a padding ID or a request for a zero vector.

Both C# `int` and `long` index buffers are supported. The variation uses `GpuBuffer<long>` and returns rows for `[-1, 1, 0]`. You do not have to change the float payload type when changing the index type. Int64 indices are represented with pairs of 32-bit words, so they do not require native 64-bit integer operations on the GPU.

Float is not the only payload option: the documented family also moves Half, int, uint, long, byte and sbyte payloads. Packed narrow types have word-alignment rules, so check the buffer padding requirements before replacing this float32 fixture.

## What happens when a gather index is out of range?

An invalid index never reads outside the table. The kernel writes zero for the affected output element and records the smallest offending index position in the supplied status. `ThrowIfFailedAsync` waits for completion, reads that record and throws `GpuKernelIndexException`. In the variation, ID `3` is invalid and the exception identifies position zero.

Do not consume the result as a successful embedding batch before checking the status. Passing no status makes invalid indices produce zeros silently, which is usually not the error behavior wanted for an embedding lookup. Call `Reset` before reusing a status for another independent batch; otherwise a previous failure remains recorded. If the IDs are already on the CPU, the library also exposes host-side index validation before upload. Shape mistakes, such as an invalid axis or insufficient output capacity, are separate `GpuKernelShapeException` errors checked before recording.
