---
title: "Automatic differentiation of tensors in C#"
url: https://recursiveai.org/onemodel/cookbook/differentiate-tensor-expression/
description: "Compute tensor gradients in C# with reverse-mode automatic differentiation, compare an analytic derivative, and manage gradient accumulation and graph lifetime."
version: "1.3.0"
---

# Automatic differentiation of tensors in C#

**OneModel** is a free, open-source neural-network library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneModel) · [GitHub](https://github.com/recursive-ai-oss/onemodel-csharp) · [Docs](https://recursiveai.org/onemodel/docs/) · [Download example (.zip)](https://recursiveai.org/onemodel/cookbook/differentiate-tensor-expression/differentiate-tensor-expression.zip)

Automatic differentiation computes derivatives for tensor expressions.

```shell
dotnet add package RecursiveAI.OneModel
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneModel.Tensors;
using var x = Tensor.Scalar(3f)
    .WithRequiresGrad();
using var loss = Ops.Mul(x, x);
loss.Backward();
Console.WriteLine(x.Grad.Data<float>()[0]);
Backprop.ClearGrad(x);
```

Prints:

```text
6
```

Unzip [the example](https://recursiveai.org/onemodel/cookbook/differentiate-tensor-expression/differentiate-tensor-expression.zip), then `dotnet run`.

## The full program

The full program differentiates `loss = x² + x` at `x = 3`. The analytic derivative is `2x + 1 = 7`. It then reuses the retained graph, clears the accumulated gradient and tries the released graph.

```csharp
using RecursiveAI.OneModel.Tensors;

using var x = Tensor.Scalar(3f).WithRequiresGrad();
using var square = Ops.Mul(x, x);
using var loss = Ops.Add(square, x);
loss.Backward(retainGraph: true);
Console.WriteLine($"loss = {loss.Data<float>()[0]}");
Console.WriteLine($"gradient = {x.Grad.Data<float>()[0]} (expected 7)");
loss.Backward();
Console.WriteLine($"accumulated = {x.Grad.Data<float>()[0]}");
Backprop.ZeroGrad(x);
Console.WriteLine($"zeroed = {x.Grad.Data<float>()[0]}");
try { loss.Backward(); }
catch (InvalidOperationException) { Console.WriteLine("graph released"); }
Backprop.ClearGrad(x);
```

`dotnet run` prints:

```text
loss = 12
gradient = 7 (expected 7)
accumulated = 14
zeroed = 0
graph released
```

A second backward pass through a released graph throws `InvalidOperationException`; changing saved tensor storage before backward can throw `TensorVersionException`.

## Variations

### Seed a vector output

A non-scalar output needs an explicit gradient with the same shape. This seed weights the two output elements by one and two:

```csharp
using RecursiveAI.OneModel.Tensors;
using var x = Tensor.FromSpan<float>([2, 3], Shape.Create(2))
    .WithRequiresGrad();
using var output = Ops.Mul(x, x);
using var seed = Tensor.FromSpan<float>([1, 2], Shape.Create(2));
output.Backward(seed);
Console.WriteLine(string.Join(",", x.Grad.Data<float>().ToArray()));
Backprop.ClearGrad(x);
```

```text
4,12
```

## Related recipes

- [Train a neural network](https://recursiveai.org/onemodel/cookbook/train-a-model/)
- [Run an ONNX model](https://recursiveai.org/onemodel/cookbook/run-onnx-model/)
- [Slice and reshape tensor views](https://recursiveai.org/onemodel/cookbook/slice-and-reshape-tensor-views/)

## The short answer

Use `WithRequiresGrad()` on a float32 input, build the expression with `Ops`, then call `Backward()` on its scalar result. Read the derivative from the input’s `Grad`. OneModel uses reverse-mode automatic differentiation: it records the executed operations, then walks those records backward to propagate derivatives to the inputs.

## How do you implement automatic differentiation in C#?

Keep the calculation in tensor operations. Multiplying two numbers read from `Data<float>()` with ordinary C# arithmetic does not create a tensor operation for the recorder to follow. `Ops.Mul(x, x)` does. Store the handle returned by `WithRequiresGrad()`; calling it and ignoring the return value does not mark your original handle as a leaf.

For a first check, choose an expression with a derivative you can calculate directly. The snippet at the top uses `x²`, whose derivative is `2x`, so the result at three is six. The full program adds `x`, changing the derivative to seven. These tiny checks distinguish the value of the expression from its gradient: twelve is the loss, not the derivative.

The [automatic differentiation reference](https://recursiveai.org/onemodel/docs/autograd/) specifies float32 gradients with the same shape as their tensors. This is not symbolic algebra: it computes numerical derivatives at the values you supplied. It also does not estimate a derivative by perturbing the input. The backward rules for the recorded operations determine the result.

## How does reverse-mode automatic differentiation work?

A leaf is an input marked with `WithRequiresGrad()`. A recorded operation keeps edges to inputs needing gradients and any tensors needed by its backward rule. Backward processes consumers before producers, combining contributions from each use of an input. That is why using `x` twice in a multiplication contributes both terms of the derivative.

A one-element output gets a default seed of one. A vector output instead needs an explicit seed of matching shape, as the variation shows. The result is a weighted combination of the output derivatives, not a full Jacobian matrix. For the independent squares at inputs two and three, the seed `[1, 2]` produces `[4, 12]`.

Leaf gradients accumulate. The full program retains its graph for the first backward call and releases it on the second, producing fourteen after two contributions of seven. Call `Backprop.ZeroGrad` before the next independent training step when you do not want that sum. `ClearGrad` frees the gradient instead of keeping its storage filled with zeros.

Higher-order derivatives are not supported. Backward runs with recording disabled, so gradients cannot themselves be differentiated through this recorded graph. The reference explicitly excludes higher-order derivatives.

Graph lifetime and tensor lifetime are separate concerns. The default backward pass releases saved records, but you must still dispose tensors you own. This example disposes each forward result and clears the leaf gradient. For larger forward computations, an `ActivationScope` can collect operator-allocated tensors; leaf gradients remain outside that scope. Do not mutate an input that a backward rule saved between forward and backward: the shared storage version is checked when the rule runs.
