Recursive AI

Automatic differentiation of tensors in C#

OneModel is a free, open-source neural-network library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

Automatic differentiation computes derivatives for tensor expressions.

Terminal
dotnet add package RecursiveAI.OneModel

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using RecursiveAI.OneModel.Tensors;
using var x = Tensor.Scalar(3f)
    .WithRequiresGrad();
using var loss = Ops.Mul(x, x);
loss.Backward();
Console.WriteLine(x.Grad.Data<float>()[0]);
Backprop.ClearGrad(x);

Prints:

Text
6

Unzip the example, then dotnet run.

The full program

The full program differentiates loss = x² + x at x = 3. The analytic derivative is 2x + 1 = 7. It then reuses the retained graph, clears the accumulated gradient and tries the released graph.

C#
using RecursiveAI.OneModel.Tensors;
 
using var x = Tensor.Scalar(3f).WithRequiresGrad();
using var square = Ops.Mul(x, x);
using var loss = Ops.Add(square, x);
loss.Backward(retainGraph: true);
Console.WriteLine($"loss = {loss.Data<float>()[0]}");
Console.WriteLine($"gradient = {x.Grad.Data<float>()[0]} (expected 7)");
loss.Backward();
Console.WriteLine($"accumulated = {x.Grad.Data<float>()[0]}");
Backprop.ZeroGrad(x);
Console.WriteLine($"zeroed = {x.Grad.Data<float>()[0]}");
try { loss.Backward(); }
catch (InvalidOperationException) { Console.WriteLine("graph released"); }
Backprop.ClearGrad(x);

dotnet run prints:

Text
loss = 12
gradient = 7 (expected 7)
accumulated = 14
zeroed = 0
graph released

A second backward pass through a released graph throws InvalidOperationException; changing saved tensor storage before backward can throw TensorVersionException.

Variations

Seed a vector output

A non-scalar output needs an explicit gradient with the same shape. This seed weights the two output elements by one and two:

C#
using RecursiveAI.OneModel.Tensors;
using var x = Tensor.FromSpan<float>([2, 3], Shape.Create(2))
    .WithRequiresGrad();
using var output = Ops.Mul(x, x);
using var seed = Tensor.FromSpan<float>([1, 2], Shape.Create(2));
output.Backward(seed);
Console.WriteLine(string.Join(",", x.Grad.Data<float>().ToArray()));
Backprop.ClearGrad(x);
Text
4,12

The short answer

Use WithRequiresGrad() on a float32 input, build the expression with Ops, then call Backward() on its scalar result. Read the derivative from the input’s Grad. OneModel uses reverse-mode automatic differentiation: it records the executed operations, then walks those records backward to propagate derivatives to the inputs.

How do you implement automatic differentiation in C#?

Keep the calculation in tensor operations. Multiplying two numbers read from Data<float>() with ordinary C# arithmetic does not create a tensor operation for the recorder to follow. Ops.Mul(x, x) does. Store the handle returned by WithRequiresGrad(); calling it and ignoring the return value does not mark your original handle as a leaf.

For a first check, choose an expression with a derivative you can calculate directly. The snippet at the top uses x², whose derivative is 2x, so the result at three is six. The full program adds x, changing the derivative to seven. These tiny checks distinguish the value of the expression from its gradient: twelve is the loss, not the derivative.

The automatic differentiation reference specifies float32 gradients with the same shape as their tensors. This is not symbolic algebra: it computes numerical derivatives at the values you supplied. It also does not estimate a derivative by perturbing the input. The backward rules for the recorded operations determine the result.

How does reverse-mode automatic differentiation work?

A leaf is an input marked with WithRequiresGrad(). A recorded operation keeps edges to inputs needing gradients and any tensors needed by its backward rule. Backward processes consumers before producers, combining contributions from each use of an input. That is why using x twice in a multiplication contributes both terms of the derivative.

A one-element output gets a default seed of one. A vector output instead needs an explicit seed of matching shape, as the variation shows. The result is a weighted combination of the output derivatives, not a full Jacobian matrix. For the independent squares at inputs two and three, the seed [1, 2] produces [4, 12].

Leaf gradients accumulate. The full program retains its graph for the first backward call and releases it on the second, producing fourteen after two contributions of seven. Call Backprop.ZeroGrad before the next independent training step when you do not want that sum. ClearGrad frees the gradient instead of keeping its storage filled with zeros.

Higher-order derivatives are not supported. Backward runs with recording disabled, so gradients cannot themselves be differentiated through this recorded graph. The reference explicitly excludes higher-order derivatives.

Graph lifetime and tensor lifetime are separate concerns. The default backward pass releases saved records, but you must still dispose tensors you own. This example disposes each forward result and clears the leaf gradient. For larger forward computations, an ActivationScope can collect operator-allocated tensors; leaf gradients remain outside that scope. Do not mutate an input that a backward rule saved between forward and backward: the shared storage version is checked when the rule runs.

All onemodel recipes