Recursive AI

Disable gradient recording for inference in C#

OneModel is a free, open-source neural-network library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

Use GradMode.NoGrad() to predict without recording operations for the backward pass.

Terminal
dotnet add package RecursiveAI.OneModel

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using RecursiveAI.OneModel.Tensors;
 
using var value =
    Tensor.FromSpan<float>([3], 1);
var input = value.WithRequiresGrad();
using (GradMode.NoGrad())
{
    using var square = Ops.Mul(input, input);
    Console.WriteLine(square.Item<float>());
    Console.WriteLine(square.RequiresGrad);
}
Console.WriteLine(GradMode.IsEnabled);

Unzip the example, then dotnet run.

The scope applies only to the current thread; don’t carry it across await.

The square is 9. It doesn’t require gradients, and recording is back on after the block in this console app:

Text
9
False
True

The full program

The download tries Backward() inside the block, then computes a fresh square outside it. The input’s gradient tracking stays on throughout the calculation.

C#
using RecursiveAI.OneModel.Tensors;
 
using var value = Tensor.FromSpan<float>([3], 1);
var input = value.WithRequiresGrad();
bool before = GradMode.IsEnabled;
using (GradMode.NoGrad())
{
    using var square = Ops.Mul(input, input);
    Console.WriteLine(square.Item<float>());
    Console.WriteLine(square.RequiresGrad);
    Console.WriteLine($"Input still tracks: {input.RequiresGrad}");
    try
    {
        square.Backward();
    }
    catch (InvalidOperationException)
    {
        Console.WriteLine("No backward graph");
    }
}
Console.WriteLine(GradMode.IsEnabled == before);
using var tracked = Ops.Mul(input, input);
tracked.Backward();
Console.WriteLine($"Gradient after scope: {input.Grad.Item<float>()}");
input.SetRequiresGrad(false);
Text
9
False
Input still tracks: True
No backward graph
True
Gradient after scope: 6

If you call Backward() on the square computed inside NoGrad(), it throws InvalidOperationException; leaving the scope doesn’t give that result a backward graph.

Variations

Run a model’s forward pass. Add the training package for modules such as Linear:

Terminal
dotnet add package RecursiveAI.OneModel.Training

Eval() sets the module’s mode; NoGrad() turns off recording for the forward pass. This example uses a one-input, one-output layer with seeded weights and a batch containing one sample:

C#
using RecursiveAI.OneModel.Tensors;
using RecursiveAI.OneModel.Training;
 
using var model = new Linear(
    1, 1, new Generator(42));
using var batch =
    Tensor.FromSpan<float>([3], 1, 1);
model.Eval();
using (GradMode.NoGrad())
{
    using var prediction = model.Forward(batch);
    Console.WriteLine(FormattableString.Invariant(
        $"Prediction: {prediction.Item<float>():F3}"));
    Console.WriteLine(prediction.RequiresGrad);
}
Text
Prediction: 0.769
False

Return to an outer no-grad scope. Disposing the inner block restores the previous setting, which is still off here:

C#
using RecursiveAI.OneModel.Tensors;
 
using (GradMode.NoGrad())
{
    using (GradMode.NoGrad())
        Console.WriteLine(GradMode.IsEnabled);
    Console.WriteLine(GradMode.IsEnabled);
}
Console.WriteLine(GradMode.IsEnabled);
Text
False
False
True

The short answer

Wrap synchronous inference work in using (GradMode.NoGrad()). Operations inside the block still compute their outputs, but they don’t save records for the backward pass. The scope restores the previous recording mode when disposed. It doesn’t switch layers to evaluation mode or change which input tensors require gradients.

What does no-grad do during inference?

OneModel records a backward operation only when recording is enabled on the current thread and at least one input requires gradients. In the example, the input tracks gradients in both runs. The first multiply produces 9 without storing the record that would connect it to the input. After the scope, a fresh multiply can record that connection, so Backward() computes 6, the derivative of the square at 3. You can check RequiresGrad on the result without trying to backpropagate through it.

The setting is thread-local. Open and dispose the scope on the thread that performs the tensor operations. If you fetch inputs asynchronously, await the fetch first and put the synchronous calculation inside the block; don’t carry the scope across await or assume a worker thread inherits it. Also leave it out when tracing a graph: OneModel throws TraceException if you enter NoGrad() during tracing.

What’s the difference between no-grad and evaluation mode?

Eval() changes how a module and its children run. For example, a batch normalization layer uses its saved statistics instead of updating them from the current batch. NoGrad() controls recording instead. To run a trained module for prediction without recording, put the module in evaluation mode and scope the forward pass with NoGrad(). If you need gradients to inspect a model’s predictions, evaluation mode alone doesn’t turn recording off.

Is no-grad the same as freezing parameters or clearing gradients?

No. Freezing selects which parameters don’t receive gradients in later training passes. A frozen weight can take part in an operation whose other inputs still require gradients. Clearing an accumulated gradient resets what a leaf has collected so far; it doesn’t change whether the next pass records. A no-grad block does neither: it skips recording for operations inside the block without changing input flags or clearing their existing gradients.

All onemodel recipes