Use GradMode.NoGrad() to predict without recording operations for the backward pass.
dotnet add package RecursiveAI.OneModelRequires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.
using RecursiveAI.OneModel.Tensors; using var value = Tensor.FromSpan<float>([3], 1); var input = value.WithRequiresGrad(); using (GradMode.NoGrad()) { using var square = Ops.Mul(input, input); Console.WriteLine(square.Item<float>()); Console.WriteLine(square.RequiresGrad); } Console.WriteLine(GradMode.IsEnabled);
Unzip the example, then dotnet run.
The scope applies only to the current thread; don’t carry it across await.
The square is 9. It doesn’t require gradients, and recording is back on after the block in this console app:
9 False True
The full program
The download tries Backward() inside the block, then computes a fresh square outside it. The input’s gradient tracking stays on throughout the calculation.
using RecursiveAI.OneModel.Tensors; using var value = Tensor.FromSpan<float>([3], 1); var input = value.WithRequiresGrad(); bool before = GradMode.IsEnabled; using (GradMode.NoGrad()) { using var square = Ops.Mul(input, input); Console.WriteLine(square.Item<float>()); Console.WriteLine(square.RequiresGrad); Console.WriteLine($"Input still tracks: {input.RequiresGrad}"); try { square.Backward(); } catch (InvalidOperationException) { Console.WriteLine("No backward graph"); } } Console.WriteLine(GradMode.IsEnabled == before); using var tracked = Ops.Mul(input, input); tracked.Backward(); Console.WriteLine($"Gradient after scope: {input.Grad.Item<float>()}"); input.SetRequiresGrad(false);
9 False Input still tracks: True No backward graph True Gradient after scope: 6
If you call Backward() on the square computed inside NoGrad(), it throws InvalidOperationException; leaving the scope doesn’t give that result a backward graph.
Variations
Run a model’s forward pass. Add the training package for modules such as Linear:
dotnet add package RecursiveAI.OneModel.TrainingEval() sets the module’s mode; NoGrad() turns off recording for the forward pass. This example uses a one-input, one-output layer with seeded weights and a batch containing one sample:
using RecursiveAI.OneModel.Tensors; using RecursiveAI.OneModel.Training; using var model = new Linear( 1, 1, new Generator(42)); using var batch = Tensor.FromSpan<float>([3], 1, 1); model.Eval(); using (GradMode.NoGrad()) { using var prediction = model.Forward(batch); Console.WriteLine(FormattableString.Invariant( $"Prediction: {prediction.Item<float>():F3}")); Console.WriteLine(prediction.RequiresGrad); }
Prediction: 0.769 False
Return to an outer no-grad scope. Disposing the inner block restores the previous setting, which is still off here:
using RecursiveAI.OneModel.Tensors; using (GradMode.NoGrad()) { using (GradMode.NoGrad()) Console.WriteLine(GradMode.IsEnabled); Console.WriteLine(GradMode.IsEnabled); } Console.WriteLine(GradMode.IsEnabled);
False False True
Related recipes
The short answer
Wrap synchronous inference work in using (GradMode.NoGrad()). Operations inside the block still compute their outputs, but they don’t save records for the backward pass. The scope restores the previous recording mode when disposed. It doesn’t switch layers to evaluation mode or change which input tensors require gradients.
What does no-grad do during inference?
OneModel records a backward operation only when recording is enabled on the current thread and at least one input requires gradients. In the example, the input tracks gradients in both runs. The first multiply produces 9 without storing the record that would connect it to the input. After the scope, a fresh multiply can record that connection, so Backward() computes 6, the derivative of the square at 3. You can check RequiresGrad on the result without trying to backpropagate through it.
The setting is thread-local. Open and dispose the scope on the thread that performs the tensor operations. If you fetch inputs asynchronously, await the fetch first and put the synchronous calculation inside the block; don’t carry the scope across await or assume a worker thread inherits it. Also leave it out when tracing a graph: OneModel throws TraceException if you enter NoGrad() during tracing.
What’s the difference between no-grad and evaluation mode?
Eval() changes how a module and its children run. For example, a batch normalization layer uses its saved statistics instead of updating them from the current batch. NoGrad() controls recording instead. To run a trained module for prediction without recording, put the module in evaluation mode and scope the forward pass with NoGrad(). If you need gradients to inspect a model’s predictions, evaluation mode alone doesn’t turn recording off.
Is no-grad the same as freezing parameters or clearing gradients?
No. Freezing selects which parameters don’t receive gradients in later training passes. A frozen weight can take part in an operation whose other inputs still require gradients. Clearing an accumulated gradient resets what a leaf has collected so far; it doesn’t change whether the next pass records. A no-grad block does neither: it skips recording for operations inside the block without changing input flags or clearing their existing gradients.