Recursive AI

Release training activation memory in C#

OneModel is a free, open-source neural-network library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

Close ActivationScope after Backward() to return temporary forward tensors’ memory to OneModel’s buffer pool for reuse rather than freeing it back to the operating system.

Terminal
dotnet add package RecursiveAI.OneModel.Training

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using RecursiveAI.OneModel.Tensors;
 
using var weight = Tensor.Scalar(3f)
    .WithRequiresGrad();
using (var step = new ActivationScope())
{
    var loss = Ops.Mul(weight, weight);
    loss.Backward();
}
var gradient = weight.Grad.Data<float>();
Console.WriteLine(gradient[0]);

Unzip the example, then dotnet run.

Text
6

The full program

The download fits a single weight toward zero by minimizing weight squared. It warms up the shared pool with one full SGD step, runs four more, and compares pool counters between steps, after the scopes have closed.

C#
using RecursiveAI.OneModel.Tensors;
using RecursiveAI.OneModel.Training;
 
using var weight = Tensor.Scalar(3f).WithRequiresGrad();
using var optimizer = new Sgd([weight], 0.125f);
var pool = BufferPool.Shared;
float firstGradient = Step();
var warm = pool.GetStats();
for (int i = 0; i < 4; i++)
    Step();
var end = pool.GetStats();
 
Console.WriteLine($"First gradient after scope: {firstGradient}");
Console.WriteLine($"Optimizer steps: {optimizer.StepCount}");
Console.WriteLine($"Additional native allocations: {end.NativeAllocations - warm.NativeAllocations}");
Console.WriteLine($"Change in outstanding bytes: {end.OutstandingBytes - warm.OutstandingBytes}");
 
float Step()
{
    Backprop.ZeroGrad(weight);
    using (var activations = new ActivationScope())
    {
        var loss = Ops.Mul(weight, weight);
        loss.Backward();
    }
    float gradient = weight.Grad.Data<float>()[0];
    optimizer.Step();
    return gradient;
}
Text
First gradient after scope: 6
Optimizer steps: 5
Additional native allocations: 0
Change in outstanding bytes: 0

Don’t read a temporary after its scope closes unless you deliberately kept it or copied the data out.

Variations

Keep the loss tensor after the scope ends. Replace the using block and final print in the snippet at the top. Keep transfers responsibility for disposing the result to you when there’s no enclosing scope. This prints 9:

C#
Tensor keptLoss;
using (var step = new ActivationScope())
{
    keptLoss = ActivationScope.Keep(
        Ops.Mul(weight, weight));
    keptLoss.Backward();
}
using (keptLoss)
    Console.WriteLine(keptLoss.Data<float>()[0]);

The short answer

Wrap the forward pass and Backward() in an ActivationScope for each step. When the scope closes, OneModel returns the temporary tensor buffers it collected to their pools. Leaf gradients belong to the tensors they differentiate, not the scope. In the example, the gradient of the original trainable weight is still 6 when you read it outside the block.

How do you reduce autograd memory usage during training?

Use one scope per step, not one around the whole training loop. The scope collects tensors allocated by eager Ops calls and copies made by Contiguous or a copying Reshape. The backward engine handles its own temporaries. Tensors you create directly aren’t automatically collected, and views share storage rather than extending the life of their source. Keep parameters and inputs you intend to reuse outside the scope. See the ActivationScope contract.

For a metric such as loss, read its scalar value into a float while the scope is open. That leaves you with the number, not another live tensor. If you need the tensor itself, use Keep as shown above. In nested scopes, Keep moves a result to the enclosing scope, not past both scopes. Scopes are thread-local, so keep the enclosed work and disposal on the same thread rather than crossing an await that can resume on another thread.

Why doesn’t memory drop after backward?

There are two separate lifetimes. By default Backward() releases the graph’s saved records as it uses them. Disposing the activation scope returns the buffers for the tracked forward outputs. Returning a buffer to the pool makes it available for reuse; it doesn’t necessarily free it back to the OS. It also doesn’t clear a parameter’s gradient. Call Backprop.ZeroGrad to reuse that gradient buffer for a new step, or Backprop.ClearGrad to release it.

Compare OutstandingBytes at the same point between steps: it’s the capacity currently rented from the pool, not the process’s memory usage. NativeAllocations counts rents that had to allocate because no matching idle block was available. In this fixed-shape example neither counter increased after warm-up, though the step still creates managed objects for autograd records. Other workloads and changes in tensor shape can require new buffers.

All onemodel recipes