---
title: "Release training activation memory in C#"
url: https://recursiveai.org/onemodel/cookbook/scope-training-activation-memory/
description: "Use OneModel ActivationScope to return temporary tensors to the pool after backward in C#. Keep leaf gradients available for the optimizer step."
version: "1.3.0"
---

# Release training activation memory in C#

**OneModel** is a free, open-source neural-network library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

[NuGet](https://www.nuget.org/packages/RecursiveAI.OneModel.Training) · [GitHub](https://github.com/recursive-ai-oss/onemodel-csharp) · [Docs](https://recursiveai.org/onemodel/docs/) · [Download example (.zip)](https://recursiveai.org/onemodel/cookbook/scope-training-activation-memory/scope-training-activation-memory.zip)

Close `ActivationScope` after `Backward()` to return temporary forward tensors’ memory to OneModel’s buffer pool for reuse rather than freeing it back to the operating system.

```shell
dotnet add package RecursiveAI.OneModel.Training
```

Requires [.NET 10](https://dotnet.microsoft.com/download/dotnet/10.0) (net10.0); .NET 9 and earlier aren’t supported.

```csharp
using RecursiveAI.OneModel.Tensors;

using var weight = Tensor.Scalar(3f)
    .WithRequiresGrad();
using (var step = new ActivationScope())
{
    var loss = Ops.Mul(weight, weight);
    loss.Backward();
}
var gradient = weight.Grad.Data<float>();
Console.WriteLine(gradient[0]);
```

Unzip [the example](https://recursiveai.org/onemodel/cookbook/scope-training-activation-memory/scope-training-activation-memory.zip), then `dotnet run`.

```text
6
```

## The full program

The download fits a single weight toward zero by minimizing weight squared. It warms up the shared pool with one full SGD step, runs four more, and compares pool counters between steps, after the scopes have closed.

```csharp
using RecursiveAI.OneModel.Tensors;
using RecursiveAI.OneModel.Training;

using var weight = Tensor.Scalar(3f).WithRequiresGrad();
using var optimizer = new Sgd([weight], 0.125f);
var pool = BufferPool.Shared;
float firstGradient = Step();
var warm = pool.GetStats();
for (int i = 0; i < 4; i++)
    Step();
var end = pool.GetStats();

Console.WriteLine($"First gradient after scope: {firstGradient}");
Console.WriteLine($"Optimizer steps: {optimizer.StepCount}");
Console.WriteLine($"Additional native allocations: {end.NativeAllocations - warm.NativeAllocations}");
Console.WriteLine($"Change in outstanding bytes: {end.OutstandingBytes - warm.OutstandingBytes}");

float Step()
{
    Backprop.ZeroGrad(weight);
    using (var activations = new ActivationScope())
    {
        var loss = Ops.Mul(weight, weight);
        loss.Backward();
    }
    float gradient = weight.Grad.Data<float>()[0];
    optimizer.Step();
    return gradient;
}
```

```text
First gradient after scope: 6
Optimizer steps: 5
Additional native allocations: 0
Change in outstanding bytes: 0
```

Don’t read a temporary after its scope closes unless you deliberately kept it or copied the data out.

## Variations

**Keep the loss tensor after the scope ends.** Replace the `using` block and final print in the snippet at the top. `Keep` transfers responsibility for disposing the result to you when there’s no enclosing scope. This prints `9`:

```csharp
Tensor keptLoss;
using (var step = new ActivationScope())
{
    keptLoss = ActivationScope.Keep(
        Ops.Mul(weight, weight));
    keptLoss.Backward();
}
using (keptLoss)
    Console.WriteLine(keptLoss.Data<float>()[0]);
```

## Related recipes

- [Differentiate a tensor expression](https://recursiveai.org/onemodel/cookbook/differentiate-tensor-expression/)
- [Accumulate gradients across microbatches](https://recursiveai.org/onemodel/cookbook/accumulate-gradients-across-microbatches/)

## The short answer

Wrap the forward pass **and** `Backward()` in an `ActivationScope` for each step. When the scope closes, OneModel returns the temporary tensor buffers it collected to their pools. Leaf gradients belong to the tensors they differentiate, not the scope. In the example, the gradient of the original trainable weight is still `6` when you read it outside the block.

## How do you reduce autograd memory usage during training?

Use one scope per step, not one around the whole training loop. The scope collects tensors allocated by eager `Ops` calls and copies made by `Contiguous` or a copying `Reshape`. The backward engine handles its own temporaries. Tensors you create directly aren’t automatically collected, and views share storage rather than extending the life of their source. Keep parameters and inputs you intend to reuse outside the scope. See [the ActivationScope contract](https://recursiveai.org/onemodel/docs/autograd/#3-memory-activationscope).

For a metric such as loss, read its scalar value into a `float` while the scope is open. That leaves you with the number, not another live tensor. If you need the tensor itself, use `Keep` as shown above. In nested scopes, `Keep` moves a result to the enclosing scope, not past both scopes. Scopes are thread-local, so keep the enclosed work and disposal on the same thread rather than crossing an `await` that can resume on another thread.

## Why doesn’t memory drop after backward?

There are two separate lifetimes. By default `Backward()` releases the graph’s saved records as it uses them. Disposing the activation scope returns the buffers for the tracked forward outputs. Returning a buffer to the pool makes it available for reuse; it doesn’t necessarily free it back to the OS. It also doesn’t clear a parameter’s gradient. Call `Backprop.ZeroGrad` to reuse that gradient buffer for a new step, or `Backprop.ClearGrad` to release it.

Compare `OutstandingBytes` at the **same point** between steps: it’s the capacity currently rented from the pool, not the process’s memory usage. `NativeAllocations` counts rents that had to allocate because no matching idle block was available. In this fixed-shape example neither counter increased after warm-up, though the step still creates managed objects for autograd records. Other workloads and changes in tensor shape can require new buffers.
