Recursive AI

Freeze neural network parameters in C#

OneModel is a free, open-source neural-network library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

Freeze selected weights before the forward pass while other parameters train.

Terminal
dotnet add package RecursiveAI.OneModel.Training

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using RecursiveAI.OneModel.Training;
 
using var model = new Linear(
    1, 1, new Generator(42));
model.Freeze("weight");
using var optimizer = new Sgd(
    model.Parameters(), learningRate: 0.1f);
Console.WriteLine(model.IsFrozen("weight"));
Text
True

Unzip the example, then dotnet run.

The full program

The snippet at the top freezes weight and creates an optimizer that also knows about it. The full CPU program takes one training step on the inline pair x = 1, target = 5, then compares the weight and bias with their earlier values.

C#
using RecursiveAI.OneModel.Tensors;
using RecursiveAI.OneModel.Training;
 
using var model = new Linear(1, 1, new Generator(42));
model.Freeze("weight");
Console.WriteLine($"Frozen weight: {model.IsFrozen("weight")}");
Console.WriteLine($"Trainable: {model.TrainableParameters().Count()}");
using var optimizer = new Sgd(model.Parameters(), learningRate: 0.1f);
using var input = Tensor.FromSpan<float>([1], 1, 1);
using var target = Tensor.FromSpan<float>([5], 1, 1);
float weightBefore = model.Weight.Item<float>();
float biasBefore = model.Bias.Item<float>();
using (var scope = new ActivationScope())
{
    optimizer.ZeroGrad();
    var loss = Losses.Mse(model.Forward(input), target);
    loss.Backward();
    optimizer.Step();
}
Console.WriteLine($"Weight unchanged: {model.Weight.Item<float>() == weightBefore}");
Console.WriteLine($"Bias changed: {model.Bias.Item<float>() != biasBefore}");
Console.WriteLine($"Saved tensors: {model.StateDict().Count}");

dotnet run prints:

Text
Frozen weight: True
Trainable: 1
Weight unchanged: True
Bias changed: True
Saved tensors: 2

The frozen weight stays in the model’s saved state; it is not removed from the model.

Variations

Unfreeze for a later training step

Append this to the full program. The existing optimizer was built over all parameters, so it still includes the weight when training resumes.

C#
model.Unfreeze("weight");
Console.WriteLine($"Frozen weight: {model.IsFrozen("weight")}");
Console.WriteLine($"Trainable: {model.TrainableParameters().Count()}");
Text
Frozen weight: False
Trainable: 2

The short answer

Call model.Freeze("weight") before the forward pass. OneModel stops gradient tracking for that parameter, and its optimizers skip it. Other parameters remain trainable. The CPU example takes a real optimizer step and prints that the weight is unchanged while the bias changed.

Use the RecursiveAI.OneModel.Training package for these module and optimizer APIs. The input is synthetic, so the download needs no model file, data set or network service. The example freezes one parameter of a tiny linear layer; the same module APIs accept full module paths in larger models.

How do you freeze neural network parameters in C#?

Use Freeze() for every parameter in a module, or pass a name or module prefix to freeze a subset. A full name selects one parameter. A module path selects parameters beneath it, matching whole path segments. A predicate overload is available when selection needs more than a name or prefix. Use IsFrozen(name) to inspect a particular parameter’s state.

Freeze before computing the forward pass. Operations record backward information according to which inputs require gradients when they run. Changing a flag after forward is not a substitute for building the forward pass with the intended trainable parameters.

The training documentation specifies that freezing frees an existing dense or row-sparse gradient and backward does not create another for the frozen leaf. TrainableParameters() excludes it. NamedParameters(), StateDict(), state loading and checkpoints still include it. Freezing controls training; it does not shrink the saved model or delete weights.

What is the difference between freezing and turning off a layer?

A frozen layer still computes its forward result. Its weights are held fixed rather than removed from the computation. Downstream trainable parameters can still use that result. The sample illustrates this distinction within one layer: the weight contributes to the prediction, while only the bias receives an update.

Evaluation mode is a different control. Eval() changes the module’s training mode and propagates it to children. For example, the documented dropout behavior is identity in evaluation mode, and batch normalization uses training or evaluation behavior according to its mode. Calling Eval() is not the parameter-freezing API.

Choose freezing when weights must stay fixed. Choose training or evaluation mode for the forward behavior you need. A model can contain frozen parameters while remaining in training mode for the parts you are adapting.

How does unfreezing work in transfer learning?

Call the matching Unfreeze overload before the next forward pass. If your optimizer was built over Parameters(), it already holds the parameter and sees its new trainable state. If it was built from TrainableParameters() while that weight was frozen, the excluded parameter is not in that optimizer’s list; build an optimizer that includes it when expanding the trainable set.

Optimizer state needs a separate decision when resuming training. The documentation says state for a parameter frozen at optimizer construction starts empty. Its first update after unfreezing creates zeroed state. Freezing a parameter that already has optimizer state keeps that state untouched while frozen. For Adam, newly created moments still use the optimizer’s global step number, not a fresh parameter-specific training run.

Checkpoints retain the saved optimizer state shapes. To reproduce an uninterrupted run after loading, resume with parameters frozen the same way as at save time. Do not assume that the weight’s presence in a checkpoint means it was trainable: frozen parameters remain members of the saved model state.

All onemodel recipes