Copy weights and buffers by name without writing a file. Buffers hold state the optimizer doesn’t train, such as a layer’s running mean (running_mean).
dotnet add package RecursiveAI.OneModel.TrainingRequires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.
using RecursiveAI.OneModel.Training; using var source = new BatchNorm2d(1); using var copy = new BatchNorm2d(1); source.Weight.MutableData<float>()[0] = 3; source.RunningMean.MutableData<float>()[0] = 2; copy.LoadStateDict(source.StateDict()); Console.WriteLine(copy.Weight.Data<float>()[0]); Console.WriteLine(copy.RunningMean.Data<float>()[0]);
Unzip the example, then dotnet run.
Copies parameters and buffers; prints the copied weight and running mean:
3 2
Full program
A one-channel normalization layer sees the batch [2, 6]; its running mean moves from zero to two. The program sets the layer’s weight to three and copies both values. It then changes the source weight again: the destination still prints three. The layer is child 0 of a Sequential module, so each name has a 0. prefix.
using RecursiveAI.OneModel.Tensors; using RecursiveAI.OneModel.Training; var sourceNorm = new BatchNorm2d(1, momentum: 0.5f); using var source = new Sequential(sourceNorm); using var destination = new Sequential(new BatchNorm2d(1, momentum: 0.5f)); sourceNorm.Train(true); using var batch = Tensor.FromSpan<float>([2, 6], 2, 1, 1, 1); using (var scope = new ActivationScope()) source.Forward(batch); // Mean 4 updates the running mean to 2. var state = source.StateDict(); sourceNorm.Weight.MutableData<float>()[0] = 3; destination.LoadStateDict(state); sourceNorm.Weight.MutableData<float>()[0] = 9; foreach (var (name, tensor) in destination.StateDict()) Console.WriteLine($"{name}: {tensor.Data<float>()[0]}"); using var differentShape = new Sequential(new BatchNorm2d(2)); using var differentNames = new BatchNorm2d(1); // No "0." prefix. TryCopy(differentShape, "Different shape"); TryCopy(differentNames, "Different names"); void TryCopy(NnModule target, string label) { try { target.LoadStateDict(state); } catch (ArgumentException) { Console.WriteLine($"{label}: rejected"); return; } throw new InvalidOperationException($"{label}: unexpectedly accepted"); }
0.weight: 3 0.bias: 0 0.running_mean: 2 0.running_var: 4.5 Different shape: rejected Different names: rejected
LoadStateDict throws ArgumentException before copying anything if a name is missing, unexpected, or duplicated, or if a tensor’s type or shape differs.
Variations
Copy from a list in a different order. In the full program, replace the first LoadStateDict call. Matching is by name, so the destination prints the same values in its original order.
destination.LoadStateDict(state.AsEnumerable().Reverse());Related recipes
The short answer
Call copy.LoadStateDict(source.StateDict()) on a second module with matching state names, types, and shapes. The load copies values into the destination’s existing tensors. You can then update the source without changing the copied values in the destination.
What is a model state dictionary?
In OneModel it’s a list of (Name, Tensor) pairs. It contains parameters, which an optimizer can update, and buffers, which the module maintains without gradients. Batch normalization is a useful example: the weight and bias are parameters, while running_mean and running_var are buffers. Copying only the parameters would leave the destination using its own statistics in evaluation mode.
Names describe where the tensor lives, not just its role. When the normalization layer is under child 0, its weight is 0.weight, not weight. A bare normalization layer can hold tensors with the same shapes and still fail to load the wrapped layer’s list. Conversely, increasing the wrapped layer from one channel to two keeps the names the same but changes the shapes. Build the destination with the same structure and dimensions when you intend to copy the entire state.
Does a state dictionary change when the model trains?
The list returned by StateDict() holds references to the source tensors. If an optimizer updates a weight after you obtain the list, loading that list later will copy the updated value, not the earlier one. Copy to another matching module as soon as you want to capture the values. Keeping that module separate lets you continue training the source without losing the earlier values.
Loading leaves the destination tensor objects in place. As the training reference puts it, “identity, gradients and optimizer references stay valid.” That also means this operation doesn’t reset an existing destination optimizer’s history or copy the source’s gradient values. If you’re replacing the destination weights to start a fresh run, use a fresh optimizer. If you need to continue the source run later, save the optimizer and progress in a checkpoint instead.