Dropout randomly zeros values. Use Eval() to turn it off for validation and Train() to turn it back on.
dotnet add package RecursiveAI.OneModel.TrainingRequires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.
using RecursiveAI.OneModel.Tensors; using RecursiveAI.OneModel.Training; var dropout = new Dropout(0.5f, seed: 42); using var model = new Sequential(dropout); using var input = Tensor.FromSpan<float>( [1, 1, 1, 1], Shape.Create(4)); using var scope = new ActivationScope(); Show("Train", model.Train()); Show("Eval", model.Eval()); Show("Train", model.Train()); void Show(string mode, NnModule network) { var values = network.Forward(input) .ToArray<float>(); Console.WriteLine($"{mode}: " + string.Join(", ", values)); }
Unzip the example, then dotnet run.
Train: 2, 0, 0, 0 Eval: 1, 1, 1, 1 Train: 0, 2, 0, 0
The snippet uses ActivationScope to return the temporary tensors made by dropout to the pool when the snippet finishes.
The full program
The download prints the child’s training flag after switching back to Train() and counts two masks for three passes. It then uses the same layer and two SGD updates to fit four trainable values toward zero.
using RecursiveAI.OneModel.Tensors; using RecursiveAI.OneModel.Training; var dropout = new Dropout(0.5f, seed: 42); using var model = new Sequential(dropout); using var input = Tensor.FromSpan<float>( [1, 1, 1, 1], Shape.Create(4)); using var scope = new ActivationScope(); Show("Train", model.Train().Forward(input)); Show("Eval", model.Eval().Forward(input)); Show("Train", model.Train().Forward(input)); Console.WriteLine($"Child training: {dropout.IsTraining}"); Console.WriteLine($"Masks drawn: {dropout.Draws}"); // Fit four trainable values toward zero using the same dropout layer. using var weights = input.Clone().WithRequiresGrad(); using var target = Tensor.Zeros(DType.F32, 4); using var optimizer = new Sgd([weights], 0.125f); for (int step = 0; step < 2; step++) { using var training = new ActivationScope(); var loss = Losses.Mse(model.Forward(weights), target); loss.Backward(); optimizer.Step(); Backprop.ZeroGrad(weights); } Console.WriteLine($"Optimizer steps: {optimizer.StepCount}"); void Show(string mode, Tensor output) => Console.WriteLine($"{mode}: " + string.Join(", ", output.ToArray<float>()));
Train: 2, 0, 0, 0 Eval: 1, 1, 1, 1 Train: 0, 2, 0, 0 Child training: True Masks drawn: 2 Optimizer steps: 2
If you forget to call Train() before the next training forward pass, the dropout layer keeps returning its input unchanged.
Variations
Leave dropout off in both modes. Replace the Dropout constructor in the snippet at the top. All three lines then show 1, 1, 1, 1:
var dropout = new Dropout(0f, seed: 42);Related recipes
The short answer
Call model.Eval() before the validation forward pass and model.Train() before the next training forward pass. On a Sequential, those calls set IsTraining on the container and every child, so you don’t need to find each Dropout layer. The training reference describes how the flag propagates.
What is dropout in a neural network?
Dropout is a training operation that randomly replaces some values with zero. In OneModel, each training call with a nonzero drop probability draws a new mask, a set of multipliers that specifies which entries to keep. The probability applies per element, not as a quota for the whole tensor. In the example, p = 0.5 and three of the four values drop out on each of the two training passes. That doesn’t mean the layer is set to drop 75 percent of the input; it’s the outcome of those two draws on a small tensor.
What is the difference between inverted dropout and dropout?
OneModel uses inverted dropout: it multiplies kept values by 1 / (1 − p) during training. That’s why an input of one can turn into two when p = 0.5. With the scale applied during training, the expected value of an entry stays the same, and evaluation doesn’t need to rescale anything. In Eval(), OneModel returns the input tensor itself, not a copy. If you’re managing tensor lifetimes by hand, don’t dispose that result as though it owned storage separate from the input. See the mask and scaling rules.
Do you switch back to training mode after validation?
Yes. Changing the mode after Forward() doesn’t undo the mask used for that pass. Set the mode before the forward pass whose result you intend to use. Eval() doesn’t switch off gradient recording; it changes how modules such as dropout behave. The autograd reference describes recording separately.
Switching back to Train() also doesn’t reseed dropout. When dropout is on, each training pass draws a new mask, but evaluation passes don’t use one up. This is why the full program reports two draws, not three, and why the second training output uses a different mask even though you didn’t change the seed. If you save and restore module state, the saved draws buffer lets the same seed resume at the next mask.