Exact mode: the same bits everywhere

Run a model in OneModel’s exact mode for results that are bit-identical on every OS, CPU, instruction set, thread count and WebAssembly, and see how it differs from the default fast mode.

The model is wide.onnx from the sample models: 256 inputs, a 512-wide hidden layer and 64 outputs. The program runs it in both modes at three thread counts and hashes the output bytes:

using System.Security.Cryptography;
using RecursiveAI.OneModel;
using RecursiveAI.OneModel.Graph;
 
float[] input = new float[256];
for (int i = 0; i < input.Length; i++) input[i] = MathF.Sin(i * 0.37f);
 
foreach (var mode in new[] { ExecutionMode.Fast, ExecutionMode.Exact })
{
    foreach (int threads in new[] { 1, 4, 16 })
    {
        var options = new SessionOptions { Mode = mode, IntraOpThreads = threads };
        using var model = Model.Load("wide.onnx", options);
        float[] y = model.Run(input);
        string hash = Convert.ToHexString(SHA256.HashData(MemoryMarshalBytes(y)))[..16];
        Console.WriteLine($"{mode,-5} threads={threads,-2} y[0]={y[0]:R}  sha256={hash}");
    }
}
 
static byte[] MemoryMarshalBytes(float[] v) =>
    System.Runtime.InteropServices.MemoryMarshal.AsBytes(v.AsSpan()).ToArray();

Fast  threads=1  y[0]=-0.4711962  sha256=53778A7B3BF644BF
Fast  threads=4  y[0]=-0.4711962  sha256=53778A7B3BF644BF
Fast  threads=16 y[0]=-0.4711962  sha256=53778A7B3BF644BF
Exact threads=1  y[0]=-0.47119626  sha256=FF8C37256201B5B4
Exact threads=4  y[0]=-0.47119626  sha256=FF8C37256201B5B4
Exact threads=16 y[0]=-0.47119626  sha256=FF8C37256201B5B4

What the two modes promise

• Fast (the default) is deterministic on one machine and kernel path: the thread count never changes a result, because reductions use fixed chunk sizes. Another CPU, with other kernels or fused multiply-add, can round differently in the last bits.
• Exact gives the same bits everywhere. Every exact kernel uses only IEEE + - * / and square root, one accumulation order per output, no fused multiply-add, OneModel’s own exp, log, tanh and erf, and one canonical NaN. The rules are in exact mode, and the cross-platform check (Windows, Linux and macOS on x64 and arm64, JIT and NativeAOT, and browser WebAssembly) in golden hashes.

The two modes agree here to seven digits, but they are not the same bits: choose one and keep it for every machine that has to agree.

Good to know

• Exact mode costs time: on the float32 reference models it takes up to about 2.7 times fast mode’s time, mostly because fused rewrites don’t run and the matrix product can’t fuse multiply and add (golden hashes). That is why fast is the default.
• An operator with no exact kernel makes Model.Load fail in exact mode with SessionException (ExactModeUnavailable) naming it, rather than silently approximating: see When a model won’t load.
• Graph passes that would change rounding are skipped in exact mode (graph passes), so OptimizationLevel never moves an exact result.