Hash matrix

The compressed bytes of every codec depend only on the input and the options. The hash matrix checks that: it runs the same list of entries in every column below, writes a manifest of SHA-256 digests per column, and a compare step names the first entry that differs.

Entries

An entry is (codec, option set, input) -> bytes, named codec/options/input. Its digest is the SHA-256 of the output bytes; the manifest also records the output length. An entry that throws gets the digest ERROR:<exception type>, so a failure in one column shows as a mismatch against the others. Entries are listed in ordinal (byte-wise) name order.

Entries receive a context with a Threads count. An entry for a multithreaded codec passes it on as the codec's thread option; its output must be the same for every value.

Inputs

Generated by committed code from SplitMix64 streams with fixed seeds (no System.Random, no files):

An entry registers for all of them unless it names a subset.

Columns

Every manifest must equal the reference, x64-jit. Manifests are named <prefix>-<column>.tsv.

Manifest format

Tab-separated, UTF-8, \n line ends. Lines starting with # are the header (column name, framework, OS, architecture, thread count, vector widths, entry count); the compare step ignores them. Each other line is entry <TAB> sha256-hex-lowercase <TAB> output-length.

Commands

All scripts live in tools/hashmatrix/ and take paths and hosts as arguments.

• columns.sh --out DIR --prefix NAME [--columns jit,aot,scalar,noavx,threadsN,wasm] runs columns on this machine. Add --include-opt-in to include opt-in entries.
• remote.sh --target user@host --prefix NAME --out DIR [--columns ...] copies the working tree to another machine over ssh, runs columns.sh there, and fetches the manifests. Extra ssh options go in HASHMATRIX_SSH_OPTS. For the arm64 host: --prefix arm64 --columns jit,aot,scalar,threadsN.
• compare.sh DIR compares every *.tsv in DIR against x64-jit.tsv, prints equal or the first differing entry per manifest (with both digests), and exits 1 on any mismatch.
• install-dotnet.sh [dir] and tools/node/install.sh [dir] install the pinned .NET SDK and Node.js user-locally; both skip an install that already matches the pin. Node is needed only for wasm.

The matrix is not a timing run. The threadsN column briefly uses every core; on a shared machine, run it when no benchmark holds the machine.

Registering entries

The host is tools/hashmatrix/Host (JIT and NativeAOT) with tools/hashmatrix/Wasm compiling the same sources for browser-wasm. A codec adds one file under tools/hashmatrix/Host/Entries/ with a module initializer; the runner is never edited:

namespace RecursiveAI.HashMatrix;
 
internal static class FooEntries
{
    [ModuleInitializer]
    internal static void Register()
    {
        for (int level = 1; level <= 9; level++)
        {
            int l = level;
            Matrix.Add("foo", $"level{l}", (input, ctx) => Foo.Compress(input, new FooOptions { Level = l, Threads = ctx.Threads }));
        }
    }
}

Matrix.Add(codec, options, run, inputs...) registers one entry per input (all standard inputs when none are named). Matrix.AddOptIn registers entries that run only with --include-opt-in; the divergent entry (it reads the process architecture and vector width) is one, and exists to show that the compare step fails when output depends on the machine. Registering the same name twice is an error.

Option sets that the codec's options object can express belong in the matrix: every level, and every option that changes the stream (dictionary, window size, checksum, block size, filters, job size).

Entries a platform cannot run

An entry that throws PlatformNotSupportedException gets the digest ERROR:PlatformNotSupportedException. The run does not count it as a failure, and the compare step reports it as "not run" instead of a mismatch when the reference column produced a digest. Today this is the encrypted zip and 7z entries (zip-encrypt, sevenzip-encrypt) in the wasm column, because the browser has no AES; the encrypt-primitive entries show which primitive each platform lacks. Encrypted entries pass RandomBytes to fix the salt or IV.