---
title: "Nemotron 3 Diarization"
url: https://recursiveai.org/onespeech/docs/nemotron-diarization/
description: "Speech recognition for .NET. OneSpeech.Diarization.NemotronDiarizer is an explicitly selected, overlap-aware, tier-2 file diarizer for up to eight…"
version: "1.0.0"
source: https://github.com/recursive-ai-oss/onespeech-csharp/blob/63002de486786163a0a7800a51811e76d511fae6/docs/nemotron-diarization.md
---

# Nemotron 3 Diarization

Speech recognition for .NET. `OneSpeech.Diarization.NemotronDiarizer` is an explicitly selected,
overlap-aware, tier-2 file diarizer for up to eight anonymous speakers. It is never a default.
The [clustering documentation](https://recursiveai.org/onespeech/docs/diarization/#voice-clustering-and-overlap-limitation) shows the
one-call pipeline replacement. This is diarization, not source separation or identity recognition.

## Checkpoint and loading

Only `nvidia/Nemotron-3-Diarization` revision
`f667ed73aee57d40cc39428eb768b4fd87a0a29e` is accepted. The publisher's float32
`model.safetensors` is 396,954,592 bytes, SHA-256
`c074d86335b3b794f8fa5edc25594558f128bdb3914d27806a3a5a2e44963cb6`.
`LoadAsync` uses `ModelStore` to fetch and verify this file and build runtime graphs locally,
under the model-cache lock. `Load(ResolvedModel)` loads an already resolved cache only.
Local runtime hashes are fixed registrations, not download or mirror URLs. There is no
executable publisher serialization, remote Python, redistributed tier-2 graph, or default selection.
Publisher origin and the full OpenMDW-1.1 agreement remain in `ModelInfo`.

Instances borrow no audio beyond processing and own disposable CPU sessions. Do not process
concurrently on one instance. `DiarizeBlocks` accepts consecutive 16 kHz blocks and averages
channels; `AudioFileSource` supplies decoded/resampled file blocks. The transcription pipeline
makes a diarization pass and a recognition pass, so it does not keep the waveform in memory.

## Features and graph

The pinned card/config specifies a 31-layer, width-512, eight-head Transformer with pre-block
LayerNorm, exact-form GELU, RoPE and a final LayerNorm. It is not a FastConformer encoder.
Eight successive 128-bin mel frames are stacked and projected to width 512. Features use the
shared deterministic NeMo log-mel extractor without utterance normalization or dither: 16 kHz,
25 ms symmetric Hann window, 10 ms hop, 512-point centered, zero-padded STFT, preemphasis
0.97, Slaney filters and natural-log energies with an additive 2^-24 guard. The centered trailing
frame is retained but masked to zero. Subsampling padding is zero; an encoder group containing
only padding is excluded as an attention key. Inputs shorter than one complete 10 ms hop yield
no turns.

The classifier projects 512 to 192, convolves over time with kernel 3/padding 1 and 1536 output
channels, and rearranges factor-major channels to eight 10 ms frames. ReLU precedes the dense
192-to-192 projection and follows it; the last projection produces eight activity logits. Sigmoid
is applied independently, so simultaneous speakers remain present. Head ordering and activation
placement were checked through the released public bare-model API, not implementation source.

Parity probes use float32 and OneModel fast arithmetic, not its cross-platform bit-identical exact
mode. The targeted short feature/encoder/probability probes use max-absolute tolerances 1e-4.
Long-recording quality is checked by DER/JER rather than promising token- or probability-bit identity
through discrete cache selection.

## Bounded chunking and speaker cache

The offline-style card profile is fixed: speaker cache 264, FIFO 40, new chunk 340,
right context 40, cache update period 300, in 80 ms encoder-frame units. A model window is at
most 684 frames. RoPE positions restart at zero for each model window. The waveform buffer holds
30.4 s plus small STFT edge context; carried left-edge samples make features independent of caller
block boundaries. No VAD removes gaps before diarization: the original audio clock is preserved.

Following Streaming Sortformer §§3.1–3.3, the cache stores projected input frames, not Transformer
KV state. Activity is averaged over each group of eight output frames. Frame scores use
`log(max(0.25, p[s])) + sum(log(max(0.25, 1-p[j])))` for other speakers, with scores disabled below
p[s]=0.5. If a speaker has at least 16 frames whose joint exclusive activity exceeds 0.5,
its nonexclusive frames are excluded before bonuses. Recency adds 0.05 starting at the nominal
264-frame cache boundary, including the first compression when the cache was empty. These
boundary/filter semantics were established with crafted public-cache probes. Per-speaker representation boosts are 24 frames
by 2 log(2), and 48 frames by log(2). Strong selection precedes weak selection, which uses the already boosted scores. A partition-based
TopK includes one infinite-score learned-silence candidate per speaker in the global 264-slot
selection; its treatment of tied scores matches the released CPU oracle on crafted cache probes.
Selected non-silence slots are grouped by speaker and original frame order. Cached frames retain
their original activity scores rather than being relabelled by later chunks, as verified by a
two-update probe switching the active speaker. Learned silence follows represented speakers; unused/invalid slots sort to
the tail. This silence placement was checked with both crafted cache inputs and observed public
cache outputs; placing unused slots in front changes arrival-order identities.

Only bounded audio/features, FIFO/cache and two cached execution plans are retained. Returned
turn/change-point metadata necessarily grows with conversation activity; no duration-independent
bound is claimed for the returned document itself. Embeddings are temporary frame features and
are never persisted by the library.

## Turns, overlap and word assignment

A speaker is active at probability ≥0.5 on the 10 ms grid. Adjacent active frames coalesce per
speaker; intervals may overlap. Endpoints are clipped to actual audio duration. Empty audio
returns no turns. Nonfinite PCM is rejected and other rates require resampling.

`DiarizeActive`/`DiarizeActiveBlocks` additionally retain changes in the highest-probability active
speaker. Relative activity is 1 for that channel and 0.5 for other simultaneous channels, not a
calibrated probability. Exact ties choose the lower channel index. The pipeline uses this view to
apply its ordinary majority-covered-duration rule, `Overlap` flag and `Unassigned` gaps, without
changing words or timing. The returned transcript records the tier-2 model and licence provenance.

## Reference evaluation

The reference is the released Transformers **5.22.2** public Nemotron diarization API, float32,
CPU, batch size one, using the same publisher safetensors and offline-style profile. Oracle
runners are isolated under `tools/oracles/nemotron` with a locked `uv` environment. Their console
output is discarded; only bounded statuses and explicit output artifacts are exposed. NeMo 3.0.0
cannot load this publisher checkpoint's RoPE encoder; its sanitized repro is retained under
`tools/oracles/nemo`. No unreleased source build is substituted.

The four targeted comparisons use AMI ES2004a/ES2004b Mix-Headset and AliMeeting Eval
R8001_M8004_MS801/R8003_M8001_MS801 far-field channel 1, with the existing preparation/reference
labels, no collar, and overlap scored. Both hypotheses are scored by `DiarizationMetrics` in
`onespeech-eval`; a per-file comparison uses a 0.5 percentage-point bound, not an utterance bootstrap.
These labels differ from the publisher card's forced-aligned benchmark protocol, so card benchmark
numbers are not claimed as reproduced.