# onespeech docs, all pages (OneSpeech 1.0.0)

Every page of https://recursiveai.org/onespeech/docs/ as markdown, getting started first. Index: https://recursiveai.org/onespeech/llms.txt


<!-- https://recursiveai.org/onespeech/docs/getting-started/ -->

# Getting started with OneSpeech

Install OneSpeech for .NET 10, transcribe a recording with timed words and speaker turns, run a live session from packaged models with no network, and use the `onespeech` command-line tool. Every program below was run on the 1.0.0 packages from NuGet, and the output shown is what it printed.

## Install

OneSpeech targets `net10.0`, so you need the .NET 10 SDK.

```shell
dotnet new console -o SpeechStart
cd SpeechStart
dotnet add package RecursiveAI.OneSpeech
```

The core package depends on `RecursiveAI.OneAudio` (decoding and resampling) and `RecursiveAI.OneModel` (inference) only, and none of the three contains native code. Silero VAD is embedded in it. Every other model either comes in its own package or downloads on first use into a per-user cache, where it is verified against the catalog’s size and SHA-256.

The samples use a LibriSpeech test-clean recording that ships in the OneSpeech repository as `testdata/1089-134686-0001.flac` (CC BY 4.0; about three seconds of one speaker). LibriSpeech’s reference text is “stuff it into you his belly counselled him”.

## Transcribe a file

`Program.cs`:

```csharp
using OneSpeech;

Transcript transcript = Speech.Transcribe("1089-134686-0001.flac", language: "en");

Console.WriteLine(transcript.Text);
Console.WriteLine($"{transcript.Language}, {transcript.Duration.TotalSeconds:0.00} s");

foreach (Word word in transcript.Words.Take(6))
    Console.WriteLine($"{word.Start?.TotalSeconds:0.00}–{word.End?.TotalSeconds:0.00}  {word.Text}  ({word.TimingMethod})");

foreach (Turn turn in transcript.Turns)
    Console.WriteLine($"{turn.Speaker.Id}: {turn.Start.TotalSeconds:0.00}–{turn.End.TotalSeconds:0.00}");

foreach (ModelInfo model in transcript.Models)
    Console.WriteLine($"{model.Id} ({model.License})");

File.WriteAllText("transcript.json", transcript.ToJson());
```

`dotnet run` prints:

```text
 stuffered into you, his belly counseled him.
en, 3.28 s
0.55–0.87   stuffered  (ForcedAligner)
0.87–1.19   into  (ForcedAligner)
1.19–1.43   you,  (ForcedAligner)
1.59–1.75   his  (ForcedAligner)
1.75–2.15   belly  (ForcedAligner)
2.15–2.63   counseled  (ForcedAligner)
speaker-1: 0.20–3.09
silero/vad-v6.2.3 (MIT)
openai/whisper-large-v3-turbo (MIT)
Qwen/Qwen3-ForcedAligner-0.6B (Apache-2.0)
iic/speech_campplus_sv_en_voxceleb_16k (Apache-2.0)
```

This is the default file model’s real output, mistake included: Whisper large-v3-turbo heard “stuff it” as “stuffered”. Word text keeps Whisper’s leading space.

`Models` lists everything that took part. The `model` argument selects one role, not the whole pipeline: turbo recognized the words, Qwen3-ForcedAligner timed them, and CAM++ English labeled the speaker. The first run downloaded and converted all of them, several GB, and took about two minutes; later runs reuse the cache. To prepare a machine ahead of time, call `await Speech.PrefetchAsync(language: "en", model: "openai/whisper-large-v3-turbo")`. [First-use models](https://recursiveai.org/onespeech/docs/model-downloads/) lists every download by verb.

A speaker label such as `speaker-1` is anonymous and local to this transcript, not a person’s identity. Words that can’t be timed have a null `Start` and `End`; a time is never invented. `transcript.json` is the [Transcript JSON schema, version 1](https://recursiveai.org/onespeech/docs/transcript-json/).

## A live session, from packages, offline

`Speech.Listen` reads mono 16 kHz PCM that you supply and returns partials, finals, endpoints and speaker turns on one audio clock. It doesn’t capture a microphone: an app pushes its own capture. Here the blocks come from the same file, 100 ms at a time.

The live default, Moonshine streaming tiny, and the CAM++ English speaker embedder each ship as a NuGet package, so this project needs no download at runtime:

```shell
dotnet new console -o SpeechLive
cd SpeechLive
dotnet add package RecursiveAI.OneSpeech
dotnet add package RecursiveAI.OneSpeech.Models.MoonshineStreamingTiny
dotnet add package RecursiveAI.OneSpeech.Models.CamPlusEnglish
```

`Program.cs`:

```csharp
using OneSpeech;
using OneSpeech.Audio;

await foreach (LiveEvent update in Speech.Listen(Blocks("1089-134686-0001.flac"), language: "en"))
{
    switch (update)
    {
        case PartialEvent partial:
            Console.WriteLine($"{partial.AudioTime.TotalSeconds:0.0}s  draft {partial.Revision}: {partial.Segment.Text}");
            break;
        case FinalEvent final:
            Console.WriteLine($"{final.AudioTime.TotalSeconds:0.0}s  final: {final.Segment.Text}");
            break;
        case EndpointEvent endpoint:
            Console.WriteLine($"{endpoint.AudioTime.TotalSeconds:0.0}s  endpoint ({endpoint.Reason})");
            break;
        case SpeakerTurnEvent turn:
            Console.WriteLine($"{turn.AudioTime.TotalSeconds:0.0}s  speaker {turn.Turn.Speaker.Id} {turn.Turn.Start.TotalSeconds:0.00}–{turn.Turn.End.TotalSeconds:0.00}");
            break;
    }
}

static async IAsyncEnumerable<AudioBuffer> Blocks(string path)
{
    using AudioFileSource source = AudioFileSource.Open(path, CancellationToken.None);
    while (true)
    {
        float[] block = new float[1600 * source.Channels]; // 100 ms at 16 kHz
        int read = source.Read(block, CancellationToken.None);
        if (read == 0) yield break;
        yield return new AudioBuffer(block.AsMemory(0, read), AudioFileSource.OutputSampleRate, source.Channels);
        await Task.Yield();
    }
}
```

Run with an empty model cache and no network at all, it prints:

```text
0.9s  draft 1: Duff
1.1s  draft 2: Duff it
1.3s  draft 3: Stuff it into
1.5s  draft 4: Stuff it into you.
1.7s  draft 5: Stuff it into you.
1.9s  draft 6: Stuff it into you, his
2.1s  draft 7: Stuff it into you, his belly.
2.3s  draft 8: Stuff it into you, his belly came.
2.5s  draft 9: Duff it into you, his belly counts.
2.7s  draft 10: Stuff it into you, his belly counselled.
2.9s  draft 11: Stuff it into you, his belly counseled him.
3.1s  speaker speaker-1 0.55–2.05
3.1s  speaker speaker-1 2.05–2.91
3.1s  final: Stuff it into you, his belly counseled him.
3.1s  endpoint (Silence)
```

Each draft replaces the one before it; the final is immutable. `AudioTime` is how much audio had been accepted when the event became available, not a word boundary. On build, the model packages copy their files into `onespeech-models/` beside the executable; keep that directory when you deploy. A model package covers only its own model: `Speech.Transcribe` on files can still need an aligner that isn’t packaged.

## The command-line tool

`RecursiveAI.OneSpeech.Cli` is the `onespeech` .NET tool:

```shell
dotnet new tool-manifest
dotnet tool install RecursiveAI.OneSpeech.Cli
dotnet onespeech transcribe 1089-134686-0001.flac --language en --format srt
```

It prints:

```text
1
00:00:00,546 --> 00:00:02,786
 stuffered into you, his belly counseled
 him.
```

`--format` takes `text`, `json`, `srt` or `vtt`, and `--output` writes a file instead. `onespeech --help` lists `align`, `diarize`, `detect-language`, `models list` and `models pull`, and the speaker commands. For containers such as MP4 and MKV, and for ASS and TTML captions in your own code, add `RecursiveAI.OneSpeech.Media` and call `MediaInput.Register()` once; see [Containers and captions](https://recursiveai.org/onespeech/docs/media/).

## Speakers

The default diarizer labels who spoke when, but it doesn’t detect overlapping speech. `NemotronDiarizer` is an overlap-aware upgrade for up to eight speakers; it is a tier-2 model that loads only from its publisher’s files, by name, never by default. See [Diarization](https://recursiveai.org/onespeech/docs/diarization/).

`Speech.Enroll`, `Speech.Identify` and `Speech.Verify` recognize enrolled voices; enrollments live only in process memory. In the README’s words: “Speaker embeddings are biometric data. Obtain informed consent, limit retention and access, and check applicable law, including Illinois BIPA and GDPR Article 9, before enrollment, identification or verification.” See [Speaker recognition](https://recursiveai.org/onespeech/docs/speaker-recognition/).

## Where next

- [Overview](https://recursiveai.org/onespeech/docs/overview/): the README, with the models by role and their licences.
- [Transcribing files](https://recursiveai.org/onespeech/docs/transcribing-files/) and [Live recognition and file streaming](https://recursiveai.org/onespeech/docs/live-sessions/).
- [Word timing](https://recursiveai.org/onespeech/docs/word-timing/), and [Contextual phrase biasing](https://recursiveai.org/onespeech/docs/biasing/).
- [Model files and pre-fetch](https://recursiveai.org/onespeech/docs/models/) for caches and offline hosts.
- [Measured results and limits](https://recursiveai.org/onespeech/docs/results/): every row, with its reference, interval and verdict.


<!-- https://recursiveai.org/onespeech/docs/overview/ -->

# Speech recognition for .NET

OneSpeech transcribes audio, times words, labels speakers and supports live sessions in managed .NET 10. Open-weight models run on OneModel; OneAudio decodes and resamples audio. No native inference runtime is required.

OneSpeech is unaffiliated with other products or companies named OneSpeech.

See [measured results and limitations](https://recursiveai.org/onespeech/docs/results/) before choosing a model. Licensed under [MIT](https://github.com/recursive-ai-oss/onespeech-csharp/tree/63002de486786163a0a7800a51811e76d511fae6/LICENSE-MIT) OR [Apache-2.0](https://github.com/recursive-ai-oss/onespeech-csharp/tree/63002de486786163a0a7800a51811e76d511fae6/LICENSE-APACHE).

## Transcribe, stream and align

```csharp
using OneSpeech;

Transcript transcript = Speech.Transcribe("meeting.m4a", language: "en");
Console.WriteLine(transcript.Text);
File.WriteAllText("meeting.json", transcript.ToJson());

await foreach (Segment segment in Speech.TranscribeStream("lecture.wav"))
    Console.WriteLine(segment.Text);

Transcript aligned = Speech.Align("short.wav", "The words that were spoken.", language: "en");
```

`Transcribe` returns a timed, diarized, language-tagged `Transcript`: inspect `Words`, `Segments`, `Turns` and `Models`. Failed word timing is **null**, never invented. A speaker label is anonymous and local to the document, not a person's identity. `TranscribeStream` yields final segments in bounded chunks; it does **not** perform whole-file speaker assignment. Standalone `Align` accepts at most 60 seconds. All verbs take an optional final `cancellationToken`; path and encoded-stream I/O have `*Async` forms. CPU inference remains synchronous within those calls. Language tags are case-insensitive: `en-US`, `EN`, `eng` and `English` resolve to `en`; unsupported hints throw `CapabilityNotSupportedException`.

`Speech.Listen` consumes caller-supplied mono 16 kHz PCM as an `IAsyncEnumerable<AudioBuffer>`. It emits replacing partials, immutable finals, endpoints and speaker turns on one audio clock. It does not capture a microphone. See [live sessions](https://recursiveai.org/onespeech/docs/live-sessions/).

For speaker recognition, use `Speech.Enroll("alice", "enrollment.wav")`, `Speech.Identify("query.wav")` and `Speech.Verify("first.wav", "second.wav")`; encoded streams have `*Async` twins. Enrollments live only in process memory: `Speech.ClearSpeakers()` removes them and releases speaker models. File transcription adds matching enrolled names to speaker labels without changing their document-local IDs; unknown clusters keep anonymous labels. Obtain consent and check local biometric law before using these verbs. See [speaker recognition](https://recursiveai.org/onespeech/docs/speaker-recognition/) for calibrated thresholds, limitations and the low-level registry.

### Overlapping speech

**The default clustering diarizer does not detect overlapping speech.** For an overlap-aware upgrade (up to eight speakers), load the named tier-2 diarizer in one call: `await NemotronDiarizer.LoadAsync()`, then pass it to your transcription pipeline. This does not separate simultaneous voices. See the [complete replacement example](https://recursiveai.org/onespeech/docs/diarization/#voice-clustering-and-overlap-limitation) and [Nemotron guide](https://recursiveai.org/onespeech/docs/nemotron-diarization/). It is never downloaded as a default.

## First-use downloads and offline preparation

The `model` argument selects a role, **not the entire pipeline**. Transcription also needs a speaker embedder and, without native timings, companion aligners. Even Whisper tiny can require several GB on first use. Silero VAD is embedded. See the [per-verb download table](https://recursiveai.org/onespeech/docs/model-downloads/) for exact IDs, sizes and conditional dependencies.

`await Speech.PrefetchAsync(language: "en", model: "openai/whisper-tiny");` prepares the complete companion set, including timing fallbacks, without running inference. For live use, pass `model: "moonshine-ai/moonshine-streaming-tiny"`. Every downloadable family exposes `Load(ResolvedModel, …)` and `LoadAsync(store, model, …)`; [composition examples](https://recursiveai.org/onespeech/docs/loading/) use documented role types.

## Contextual biasing

Biasing is a low-level request; `Speech.*` stays unchanged. A loaded recognizer
returns a borrowed wrapper with its own immutable phrase list:

```csharp
using var model = OneSpeech.Whisper.WhisperRecognizer.Load(modelDirectory);
SpeechRecognizer biased = model.WithPhrases([new Phrase("Acme Photonics"), new Phrase("Marisol")]);
Transcript result = biased.Transcribe(audio, "en", cancellationToken);
// The same biased recognizer can be passed to VadTranscriptionPipeline.
```

Keep the owning model alive until the wrappers and their sessions are finished.
Wrappers do not own or dispose it; the model's concurrency limits still apply.
Calling `WithPhrases` again replaces the list; an empty list returns the original
unbiased recognizer. The original recognizer is never modified.

Whisper conditions each window with ordinary prompt tokens. Qwen3-ASR appends
phrases to its existing system context. These prompt methods refuse explicit
numeric weights rather than pretending to apply a score bonus. Parakeet TDT
uses token-level shallow fusion; `new Phrase("Acme", 0.5)` supplies a natural-log
bonus per token (null defaults to 1). Custom CTC/transducer graphs use the same
fusion after the caller supplies `TokenVocabulary.WithPhraseEncoder` for their
exact tokenizer. CTC biasing uses prefix beam search, even if the unbiased
recognizer is greedy. Moonshine does not support biasing: a nonempty request
throws `CapabilityNotSupportedException`, as declared by `Capabilities.Biasing`.

Biasing is not a spelling replacement or a guarantee that a term will be spoken.
It can introduce false alarms. Whisper prompt conditioning in particular can
substantially increase overall errors and repeat absent terms even when recall
of requested terms improves; it is not a safe default. Check both phrase recall and overall WER, with
absent-phrase controls, on your audio. See [contextual scoring and bounds](https://recursiveai.org/onespeech/docs/contextual-biasing/)
for prompt budgets, weights, decoder semantics and the repeatable evaluation.

## Packages

| Package | Purpose |
|---|---|
| `RecursiveAI.OneSpeech` | Recognition, alignment, diarization, speakers, live sessions, algorithms and JSON |
| `RecursiveAI.OneSpeech.Media` | Container input, SRT/VTT/ASS/TTML and caption muxing using OneMedia |
| `RecursiveAI.OneSpeech.Models.CamPlusEnglish` | Optional offline-at-restore CAM++ English weights |
| `RecursiveAI.OneSpeech.Models.MoonshineTiny`, `.MoonshineBase`, `.MoonshineStreamingTiny` | Optional offline-at-restore Moonshine weights |
| `RecursiveAI.OneSpeech.Cli` | The `onespeech` dotnet tool |

Models not supplied by a package download on first use; CPU runtime graphs are built and verified in the per-user cache. An offline model package covers only that model, not every role used by `Transcribe`. See [cache, pre-fetch and offline use](https://recursiveai.org/onespeech/docs/models/) and [CLI and model packages](https://recursiveai.org/onespeech/docs/cli-and-model-packages/).

## Models by role

| Role | Models | Tier / licence | Selection |
|---|---|---|---|
| File recognition | Whisper tiny/base/small/large-v3; large-v3-turbo | 1 / Apache-2.0; turbo MIT | **turbo default** |
| File recognition | Qwen3-ASR 0.6B / 1.7B | 1 / Apache-2.0 | By name; 1.7B accuracy option |
| Live / edge recognition | Moonshine English tiny/base and streaming tiny/small/medium | 1 / MIT | **streaming-tiny live default** |
| Recognition | Parakeet-TDT-0.6B-v3 | 2 / CC-BY-4.0 | By name only |
| Timing | Qwen3-ForcedAligner-0.6B; omniASR-CTC-300M | 1 / Apache-2.0 | Timing policy; see [word timing](https://recursiveai.org/onespeech/docs/word-timing/) |
| Voice activity | Silero VAD 6.2.3 | 1 / MIT | Embedded |
| Diarization | Silero + CAM++ + spectral/AHC clustering | 1 / MIT + Apache-2.0 | Default; no overlap detection |
| Diarization | Nemotron 3 Diarization | 2 / OpenMDW-1.1 | Explicit overlap-aware upgrade |
| Segmentation | pyannote segmentation-3.0 / community-1 | 2 / MIT gated / CC-BY-4.0 gated | Local segmentation-3.0 + our overlap-aware clustering; community-1 reference only |
| Speaker embeddings | CAM++ English/Chinese; ECAPA-TDNN | 1 / Apache-2.0 | CAM++ English default; Chinese for Chinese speech |

Tier 1 is ungated MIT/Apache-2.0/BSD and may be redistributed and selected by default. Tier 2 loads only from pinned publisher files, never from our mirror or model packages, never by default. Local conversion stays in your cache. Licence, revision, file digests and attribution travel in `ModelInfo`. Only the listed licence-cleared Moonshine checkpoints are supported; Community License checkpoints are excluded. See [gated models](https://recursiveai.org/onespeech/docs/tier-2-models/) and [third-party notices](https://recursiveai.org/onespeech/docs/third-party-notices/).

## Privacy and consent

Speaker embeddings are biometric data. Obtain informed consent, limit retention and access, and check applicable law, including **Illinois BIPA** and **GDPR Article 9**, before enrollment, identification or verification. The library does not persist embeddings; applications remain responsible for audio, transcripts, logs and any copies they retain. This is not legal advice.

## Guides

- [Long files](https://recursiveai.org/onespeech/docs/long-files/), [live events](https://recursiveai.org/onespeech/docs/live-sessions/), [word timing](https://recursiveai.org/onespeech/docs/word-timing/)
- [Diarization](https://recursiveai.org/onespeech/docs/diarization/), [speaker recognition](https://recursiveai.org/onespeech/docs/speaker-recognition/), [biasing](https://recursiveai.org/onespeech/docs/biasing/)
- [Custom CTC/transducer graphs](https://recursiveai.org/onespeech/docs/custom-models/), [model cache](https://recursiveai.org/onespeech/docs/models/), [tier-2 access](https://recursiveai.org/onespeech/docs/tier-2-models/)
- [Transcript JSON schema v1](https://recursiveai.org/onespeech/docs/transcript-json/), [containers and captions](https://recursiveai.org/onespeech/docs/media/), [CLI](https://recursiveai.org/onespeech/docs/cli-and-model-packages/)
- Migration: [Whisper.net](https://recursiveai.org/onespeech/docs/migrate-whisper-net/), [sherpa-onnx .NET](https://recursiveai.org/onespeech/docs/migrate-sherpa-onnx/), [Azure Speech SDK](https://recursiveai.org/onespeech/docs/migrate-azure-speech/), [WhisperX concepts](https://recursiveai.org/onespeech/docs/migrate-whisperx/)

## Build and sample checks

Set `ONESPEECH_ARTIFACTS` to a directory outside the checkout. Run `dotnet restore`, then `dotnet build --no-restore`. Run `dotnet run --project tests/DocsSamples --no-build -- .` to exercise the no-download document examples and check local documentation links. Model-backed examples are compile-checked without downloading weights; run them with your own consented audio and fetched models.


<!-- https://recursiveai.org/onespeech/docs/transcribing-files/ -->

# Transcribing files

Speech recognition for .NET. OneSpeech is unaffiliated with other products or companies named OneSpeech.

```csharp
using OneSpeech;

Transcript transcript = Speech.Transcribe("voicemail.m4a");
Console.WriteLine(transcript.Text);
File.WriteAllText("voicemail.json", transcript.ToJson());

Transcript small = Speech.Transcribe("voicemail.wav", language: "en", model: "openai/whisper-tiny");
string language = Speech.DetectLanguage("voicemail.mp3");
```

The file default is Whisper large-v3-turbo. Tiny, base, small and large-v3 can
also be selected by their catalog IDs. Large-v3 requires substantially more
memory than turbo. Selection never silently substitutes a model.
File input is decoded by OneAudio, including AMR and AAC/M4A. Calls accept an
explicit `CancellationToken` overload. Stream input uses
`await Speech.TranscribeAsync(stream, cancellationToken: cancellationToken)` and does not own the
stream. CPU inference is synchronous; first-use downloads can block the file call.

Each call owns and disposes its models. It makes a VAD-based recognition pass,
then releases recognition graphs before a bounded replay pass for non-native
word timing, followed by whole-recording speaker clustering. Timing replay must
reproduce the original chunk boundaries exactly; a mismatch is an error rather
than a guessed word assignment. Native-timing recognizers need no timing replay. PCM is not retained for the whole recording; the returned text,
words and turns necessarily grow with the recording. Words carry confidence and
a timing method. Native timestamps take priority; otherwise Qwen3-ForcedAligner
runs for its eleven covered languages, then omniASR-CTC for remaining words, with
existing DTW as the final fallback. Failed timing remains null. Alignment models
are loaded only when a chunk needs them.
Words outside speaker turns remain unassigned rather than receiving guessed IDs.

## First use and offline deployment

The catalog holds pinned upstream revisions, byte sizes and SHA-256 digests.
Missing tier-1 files download on first use. Until release mirror assets are
available, the library converts verified publisher weights locally, then checks
every runtime graph against its catalog size and digest before installation.
No publisher code executes during conversion. Conversion uses additional temporary
disk space and may take appreciable time for large checkpoints. Runtime files are
installed atomically per file under a cross-process cache lock. Existing verified
files require no network requests. Corrupt cache files are removed and reported;
retry the pull rather than using corrupt data.

Pre-fetch the recognizer and the speaker embedder to avoid network access in the
first transcription:

```text
onespeech models pull openai/whisper-large-v3-turbo
onespeech models pull iic/speech_campplus_sv_en_voxceleb_16k
```

Silero VAD is embedded in the core package. `ONESPEECH_MODEL_CACHE` overrides the
platform cache location. For a host-owned store, assign `ModelStore.Default` once
before starting requests, or use a separate store at the low level:

```csharp
using OneSpeech.Catalog;

ModelStore.Default = new ModelStore(cacheDirectory: cache, source: myModelSource);
```

A `ModelSource` supplies pinned publisher files; size and SHA-256 checks still
apply, and converted runtime files are checked independently. No embeddings or
enrolment audio are persisted by the library.

## Speaker features and consent

The tier-1 diarizer uses Silero VAD, CAM++ English embeddings and spectral
clustering with an AHC numerical fallback. **It does not detect overlapping
speech.** Speaker count estimation and short/changed voices can be wrong;
speaker labels are local to one recording, not verified personal identities.
An overlap-aware tier-2 upgrade is not yet available.

Voice embeddings can be biometric data. Obtain any required informed consent,
choose lawful retention and deletion practices, and assess applicable local law
before enrolling or identifying people. Illinois BIPA and GDPR Article 9 may
apply, among other laws. The library's in-memory operation is not a determination
of legal compliance; this note is not legal advice.


<!-- https://recursiveai.org/onespeech/docs/live-sessions/ -->

# Live recognition and file streaming

A `LiveSession` accepts borrowed mono 16 kHz `AudioBuffer` blocks through
synchronous `Push`, or awaits those blocks through `Listen`. Resampling and
capture belong to the caller. `Complete` flushes real input; it does not append
silence. Normal async end-of-input calls `Complete`. Cancellation and early
iterator disposal do not flush. Sessions are single-use and not thread-safe.
Dispose them before disposing their borrowed models.

## Events and clock

All times are audio times relative to the first sample. The clock is computed
from an integer count of accepted samples, including silence. Event `AudioTime`
is the amount of input accepted when the update became available, not a word
boundary or a wall-clock measurement. A large push can return several events
with the same `AudioTime`. Applications needing finer availability measurements
should push smaller blocks. Every segment/word/turn boundary must lie within
accepted audio. Unknown word boundaries and confidence remain null.

* `PartialEvent` replaces the previous partial for its `UtteranceId`. Revision
  numbers start at one and strictly increase within that utterance.
* `FinalEvent` commits one immutable `Segment`. It cannot be revised. The shape
  of the segment is the same as file-streaming output.
* `EndpointEvent` closes an utterance after its final, if any, increments the
  utterance ID and resets the revision sequence. Its `End` is the retained audio
  boundary, not an inferred word boundary. Reasons are silence, duration limit
  and end-of-input.
* `SpeakerTurnEvent` describes activity separately and does not mutate finals.
  The utterance session does not perform speaker assignment.

The `LiveSession` base class assigns utterance IDs, revisions and availability
clock values. It rejects out-of-clock boundaries, duplicate finals and partials
after a final. A processing failure faults the session: create a new one rather
than retrying an ambiguously consumed block. Input validation and cancellation
before processing do not advance the clock.

## Offline-family utterance mode

`SpeechRecognizer.CreateLiveSession` defaults to `UtteranceLiveSession`, which
borrows a Silero model and creates independent recurrent state. It emits finals
only, not buffered-recompute partials. `Speech.Listen(source)` resolves the
selected catalog model and owns the models/session during enumeration. Model
resolution and source I/O are awaited; in-memory recognition is synchronous.

VAD uses 512-sample frames, carries incomplete frames across pushes and pads
only the last incomplete VAD frame at completion. Padded samples never advance
the clock or enter recognition. Default activation/deactivation thresholds are
0.5/0.35, minimum speech 250 ms, endpoint silence 100 ms, padding 30 ms. Speech
must exceed the minimum duration. Silence has hysteresis. Adjacent utterance
padding never overlaps already finalized audio.

PCM storage is bounded by the maximum utterance length (default 30 s; configurable
1–300 s), padding and two VAD frames. The duration limit closes at the next VAD
frame, at most 32 ms beyond the configured limit. Continuous speech continues
in the next utterance without dropping samples. Explicit completion retains a
trailing silence shorter than the endpoint threshold. Final segment bounds
name the retained utterance; only recognizer-provided word timings are shifted
onto the session clock. No word timing is fabricated from VAD bounds.

Steady-state session buffering reuses arrays, but inference, recognition,
immutable events and output arrays can allocate. Zero-allocation inference is
not claimed.

## File streaming

`Speech.TranscribeStream(path)` lazily loads models and yields final segments
through the same session used for live input, decoding 4096-frame blocks and
averaging channels. For the same decoded mono PCM, file and live modes emit
identical final segment content and boundaries, regardless of push boundaries.
Availability times can differ because different block sizes deliver the
endpoint at different accepted audio times. Early enumeration disposal closes
the decoder and models. Unlike whole-file `Transcribe`, it does not run a
whole-file diarization pass or invent speaker labels. The lower-level `VadTranscriptionPipeline` remains available with its own
chunking policy; whole-file `Transcribe` continues to use that pipeline.
Neither streaming entry point retains the whole recording's PCM.

Moonshine streaming checkpoints add session-local encoder caches and replacing
partials; see [Moonshine](https://recursiveai.org/onespeech/docs/moonshine/). Input samples must be finite. Silero
states reuse their output tensors through OneModel's span-based inference API.

## Allocation accounting

Live working-state allocation and immutable emitted payload allocation are
reported separately. A snapshot includes its event object, result array,
segment, text and words; the caller may retain it indefinitely. These objects
are not pooled. Zero working-state allocation must not be presented as zero
total allocation if a model backend still allocates. Measurements name the
backend version, chunk size and partial cadence, and include backend allocation
separately. A chunk that emits nothing has no payload exemption.

## Consuming updates

This method takes an application-owned PCM source and cancellation token. It does not open an audio device. Keep the current partial separately from committed finals in your UI.

```csharp
static async Task ShowLive(IAsyncEnumerable<OneSpeech.AudioBuffer> pcm,
    CancellationToken cancellationToken)
{
    await foreach (var update in OneSpeech.Speech.Listen(pcm, language: "en", cancellationToken: cancellationToken))
    {
        switch (update)
        {
            case OneSpeech.PartialEvent partial:
                Console.WriteLine($"Draft {partial.Revision}: {partial.Segment.Text}");
                break;
            case OneSpeech.FinalEvent final:
                Console.WriteLine($"Final: {final.Segment.Text}");
                break;
            case OneSpeech.EndpointEvent endpoint:
                Console.WriteLine($"Endpoint at {endpoint.End}: {endpoint.Reason}");
                break;
            case OneSpeech.SpeakerTurnEvent speaker:
                Console.WriteLine(speaker.Turn.Speaker.Id);
                break;
        }
    }
}
```


<!-- https://recursiveai.org/onespeech/docs/contracts/ -->

# Shared contracts

`AudioBuffer` views interleaved float PCM without copying it. Mono is one channel; multiple channels preserve separated-party audio for later channel diarization. Sample rates are positive Hz and buffers contain whole frames; processing implementations document rate/channel support. Float samples conventionally lie in [-1,1]; the buffer does not scan, clip or resample them. Callers must not mutate borrowed samples while processing.

`Transcript`, `Word`, `Segment`, `Turn`, `SpeakerLabel`, `ModelInfo`, and `Capabilities` are immutable records with immutable collection values. Text is explicit rather than guessed from punctuation or token spacing. `SpeakerLabel.Unassigned` is distinct from a null (unprocessed) speaker. Failed timings remain null.

Roles process in-memory audio synchronously and accept cancellation tokens; recognition, alignment, VAD and diarization entry points default them to CancellationToken.None. Recognizers and pipelines also expose synchronous bounded segment enumeration. Asynchronous APIs are reserved for I/O: the `ModelSource` stream hook and the `Speech.TranscribeAsync` input-stream entry point. Models declare support through `Capabilities`; empty languages means no support declared, not unrestricted support. Unsupported capabilities produce a typed error. Typed family options, implementations, live events, and loading policy will be added separately.

`CatalogEntry` and `CatalogFile` describe pinned bytes and their provenance, not loading authorization. `ModelSource` returns caller-owned readable streams; the consuming loader must enforce tier/gate rules, relative-path safety, size, digest, HTTPS and revision policies even for custom sources. `CatalogEntry.FromJson` uses the transcript reader's structural bounds and generated metadata; it is data deserialization only, not semantic catalog verification.

`Speech` forwards to the VAD transcription pipeline through an explicit internal recognizer resolver. Without a configured resolver it fails with `FeatureUnavailableException` before opening input. No recognition or timing is simulated. See [audio input](https://recursiveai.org/onespeech/docs/audio-input/) and [Silero VAD](https://recursiveai.org/onespeech/docs/vad/) for decoding, chunking, audio-clock and lifetime behaviour.

`SpeakerEmbedder` adds synchronous, caller-buffer extraction of speaker vectors, with explicit dimension,
mono sample rate, model provenance and disposal. `InMemorySpeakerRegistry` preserves the abstract
`SpeakerRegistry` surface and adds multi-utterance enrollment; its threshold is explicit. See
[speaker identity and scores](https://recursiveai.org/onespeech/docs/speakers/) for averaging, normalization, calibration and privacy.

## Language hints and argument failures

All entry points share one language rule: trim whitespace, match case-insensitively, accept unambiguous English language names and ISO three-letter aliases, and reduce region/script variants to the model's language-only tag. For example, `en-US`, `EN`, `English`, `eng` and `eng_Latn` become `en`; `zh-Hant-TW` and `Chinese` become `zh`. The output language uses the normalized tag. A hint unknown to the library or unsupported by the selected model throws `CapabilityNotSupportedException`, even on silence. `DetectLanguage` refuses any hint because it determines the language itself. Ordered `Align` refuses an uncovered language; no-model-run is not successful alignment. Speech absent or unalignable text **after an aligner runs** still legitimately has null word timing.

Invalid caller-supplied PCM (rate, channel shape, nonfinite samples) throws `ArgumentException`; decoding corrupt or unsupported **encoded** input throws `UnsupportedAudioException`, with `SourceName` and the path in the message for files. Resample/downmix in-memory PCM with OneAudio before calling a role. `ModelNotFoundException` means an unknown catalog ID/default; `ModelUnavailableException` means a known model/file is absent from the selected publisher or offline source. Model errors carry `ModelId` where available, and gate errors name the publisher repository.

Roles implement `IDisposable`. They release their owned graph/state only; wrappers, live sessions and composed pipelines borrow their collaborators. Dispose a session before disposing its recognizer/detector. All public `Speech` methods place optional `CancellationToken cancellationToken = default` last, after optional `language` and `model`. Awaitable single results use `Async`; segment/event sequences retain `TranscribeStream`/`Listen` in both core and Media. In-memory CPU work is synchronous; async path/stream methods await acquisition/input copying and run inference on the caller's continuation, not `Task.Run`.

See [first-use download dependencies](https://recursiveai.org/onespeech/docs/model-downloads/) and [role loading/composition](https://recursiveai.org/onespeech/docs/loading/). `Speech.PrefetchAsync` prepares all possible companions for the selected recognizer without inference; it may fetch more than one particular input needs.


<!-- https://recursiveai.org/onespeech/docs/transcript-json/ -->

# Transcript JSON, version 1

UTF-8 JSON, written and read using a generated System.Text.Json context. The root object has required `schemaVersion` (integer major version, currently **1**) and required `transcript`. Every other major version is rejected. Unknown fields are rejected; a future additive schema needs an explicit compatibility decision. Duplicate properties follow System.Text.Json's last-value behavior. Property names are case-sensitive.

All times are finite, nonnegative **seconds as JSON numbers**, relative to the beginning of the original input audio. Intervals are start-inclusive/end-exclusive; ends must not precede starts. Sub-tick precision rounds to the nearest .NET TimeSpan tick (100 ns), ties to even. Writers emit decimal seconds computed from integer ticks, preserving every tick on round-trip, including long durations. Values outside the TimeSpan range are rejected. Missing timing is represented by both `start` and `end` being null. No timing is inferred. Nulls are written explicitly. Text is preserved exactly.

## Fields

`transcript`:
- `text`: required string, the complete written output (not reconstructed from words).
- `words`: ordered array of word objects, default `[]`.
- `segments`: ordered array of segment objects, default `[]`.
- `turns`: array of possibly overlapping speaker turns, default `[]`.
- `language`: language tag or null (default).
- `duration`: nonnegative audio duration in seconds, default 0.
- `models`: array of provenance objects, default `[]`.

Word:
- `text`: required string, including any supplied whitespace/punctuation.
- `start`, `end`: number or null (default null), either both present or both null.
- `confidence`: finite number in [0,1], or null (default).
- `timingMethod`: `None` (default), `Native`, `ForcedAligner`, `CtcAligner`, or `Dtw`; numeric representations are accepted only for defined values. Future methods require a schema/library update.
- `speaker`: speaker object or null; null means not processed; `{ "id": "unassigned", "name": null }` means processed but no turn covered the word.
- `overlap`: boolean, false by default.
- `language`: language tag or null (default).

Segment: required `text`; nullable `start`, `end`, `language`; `words` array, default `[]`.

Turn: required `speaker`, `start`, `end`, `text`; `words` array, default `[]`.

Speaker: required nonblank `id`, optional nullable `name`. IDs are local to the document, not biometric identities.

Model provenance: required nonblank `id`, `source` (upstream repo URL), `revision`, `license` (SPDX ID); required `tier` (`Tier1` or `Tier2`, or corresponding numeric 1/2), required `attribution` (exact text). `gated` defaults false. `files` defaults `[]`; each file has required nonblank `name` and a required 64-hex-digit `sha256`. Provenance records do not authorize downloading or redistribution.

## Reader limits and failures

Schema 1 supports **24 hours** of dense speech, with up to **600,000 words** (6.94 words/s over 24 h), **600,000 segments** and **600,000 turns**. Audio times and duration must be between 0 and 86,400 seconds. The wire repeats words inside segments and turns: the budget permits **1,800,000 total word entries across all `words` arrays**, so the root words can each appear once in a segment and once in a turn. Independent limits still apply; unusual verbosity or exceptionally long words can reach the byte budget first.

| Budget | Maximum |
|---|---:|
| Complete UTF-8 document | 1,000,000,000 bytes (1 GB) |
| Nesting depth | 32 |
| Tokens / combined object-array containers | 80,000,000 / 8,000,000 |
| Each words, segments or turns array | 600,000 items |
| Total elements across every array, including metadata | 3,200,000 |
| Each models or model-files array | 256 items |
| Transcript, segment or turn `text` string | 32 MiB raw encoded bytes |
| Any other string or property name | 64 KiB raw encoded bytes |

Raw string sizes include escapes. These budgets accommodate the ordinary 24 h dense document with three word representations rather than treating each copy as a new recording. Catalog JSON retains its separate 8 MiB / 64 KiB limits.

Both string and UTF-8 readers preflight **all wire budgets**, including per-array and aggregate counts, with a forward-only reader before allocating the document graph or decoded strings. The System.Text.Json converter follows the same preflight before source-generated deserialization, without an intermediate DOM. String callers necessarily already own their input; UTF-8 encoding allocation occurs only after exact byte-size validation. Fixed depth bookkeeping is bounded independently of input. Services should also enforce their own request-body limits appropriate to available memory; the library's maximum is not a recommendation to accept 1 GB network requests.

Invalid Unicode, malformed/truncated JSON, required-field omissions, null object/array members, invalid intervals/probabilities/provenance, unsupported versions or exceeded budgets throw `InvalidTranscriptJsonException` (or `JsonException` through the System.Text.Json converter). Empty arrays are allowed. Writers validate shape, count, duration and strings, then count encoded bytes through a bounded scratch writer before allocating the final JSON. The completed wire is preflighted, not deserialized into a second document. Schema version, fields, ordering and numeric encodings are unchanged.

Writers use the same validation and limits; callers should avoid constructing oversized documents. Segment/turn copies of words need not be object-identical to the top-level list. No cross-list identity, ordering, text reconstruction, or speaker attribution is invented by serialization.

## Round-trip example

This no-model example preserves an untimed word and its written form. Null speaker means speaker assignment was not processed; it differs from `SpeakerLabel.Unassigned` after an unsuccessful assignment.

```csharp
var document = new OneSpeech.Transcript
{
    Text = "Hello.",
    Language = "en",
    Words = [new OneSpeech.Word { Text = "Hello.", TimingMethod = OneSpeech.TimingMethod.None }]
};
string json = document.ToJson();
var restored = OneSpeech.Transcript.FromJson(json);
if (restored.Text != document.Text || restored.Words[0].Start is not null)
    throw new InvalidOperationException("Document round-trip changed text or invented timing.");
```

## System.Text.Json and web endpoints

`Transcript` carries a stateless converter. Plain `System.Text.Json` (including Web defaults) writes and reads this same **schema version 1** envelope, seconds and enum names; naming policies cannot produce a second shape. The documented wire format has not changed. Bounds and semantic validation are the same as `ToJson` / `FromJson`. The System.Text.Json adapter bounds the raw value to 8 MiB before a temporary raw JSON DOM is allocated; token, string and depth bounds are then checked before constructing the immutable transcript. Optional arrays omitted on the wire retain their documented empty defaults, including nested word/file arrays; explicit null arrays are refused. Invalid documents are `JsonException` through System.Text.Json and `InvalidTranscriptJsonException` through the explicit methods, with a field/index path when validation fails.

For NativeAOT or trimmed applications, include `Transcript` in the application's source-generated JSON context and use its `JsonTypeInfo` overload, also when registering an ASP.NET JSON resolver. Reflection-based serialization is not required by the library. If an application disables reflection without registering a context, System.Text.Json refuses the operation rather than falling back to another wire format.

```csharp
[JsonSerializable(typeof(Transcript))]
internal sealed partial class AppJsonContext : JsonSerializerContext;
// JsonSerializer.Serialize(transcript, AppJsonContext.Default.Transcript)
```

`Transcript`, `Segment`, `Turn` and `ModelInfo` compare their immutable collections by ordered element value. A valid document equals its JSON round trip and has the same hash code within the process. Equality does not ignore provenance, spacing or null timing.


<!-- https://recursiveai.org/onespeech/docs/migrate-whisper-net/ -->

# From Whisper.net

This is a concept map, not an API-compatible shim. It uses the [published package documentation](https://www.nuget.org/packages/Whisper.net/), consulted 2026-11-08; no implementation source is needed.

| Familiar concept | OneSpeech equivalent |
|---|---|
| Factory loading a model path, then a processor builder | A catalog model ID for `Speech.Transcribe`, or a disposable low-level family recognizer |
| Asynchronously enumerating processor segments | `Speech.TranscribeStream` yields immutable `Segment` values |
| Language builder setting | Optional `language` scalar or typed family options |
| Segment timestamps | Segment boundaries plus per-word timing provenance; missing times are null |
| Runtime/model installation | Core managed package plus verified catalog weights; no native runtime package |

Start with the [README example](https://recursiveai.org/onespeech/docs/overview/#transcribe-stream-and-align). Do not pass a GGML model file to OneSpeech: use its supported publisher files and local runtime conversion. Similar model names do not imply identical decoding policy, timestamps or long-file segmentation. Check the [results](https://recursiveai.org/onespeech/docs/results/) for the exact slices compared.

For complete diarization, use `Transcribe`; segment streaming does not run whole-file speaker clustering. See [long files](https://recursiveai.org/onespeech/docs/long-files/), and keep cancellation and enumeration disposal in the application lifetime. OneSpeech is not affiliated with Whisper.net.


<!-- https://recursiveai.org/onespeech/docs/migrate-sherpa-onnx/ -->

# From sherpa-onnx .NET

This guide maps concepts from the [published C# documentation](https://k2-fsa.github.io/sherpa/onnx/csharp-api/index.html), consulted 2026-11-08. It is not a shim and does not import model configuration objects or native handles.

| Familiar concept | OneSpeech equivalent |
|---|---|
| Offline recognition of a waveform | `Speech.Transcribe` for a file, or `SpeechRecognizer.Transcribe` for PCM |
| Streaming recognition | `LiveSession` or `Speech.Listen` over caller-supplied PCM |
| Separately selected recognition and speaker models | Role interfaces plus typed family loaders and `ModelInfo` provenance |
| Custom exported CTC/transducer models | `GraphSpeechRecognizer` with an explicitly declared tensor/feature contract |
| Speaker diarization output | `Turn` values and word assignments in a `Transcript` |

An arbitrary ONNX export is not automatically compatible. OneSpeech runs supported graphs on OneModel, not ONNX Runtime. Confirm layout, blank ID, front end, temporal stride and predictor/joint state contract in [custom speech graphs](https://recursiveai.org/onespeech/docs/custom-models/). There is no native-handle escape hatch.

Live input is mono 16 kHz PCM; capture and resampling remain the caller's responsibility. `PartialEvent` replaces a hypothesis for one utterance; `FinalEvent` commits it. Do not infer finality from an unchanged partial. See [the event model](https://recursiveai.org/onespeech/docs/live-sessions/) and [diarization limitations](https://recursiveai.org/onespeech/docs/diarization/).


<!-- https://recursiveai.org/onespeech/docs/migrate-azure-speech/ -->

# From Azure Speech SDK file recognition

Here “offline recognition” means recognizing a file rather than a microphone; it does **not** imply that an Azure service call itself is offline. This concept map uses Microsoft's [recognition guide](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-recognize-speech), consulted 2026-11-08. No SDK implementation is used.

| Familiar concept | OneSpeech equivalent |
|---|---|
| Speech configuration with service credentials/region | Local catalog selection; no cloud recognition credentials or region |
| Audio configuration selecting a WAV file | `Speech.Transcribe(path)`; supported audio is decoded through OneAudio |
| Single-utterance recognition | VAD-based recognition; a complete file call is not limited to one utterance |
| Continuous recognition callbacks | `await foreach` over file segments or live events |
| Recognizing / recognized notifications | Replacing `PartialEvent` / immutable `FinalEvent` |
| Serialized service result | `Transcript.ToJson()` with the versioned OneSpeech schema |

Do not copy service JSON parsing, event-threading assumptions or service offsets into a OneSpeech application. Every time here is relative to original input audio; confidence may be null and is not calibrated across models. Cancellation is explicit and output is an immutable document/event snapshot, not an SDK result object.

Recognition runs locally after models are obtained, but first use may access the network to download weights. For disconnected deployments, [pre-fetch all required roles](https://recursiveai.org/onespeech/docs/models/). Live input does not open a device. [Speaker recognition](https://recursiveai.org/onespeech/docs/speaker-recognition/) requires separate consent and legal consideration; a recognized voice is not an authentication guarantee.


<!-- https://recursiveai.org/onespeech/docs/migrate-whisperx/ -->

# From WhisperX concepts

This guide uses only the [published package description](https://pypi.org/project/whisperx/), consulted 2026-11-08. It maps a workflow, not Python modules, dictionaries or an API signature.

| Familiar step | OneSpeech equivalent |
|---|---|
| ASR transcription | A `SpeechRecognizer`, or `Speech.Transcribe` for the integrated file path |
| Forced alignment for word timestamps | `Aligner` / `Speech.Align`, with nullable timing and per-word `TimingMethod` |
| Speaker diarization | A `Diarizer` producing original-clock `Turn` values |
| Assigning words to speakers | `TranscriptDiarization.Merge`, greatest covered duration with documented overlap rules |
| Saving segments/words | Immutable `Transcript` and schema-v1 JSON |

The integrated helper performs the supported workflow, but its defaults are different. The default diarizer **does not detect overlap**; use the explicit [Nemotron upgrade](https://recursiveai.org/onespeech/docs/nemotron-diarization/) if you need overlapping turns. An authorized local segmentation-3.0 snapshot can also supply [overlapping turns with our clustering](https://recursiveai.org/onespeech/docs/gated-segmentation/); community-1 has no native pipeline adapter. No noncommercial alignment weights are loaded.

Timing follows the [supported timing policy](https://recursiveai.org/onespeech/docs/word-timing/), not a language-to-aligner dictionary copied from another project. Standalone alignment currently has a 60-second limit. Words without valid timing stay untimed, and words with no covering turn are `Unassigned`; neither is silently fabricated. Inspect [timing and diarization results](https://recursiveai.org/onespeech/docs/results/) before treating labels or boundaries as ground truth.


<!-- https://recursiveai.org/onespeech/docs/audio-input/ -->

# Audio input and transcription pipeline

`AudioFileSource` is a synchronous pull decoder. It accepts seekable encoded streams or paths and returns interleaved float PCM at 16,000 Hz, preserving the original channels. Supported formats are those registered by OneAudio: WAV, MP3, FLAC, Ogg Vorbis/Opus, AAC-LC in M4A/3GP/ADTS, and AMR-NB/WB. Decoding and polyphase resampling keep bounded buffers; the whole recording is never loaded. Resampler latency is compensated by OneAudio, and its tail is flushed at EOF.

A removable internal adapter additionally handles RIFF/WAVE A-law (format tag 6) and μ-law (7), including their WAVE_FORMAT_EXTENSIBLE subformats. It expands companded samples to signed linear 16-bit values and divides by 32768. It validates RIFF/chunk extents, channel count, sample rate, byte rate, block alignment, bit depth, extensible GUID and whole-frame data length. Channels are limited to 32 and G.711 rates to 192,000 Hz. Unknown codecs are delegated to OneAudio; `.au` is not supported. G.711 is specified by ITU-T Recommendation G.711 (11/1988), §3. No code from the associated software tools is used.

Unsupported or corrupt encoded input produces `UnsupportedAudioException`; filesystem/access errors and cancellation are not disguised as format errors. Streams opened with `leaveOpen` remain caller-owned. The asynchronous pipeline copies caller-owned encoded input with asynchronous I/O into a delete-on-close temporary file, then decodes it synchronously. This permits arbitrary nonseekable input and codecs needing random access with constant RAM, at the cost of temporary disk space proportional to encoded input size. Cancellation or a failure deletes the spool. No background inference or async-over-sync scheduling is used.

## Chunking

`VadChunker` accepts 16 kHz PCM blocks, and averages channels with equal weight for recognition. The original channels are still available from `AudioFileSource`; the chunker does not perform channel diarization. Input block storage may be reused after advancing enumeration; emitted chunks own their storage.

A recognition chunk defaults to at most 30 seconds. A family can supply its own maximum in the range 1 second through 1 hour. Each bounded analysis horizon includes one second of lookbehind and one second of lookahead. The detector resets for each horizon; this is not a promise of uninterrupted recurrent VAD equivalence across an entire file. Lookbehind preserves short trailing continuations across forced cuts. The last retained speech-region end within the maximum is preferred as the cut; uninterrupted speech is cut at the maximum. Detected regions within a chunk are merged **without deleting internal silence**. Leading/trailing silence outside retained speech is discarded. Onsets, padding and gaps retain their original audio-time coordinates.

PCM memory is bounded by the maximum chunk duration plus the fixed analysis context, independently of file length. Enumeration can discard each chunk after inference. `AudioChunk.Start` is its origin in the file; recognition times are relative to the chunk. Progress reports the consumed audio duration, including skipped silence, not wall time. Cancellation is checked per decoder block, per VAD frame and before each recognition chunk.

## Pipeline

`VadTranscriptionPipeline` borrows a `VoiceActivityDetector` and `SpeechRecognizer`. It offsets every non-null segment/word/turn timestamp by the chunk origin and preserves null timings. It does not synthesize segments or word timings. Text from nonempty chunk results is joined by one space; text within each result is unchanged. The first detected language, or the caller's explicit language, is used for subsequent chunks. Provenance includes the detector, recognizer, and additional contributing models, deduplicated by ID and revision. Without an optional diarizer, speaker assignments supplied by the recognizer are preserved. The additive diarizer hook, two-pass whole-file processing and embedding-memory bounds are described in [diarization](https://recursiveai.org/onespeech/docs/diarization/).

`Transcribe` accumulates a document, so its text and metadata necessarily grow with output size; only PCM processing is constant-memory. `TranscribeStream` yields final segments without accumulating a document. `Speech.Transcribe` and its stream-input async twin forward to this pipeline through an internal explicit recognizer resolver. Until model-loading integration supplies a recognizer, they fail with `FeatureUnavailableException` before opening input. Language detection uses the first detected speech chunk and refuses a non-null language hint rather than ignoring it. A silent recording cannot yield a language and raises `UnsupportedAudioException`.


<!-- https://recursiveai.org/onespeech/docs/long-files/ -->

# Long files and cancellation

Use `Speech.Transcribe` when you need a complete document with whole-recording speaker labels. Use `Speech.TranscribeStream` when you want final segments as bounded VAD-first chunks finish. The latter does not perform whole-file speaker clustering. Neither API requires retaining the entire PCM waveform.

Encoded files are decoded and resampled through OneAudio. Whole-recording diarization and recognition require separate bounded passes. Caller-owned encoded streams may be spooled to a temporary seekable file; ensure adequate disk space and dispose or finish enumeration to release it. Do not modify a file while it is being processed. The final document's words, text and turns necessarily grow with output length; constant audio working storage does not mean a constant-size returned document.

File calls have `CancellationToken` overloads, and asynchronous stream calls require explicit cancellation. Cancellation is checked at audio blocks, chunk boundaries and model operations where supported, not by interrupting arbitrary instructions. It throws `OperationCanceledException`. Progress in the low-level pipeline is audio time processed, not a wall-clock ETA. First-use model acquisition can dominate startup; [pre-fetch](https://recursiveai.org/onespeech/docs/models/) before interactive use.

Streaming enumeration owns its decoder and loaded model resources. Early disposal releases them. Live cancellation discards pending speech instead of silently finalizing it; normal end-of-input flushes. See [live sessions](https://recursiveai.org/onespeech/docs/live-sessions/).

Standalone `Speech.Align` currently limits input to 60 seconds. Split longer alignment requests yourself and offset every returned time onto the original audio clock; do not concatenate chunk-relative times as if they were recording-relative. Recognition's timing pipeline has its own bounded chunks. Words that could not be aligned retain null boundaries.

Long-form reference equivalence and full working-set benchmark rows remain [pending](https://recursiveai.org/onespeech/docs/results/). VAD horizon boundaries and chunk-boundary speech can affect output; file streaming is not a promise of token identity with an oracle's whole-file policy.


<!-- https://recursiveai.org/onespeech/docs/vad/ -->

# Silero voice activity detection

The core assembly embeds the unmodified `silero_vad.onnx` from Silero VAD v6.2.3, revision `5cd7945676eb32225748052e2e6a0580e4686a08`, SHA-256 `1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3`. It runs on OneModel 1.0.0 in float32 fast arithmetic with one inference thread. The MIT attribution is in `THIRD-PARTY-NOTICES.md`. Weights are not stored in the source tree: building downloads the pinned release model to the external artifacts directory, or accepts an existing file through `-p:SileroModelPath=...`, and verifies its digest before embedding.

`SileroVad` accepts mono PCM at 8,000 or 16,000 Hz. `Detect` creates fresh recurrent state for each recording. Frames are 32 ms: 256 or 512 samples respectively. Each graph input prepends the previous 4 ms of PCM (32 or 64 samples), initially zeros; the recurrent state is initially zeros of shape `[2,1,128]`, and the returned `stateN` becomes the next state. The final short frame is right-padded with zeros; reported intervals are clipped to the actual input duration. An empty input produces no intervals. `CreateState` exposes the same frame processing for callers needing uninterrupted recurrence, and `Reset` starts a new recording. State instances are not thread-safe; the detector must outlive them.

Segmentation uses audio sample positions, not elapsed wall time. The defaults are activation probability 0.5, deactivation probability 0.35, minimum speech 250 ms, minimum silence 100 ms, and 30 ms padding. Activation begins a candidate interval at that frame's start. A probability below the deactivation threshold starts a silence candidate; it closes speech once the silence has lasted at least the configured duration, placing the end at the silence candidate's start. Activation cancels a pending silence candidate. Intermediate probabilities leave the candidate unchanged. Only speech strictly longer than the minimum is retained. A recording ending in active speech closes at the real input length. Padding is clipped to the recording, and adjoining intervals share the intervening gap equally when both pads would overlap. Cancellation is checked before loading/inference and at each frame.

The frame/context protocol and segmentation defaults are checked using only the published model data and black-box calls to `silero-vad` 6.2.3's public API. The targeted probe set comprises three LibriSpeech test-clean clips, a decimated 8 kHz version of one clip, and seeded synthetic silence/noise. The float32 probability tolerance is max absolute error ≤1e-5; this is cross-runtime fast-mode numerical parity, not a bit-exact arithmetic claim. Segment counts must match, and default-parameter boundaries must agree to within one sample. `tools/oracles/silero` contains the pinned evaluation-only harness; no oracle code or runtime is shipped.


<!-- https://recursiveai.org/onespeech/docs/word-timing/ -->

# Word timing and forced alignment

Every `Word` has nullable `Start` and `End`, nullable `Confidence`, and `TimingMethod`. Times are `TimeSpan` values on the original audio clock, not processing time. Missing boundaries remain null; confidence is not inferred from having a boundary. Speaker assignment cannot fill missing timing.

The timing preference is **native → Qwen3-ForcedAligner → omniASR-CTC → DTW**, subject to model and language coverage. The pipeline asks registered timing aligners in order; it is not permission to claim a method a loaded family cannot provide. Qwen recognition uses its paired aligner; Whisper's default file path uses the omniASR CTC aligner rather than labelling DTW as native. Parakeet supplies native timing. Provenance distinguishes `Native`, `ForcedAligner`, `CtcAligner`, `Dtw` and `None`.

The standalone `Speech.Align(path, text, language: ...)` helper currently uses omniASR-CTC by default, accepts at most 60 seconds, and downloads the roughly 1.3 GB model on first use. Passing a `Transcript` preserves its written words and non-timing fields instead of re-splitting its text. Stream overloads do not own the caller's stream. Empty or unalignable targets never justify fabricated times. See [CTC alignment](https://recursiveai.org/onespeech/docs/ctc-alignment/) and [Qwen3](https://recursiveai.org/onespeech/docs/qwen3/) for supported language and model details.

Alignment maps supplied text to audio; it does not prove that every supplied word was spoken. Check null boundaries and the [measured errors](https://recursiveai.org/onespeech/docs/results/#timing) before using timestamps as evidence. The Qwen toolkit post-processing difference is still a known gap. No method is sample-accurate ground truth.


<!-- https://recursiveai.org/onespeech/docs/ctc-alignment/ -->

# CTC forced-alignment trellis

`CtcTrellis.Align` accepts row-major natural-log frame probabilities and a fixed
sequence of nonblank vocabulary IDs. It consumes every frame and requires the
complete target. Leading and trailing blank are allowed. Blank separates equal
adjacent target IDs; distinct adjacent IDs may follow without blank. The recurrence
is the blank-expanded target in Graves et al., ICML 2006, §4.1, with maximum
instead of summation to select a single path. This is forced alignment, not CTC
prefix decoding or the marginal probability of a target.

An exact predecessor tie prefers staying in the state, then moving one state,
then moving two. An exact terminal tie prefers the final token to trailing blank.
Accumulation uses double precision; emissions are float32. Negative infinity is
an impossible emission, while NaN, positive infinity and positive log values are
rejected. The caller supplies normalized log probabilities; the trellis does not
normalize them. Traceback is bounded to 16 million cells, with two score rows.
Cancellation is checked before validation and once per emission/trellis/traceback
frame.

The result explicitly distinguishes failure from an empty target. Failure has
negative-infinite score and no intervals; no prefix or guessed timing is returned.
An empty target is the all-blank path, including a successful score-zero path for
empty audio. Each successful nonempty target returns one inclusive-start,
exclusive-end frame interval per target position. Repeated IDs remain distinct.
Only frames occupied by that token count; intervening blank frames are not
attributed to either neighbor. The score is the best path's log probability, not
word confidence. Model adapters must establish frame-to-audio conversion and word
boundary behavior separately; token occupancy is not calibrated word timing.

## omniASR-CTC-300M adapter

The fp32 adapter loads the pinned publisher checkpoint as inert tensor data and
converts its 7-layer convolutional front end, 24-layer pre-normalized Transformer
and linear 9,812-character CTC head to OneModel format. It accepts mono 16 kHz
waveforms. The graph subtracts utterance mean and divides by the square root of
population variance plus `1e-5`, then uses the checkpoint's convolution and layer
normalization parameters. The feature extractor has a 400-sample receptive field
and a 320-sample stride. The even-width positional convolution is trimmed by one
frame on the right. No oracle-exported graph or executable checkpoint object is
loaded.

The model vocabulary contains unique single-character pieces, an identity
normalizer, no dummy prefix and no escaped whitespace. Written words are kept
separate from the spoken target: casing and punctuation are retained in the
result, while the alignment target is NFC-normalized, lowercased and excludes punctuation
other than apostrophes. Combining marks are retained, typographic apostrophes
become ASCII apostrophes, and hyphens become spaces. English integers from zero through
999,999,999 are expanded to cardinals internally (including comma-separated
integers). This is not a general inverse-text-normalization engine: currency,
decimal, date, acronym and language-specific number readings are not resolved.
Unknown characters prevent that written word from receiving a timing.

Word occupancy starts at the first target character's occupied frame and ends at
the last character's exclusive end frame, converted with the fixed 20 ms stride.
No blank gap is spread over words, and no hand-tuned latency offset is applied.
A constrained word whose geometric mean occupied-frame probability is below
0.01 is left untimed. This rejection heuristic is not a calibrated probability
of word correctness. `Confidence` is not manufactured; existing word confidence,
speaker and other non-timing fields are retained by the transcript overload.

A call is bounded to 60 seconds; longer input is refused with a typed capability
error and must be split with its corresponding written words. File input is
bounded while decoding, before a model download, and is downmixed to mono.
`Speech.Align` tries Qwen3-ForcedAligner first for its eleven declared languages,
then this model for remaining untimed words. Omitted language uses CTC unless the
supplied transcript declares a covered language. An explicit model ID selects only
that aligner. First use downloads and verifies the selected publisher files and
reproduces the catalog's converted hashes. The string overload splits on whitespace. The transcript overload uses
`Words` in order without splitting them again. Corresponding segment word lists
are projected when they have the same written-word partition. Written text and
word boundaries are never reconstructed from the CTC hypothesis.

`VadTranscriptionPipeline.TimingAligners` is an ordered list of borrowed
aligners. Native timestamps win. Compatible forced aligners are tried in order
(for example Qwen3 before CTC); failed words retain existing DTW as the final
fallback. Compatibility requires `Capabilities.ForcedAlignment` and a matching
declared language. Whole-file transcription uses native timings first, then
Qwen3-ForcedAligner for zh/en/yue/fr/de/it/ja/ko/pt/ru/es, then CTC, then existing
DTW. Models are loaded lazily after the chunk's language is known; CTC is not
loaded if Qwen times every word. Failed or out-of-window alignment cannot invent
a boundary. Unsupported-audio and capability failures advance to the next method;
cancellation, corrupt models and I/O errors propagate. Successful words retain
per-word timing provenance. A lower-priority method never overwrites a successful
higher-priority timing. Qwen accepts more than 200 samples through 300 seconds;
CTC accepts at most 60 seconds. File recognition chunks remain bounded to 30
seconds; public file alignment retains its 60-second input bound.

Encoded-stream twins (`Speech.AlignAsync`) leave the caller's stream open. They
perform asynchronous bounded I/O to a temporary seekable file, capped at 512 MiB
of encoded bytes, before synchronous PCM decoding/inference. The temporary file
is removed on completion, cancellation or failure. Unsupported numeric readings
(including decimals, currencies and leading-zero strings) are left untimed, not
silently assigned the timing of a different spoken number.


<!-- https://recursiveai.org/onespeech/docs/biasing/ -->

# Contextual phrase biasing

Contextual biasing favours domain names and phrases during recognition; it is not model training and cannot guarantee that a phrase is spoken. The low-level entry point is `SpeechRecognizer.WithPhrases`. Check the recognizer's `Capabilities.Biasing` before requesting it. Unsupported requests must fail with a typed capability error rather than silently ignoring the phrases.

The family determines how context is applied: Whisper prompt conditioning, Qwen context, or CTC/transducer shallow fusion over a phrase trie. Phrase weighting is not a probability and does not change the published checkpoint. Overweighting names can insert them into unrelated speech; compare both phrase recall and ordinary WER on held-out audio.

The phrase-binding API and cross-family results are being finalized. This page intentionally provides no call-signature example until that API is present in the shipped build. The current checkout's base recognizer has no `WithPhrases` member; do not assume an older package supports it. File helper parameters remain only language and model; use the low-level recognizer for contextual options.


<!-- https://recursiveai.org/onespeech/docs/contextual-biasing/ -->

# Contextual token scoring

`OneSpeech.Decoding.PhraseTrie` is an immutable, inference-only context graph.
Construct it from `TokenPhrase` values tokenized by the model's tokenizer. Include
word-boundary markers when the tokenizer uses them; this primitive matches IDs,
not words, case variants or Unicode strings. Blank and control tokens are never
phrases. The decoder validates vocabulary bounds and excludes blank.

A phrase's weight is a natural-log bonus per token, default 1, strictly positive
and at most 20. Completion receives `token count × weight`. Duplicate phrases
use the largest weight. Multiple phrases finishing at the same token receive
the largest completion reward, not their sum. Overlapping completions at
different positions each receive credit. Lists are bounded to 4096 phrases,
256 tokens per phrase and 65536 total tokens.

For search pruning, unfinished prefixes borrow credit proportional to their
depth and weight. Failure links retain suffix matches. `Advance` returns the
completion reward plus the change in prefix credit; `Finish` repays all remaining
prefix credit. Consequently `Score` of a complete sequence counts only completed
phrases. A search must keep one context state per hypothesis and advance it
only when accepting a new nonblank token. Do not advance on CTC repeated frames
or blank frames. Context states belong to the trie that created them.

`ContextualCtcBeamSearch` sums acoustic blank/nonblank path probabilities just as
ordinary CTC prefix search does. It ranks prefixes by acoustic log probability
plus context score before pruning, then removes unfinished credit before final
ranking. It reports acoustic probability and completion reward separately; the
fused score is not a probability. With an empty trie it is equivalent to
`CtcPrefixBeamSearch` at the same beam size. Pruning can discard the global winner.

`ContextualTransducerSearch` adds context deltas before greedy token selection
without renormalizing the acoustic distribution. Blank retains context, the
predictor receives only accepted nonblanks, and TDT duration logits are unchanged.
Confidence is always the original conditional acoustic probability. The greedy
search cannot recover an alternative after pruning: unfinished-prefix credit
can influence the final token. It shares symbol bounds and native frame intervals
with the non-contextual RNN-T/TDT decoders. Empty context is neutral.

This follows the inference-only shallow-fusion principle in Le et al.,
*Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep
Biasing and Shallow Fusion* (2021), §3.1. It does not implement that paper's learned
deep-biasing modules. Context bonuses can introduce false alarms; measure both
phrase recall and WER, including controls whose phrases are absent from the audio.
These primitives alone do not enable phrase requests on a recognizer family.

## Recognizer requests

`SpeechRecognizer.WithPhrases(IEnumerable<Phrase>, CancellationToken)` copies and
validates ordinary text, then returns a borrowed recognizer. No model is loaded
again. The owner must outlive wrappers/sessions; existing single-instance
concurrency restrictions are unchanged. Wrappers are not disposable owners.
Reapplying the method replaces the phrase list, including when applied to a
wrapper. Empty lists return the original recognizer. Unbiased recognizers and
other wrappers remain unchanged. `VadTranscriptionPipeline` accepts a wrapper
without any new options. File and utterance-live chunking retain its context.
Language detection is delegated to the original recognizer without bias.

Common limits are 4096 phrases, 1024 UTF-16 units per phrase, and 262144 total
units. Text must be valid Unicode and non-whitespace; outer whitespace is removed.
Weights must be finite and strictly positive, at most 20. Model-specific limits
can be smaller. No phrases or embeddings are persisted.

- **Whisper:** comma-separated phrases encoded as ordinary text with a leading
  space; special tokens refused. The entire fixed prompt must fit 223 tokens,
  otherwise configuration throws. Each window uses `startofprev`, fixed phrase
  tokens, then the newest generated-context tokens that fit the remaining
  223-token budget, followed by task/language controls. Fixed phrases survive
  fallback context resets. Only generated tokens enter the transcript. This
  uses the prompt-conditioning mechanism described in Radford et al. (2022)
  §2.2; repeating the fixed phrases at each window is this library's policy.
  This is an opt-in hint, not constrained decoding: it can substantially worsen
  WER or repeatedly hallucinate absent prompted terms. Improved term recall alone
  is not evidence that a prompt is safe.
- **Qwen3-ASR:** append comma-separated phrases to the configured `Context`,
  separated by a newline, and use the existing system-context path. Ordinary
  tokenizer encoding validates the entire combined 4096-token budget and
  refuses special tokens. No second biasing implementation is introduced.
- **Prompt weights:** explicit numeric weights are refused with
  `CapabilityNotSupportedException` for Whisper and Qwen. Repeating words or
  adding weight text would not implement a numeric score bonus.
- **Parakeet TDT:** use the publisher's pinned BPE ranks, precompiled character
  normalization and Metaspace word boundaries. Text containing control-token
  syntax or characters that encode as unknown is refused, rather than biasing
  the shared unknown ID. Ordinary token weights feed the phrase trie.
- **Custom graphs:** `TokenVocabulary.WithPhraseEncoder` attaches the caller's
  exact encoding callback to a copy of the vocabulary. No segmentation or
  normalization is guessed from piece lengths. Nonblank ordinary IDs are the
  caller's contract; blank/out-of-range IDs and trie limits are validated.
  Contextual CTC requires at most 4096 columns and uses the configured beam or
  beam 8 when unbiased decoding is greedy. RNN-T/TDT remain greedy.
- **Unsupported families:** Moonshine retains `Biasing=false`; nonempty phrases
  throw `CapabilityNotSupportedException` before recognition, never silently
  ignored. Empty requests are neutral. Invalid/canceled requests still fail.

## Targeted evaluation

`tools/OneSpeech.BiasingEval` prepares a fixed small Earnings-22 comparison. All
audio, references, hypotheses and phrase lists stay in an external state directory,
never committed (Earnings-22 CC-BY-SA-4.0). The original fetch script
`tools/scripts/fetch-earnings22.sh` pins the publisher revision and input hashes.
The diagnostic additionally verifies its five input SHA-256 pins in
`tools/scripts/pins/biasing-earnings22.sha256` before decoding.

Selection is outcome-independent: the two shortest complete recordings by the
pinned metadata's numeric duration, ordinal file-ID tie-break. These are
4474955 (875 seconds) and 4483046 (1187 seconds), about 34 minutes total. Full
human references avoid guessed crop boundaries or alignment-dependent labels.
Phrase candidates have at least five ASCII letters, 1–3 case-insensitive
occurrences in that file's full reference, and at least one capitalized occurrence
away from a sentence start. Select at most 12 in ordinal case-insensitive order.
The scorer requires each selected term to normalize to one word and fails
preparation otherwise. Controls swap the other file's selected list, removing
terms present anywhere in the normalized control reference. This reference-derived
list is a controlled diagnostic, not an estimate of how users choose phrases.

From the repository root, run `prepare CORPUS STATE`, then each
`run FAMILY MODEL STATE CASE MODE` for families `whisper`, `qwen`, `parakeet`,
cases `0`, `1`, and modes `baseline`, `phrases`, `control`. `score STATE` writes
`scores.jsonl`. Model directories are existing publisher-verified converted caches,
never modified. Shard by (family, case, mode); atomic transcripts allow bounded
jobs to resume without rerunning completed cases.

Every mode uses the same public `VadTranscriptionPipeline`, default Silero VAD,
30-second maximum chunks and English hint. Whisper-small uses fp32, temperature
zero and no DTW; Qwen3-ASR-0.6B uses fp32 greedy; Parakeet uses fp32 greedy TDT.
Each uses four graph threads. All fusion weights retain the unmodified default 1;
prompts have no explicit weights. No model parameters or score weights are fitted
on these files.

Both sides use `en/1` normalization and the same word edit alignment. Phrase
recall counts correctly aligned reference occurrences, not bag-of-word presence.
Control false alarms count normalized occurrences of absent control terms in
the hypothesis. Report WER and occurrence counts/recall for all modes per file,
including control WER and its difference from baseline. These two files are
conditional diagnostic floors, not corpus confidence intervals or a general
quality claim; do not aggregate away a per-file regression.

`bootstrap.jsonl` also records paired file-level 95% percentile intervals (1000
resamples, seed 1729) for WER and phrase miss rate (`1 - recall`), comparing each
condition with baseline. With only two files these intervals are very coarse
and conditional on this selection; they do not replace the per-file rows.

### Whisper reference diagnostic

`chunks STATE` exports the same default-Silero, 30-second-bounded PCM chunks used
by the pipeline, reading the original prepared WAV in 4096-frame blocks.
`tools/oracles/whisper-features/biasing.py STATE CASE MODE --weights DIRECTORY`
runs the pinned openai-whisper 20250625 small model through its documented
`transcribe` API, independently per chunk. The comma-space-joined phrase text
is passed as `initial_prompt`; English, fp32, temperature zero, previous-text
conditioning and default rejection thresholds are explicit. Chunk outputs are
atomic and resumable; no model source or exported implementation graph is used.
`score STATE` also scores completed `reference-whisper` results with the same
normalizer, edit alignment and paired file bootstrap.

This comparison reproduced the prompt-induced regression and absent-term
hallucination: prompt conditioning is not a guarantee of improved WER. The
native fixed-prompt retention policy and the reference's initial-prompt option
are not claimed to produce identical decoding. Keep the per-file results and
false alarms alongside term recall; do not treat reference reproduction of a
failure as evidence that the hint is safe.


<!-- https://recursiveai.org/onespeech/docs/diarization/ -->

# Diarization

Speech recognition for .NET. Speaker labels are document-local, not biometric identities. No audio or embedding is persisted by diarization helpers.

## Word assignment

`TranscriptDiarization.Merge` is shared by diarizers. Input `ActiveTurn` values wrap original-clock turns and optional relative activity in [0,1] (default 1, not a probability). Turns require a named, assigned speaker, nonnegative start and positive duration. Null times are never filled in.

Intervals are half-open. A sweep partitions the turns at every endpoint. Where speakers are concurrent, the highest-activity turn wins; equal activity breaks by ordinal speaker ID. Covered duration in these partitions is summed per speaker for each word. The speaker covering the greatest duration wins; equal durations again break by ordinal ID. Uncovered portions do not vote. A positive-duration word can therefore be assigned even if most of its interval is silence. No coverage, null endpoints, reversed endpoints, and zero-duration words yield `SpeakerLabel.Unassigned`.

`Word.Overlap` is true only when a positive-duration part of the word intersects simultaneous turns from **different** speakers. Duplicate or overlapping turns from the same speaker do not count as overlap. Touching endpoints do not overlap. This flag is independent of which speaker wins. Merging replaces prior speaker/overlap assignments; it preserves word text, timing, confidence and timing method, document text, language and model provenance. Segment words are assigned by the same rule. The input document remains unchanged.

Original turns are ordered by start and ordinal speaker ID. Each assigned root word is attached once to the turn of its chosen speaker having greatest temporal intersection, breaking ties by earliest start. Turn text is the exact concatenation of attached word text, without inserted spacing. Segment-only words are assigned in their segments but are not added to the root word list or turn text.

`TranscriptDiarization.Exclusive` provides a separate, exclusive caption timeline: the highest-activity speaker at each instant, with the same tie rule, coalescing adjacent intervals for the same speaker. It leaves gaps uncovered and produces empty turn text/words; a caller must place captions explicitly rather than infer word timing from this view.

## Separated-party stereo

`ChannelDiarizer` uses channel labels `channel-1` and `channel-2`; it does not infer identities or distinguish multiple people on one channel. Auto-detection is deliberately conservative:

* exactly two channels;
* at least three 20 ms frames dominated by the left channel and at least three dominated by the right;
* dominated means energy at least 100 times the other channel (20 dB), above mean-square energy 1e-8;
* absolute normalized waveform dot product over the whole recording at most 0.3.

The final short frame is included. These are initial heuristic values, **not** thresholds fitted or validated on a speech development set. Silence, mono, dual-mono and one-sided conversations are inconclusive, not evidence of separation. Auto-detection returns false for them. This rule cannot prove separation: uncorrelated stereo music or alternating noises may pass. Known separated calls can set `forceSeparated: true`; known mixed audio should use a voice-clustering diarizer instead. Failure is a `CapabilityNotSupportedException`, never a silent downmix fallback.

For each channel, VAD runs on bounded, independently analyzed blocks of at most 30 seconds, retaining original-clock times. VAD state resets at block boundaries; this is not continuous-state VAD equivalence. Simultaneous channel speech is retained. Channel output metadata grows with turns, but temporary PCM does not grow with duration. The caller's in-memory input remains borrowed.

## Pipeline

The five-argument `VadTranscriptionPipeline` constructor accepts a borrowed `Diarizer` after the existing chunk-duration and progress arguments. The four-argument constructor is unchanged. In-memory transcription runs recognition on the usual downmix and diarization on the **original channels**, then merges words. Segment enumeration computes whole-recording diarization before emitting any final segments, so assignments are never revised after emission.

`ClusteringDiarizer` and `ChannelDiarizer` implement `BlockDiarizer`: consecutive PCM blocks share one recording clock, and returned IDs are global to that recording. File inputs make two bounded decoding passes: diarization first, recognition second. Encoded streams use the existing temporary seekable spool and rewind it between passes; the spool is deleted when processing ends. No recording waveform is retained. Progress remains recognition audio-time progress, not wall time or a sum over both passes. Callers must not change a file while it is being processed.

A custom plain `Diarizer` still works in memory, but file inputs require `BlockDiarizer`; otherwise they throw `CapabilityNotSupportedException` naming that requirement. BlockDiarizer declares contributing `Models`, which the pipeline adds to document provenance, deduplicated by ID and revision. Both channel and clustering implementations declare their VAD/embedding models. Omitting the diarizer preserves existing bounded file transcription.

## Voice clustering and overlap limitation

`ClusteringDiarizer` is the tier-1 voice-clustering implementation, using caller-supplied VAD and speaker-embedding roles. It **does not detect overlapped speech**: each speech interval gets only one speaker at a time. The overlap-aware [Nemotron 3 Diarization](https://recursiveai.org/onespeech/docs/nemotron-diarization/) upgrade is an explicit tier-2 option, never selected by these helpers; the pyannote segmentation slot remains separate. Channel diarization is a distinct exception because separated channels supply activity independently. The component itself always uses the caller's embedder. The `Speech.Transcribe` helpers select CAM++ Chinese for a requested or detected Chinese language (`zh`, `zh-*`); otherwise they select CAM++ English. An explicit language takes precedence over detection. Recognition runs before the whole-recording diarization pass so this policy also applies without a language hint. Encoded streams are spooled once, with bounded PCM on both passes. This is a language-based starting choice, not a quality-validated default for every language.

For an overlap-aware file transcript, keep your recognizer and VAD and replace only the diarizer:

```csharp
using var overlap = await OneSpeech.Diarization.NemotronDiarizer.LoadAsync();
var transcript = new OneSpeech.Pipeline.VadTranscriptionPipeline(recognizer, vad, null, null, overlap).Transcribe(path, language: null, CancellationToken.None);
```

The pipeline retains the higher-activity speaker in simultaneous intervals, flags overlapping words, and leaves gaps Unassigned. This does not separate simultaneous voices or recover words the recognizer missed. The named load fetches the publisher's pinned checkpoint, keeps OpenMDW agreement/origin notices in `ModelInfo`, and converts only into the local cache.

### Windows and clock

Input is consecutive 16 kHz PCM with one consistent channel count (1–32). Channels are averaged for voice clustering. VAD runs in fixed 30-second horizons, independent of caller block boundaries; state resets at each horizon. VAD regions are sorted and overlapping/touching regions unioned. Segments shorter than 100 ms are discarded. Following Landini et al., arXiv:2012.14952, §3.2, defaults are 1.5-second embedding windows and a 0.25-second stride; partial trailing windows of at least 100 ms are included. Models are borrowed, and each finite, nonzero embedding is length-normalized in double-precision accumulation before clustering. Embedding failures propagate; they are not replaced by arbitrary labels.

The boundary between neighboring window assignments is the midpoint between their centers. First/last assignments extend to the **VAD region** endpoints. Silence gaps remain gaps. Adjacent same-speaker assignments are coalesced. Labels `speaker-1`, `speaker-2`, ... follow first occurrence, are document-local, and are not enrolled identities. These are window-level speaker boundaries, not refined word or phonetic timing. Whole-file `Speech.Transcribe` helpers additionally compare normalized cluster means to compatible process-wide enrollments, setting `SpeakerLabel.Name` on accepted matches while preserving these document-local IDs. Unknown clusters remain anonymous; see [speaker recognition](https://recursiveai.org/onespeech/docs/speaker-recognition/).

### Spectral clustering and AHC

For recordings whose retained embedding span exceeds 60 seconds, the default `ClusteringMethod.Spectral` follows Park et al., *Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengap*, arXiv:2003.02405v1, §III, Eq.4–15:

1. Raw cosine affinities; row-wise top-p binarization (stable index ties), including the diagonal when ranked there.
2. Symmetrization `(A + Aᵀ)/2`, then unnormalized Laplacian `D − A`.
3. Symmetric eigendecomposition and ascending eigenvalues. The largest allowed eigengap estimates the speaker count. Select p minimizing `p × (largestEigenvalue + 1e-10) / max(gap,1e-10)`.
4. Cluster rows of the smallest-k eigenvector matrix by Euclidean Lloyd k-means, with deterministic farthest-first seeds (first seed index 0), stable ties and empty-cluster repair. At most 100 iterations.

Two bounded-work deviations from the paper are explicit: dense clustering uses up to `MaxLandmarks` uniformly spaced windows over the complete sequence (default 128, allowed 16–256); p is searched at 2,4,8,... plus `max(2,floor(L/4))`, rather than every integer up to L/4. All nonsampled windows are assigned to the maximum-cosine landmark-cluster centroid; landmark labels are retained, including explicit-count splits of indistinguishable vectors. Rare/short speakers absent from the sample can therefore be missed. This is not a reference-equivalence claim.

The symmetric eigensolver uses cyclic Jacobi rotations, at most 60 sweeps, stopping when maximum off-diagonal magnitude is ≤1e-10 times the largest diagonal magnitude (at least 1). Numerical nonconvergence uses AHC instead. When all landmark cosines are within 1e-6 of one, there is no geometric separation evidence; AHC avoids creating speakers from arbitrary graph tie-breaking.

`ClusteringMethod.Agglomerative` is the alternative and the spectral numerical fallback: average-link cosine similarities, stable pair-order ties, merging to the supplied exact count or stopping below the configurable cosine threshold while honoring min/max. This uses the cosine AHC initialization described in Landini §3.2, **not** PLDA. The generic threshold 0.65 is an exposed starting value, not a universal model calibration; use the model-specific measured profile below. No embedding-model weights are changed.

`NumSpeakers` is exact for nonempty speech; `MinSpeakers`/`MaxSpeakers` constrain estimates. Default range is 1–8, supported ceiling 32. Exact counts must lie within min/max. If speech produces fewer windows than the required count, the request is refused with `CapabilityNotSupportedException`; silence produces no turns, not fictitious speakers. Invalid parameter combinations are rejected. Cancellation is checked per PCM block, embedding window, eigen sweep, clustering iteration and output turn.

### Short recordings

For a retained embedding span of **at most 60 seconds**, the default spectral policy uses average-link cosine AHC instead. A sparse top-p rank graph can partition even a compact single-speaker cloud into disconnected nearest-neighbor components; its eigengap then counts those components rather than speakers. Absolute similarity is needed in this regime. This is a short-recording policy, not a two-speaker cap: exact/min/max hints still apply, including counts above two.

Only full-duration embedding windows provide clustering landmarks when there are enough to satisfy the requested minimum/exact count. Truncated tails down to 100 ms remain in the output activity sequence but are assigned to the full-window centroids, rather than introducing clusters from poorly supported embeddings. If there are insufficient full windows, all windows remain eligible. The landmark bound is unchanged. This full-window rule applies to both clustering methods on short recordings; longer recordings keep the original windows and S128 behavior.

`ShortRecordingThreshold` defaults to **0.3**, used only by the short-recording spectral policy. Explicit `Agglomerative` continues to use `AgglomerativeThreshold`. The choice was frozen on 24 fixed AMI development excerpts (ES2011a/b; 10, 20, 30 and 60 seconds at offsets 60 + 60i seconds, i=0..11), comparing cosine thresholds {0.15,0.3,0.45,0.65}. Threshold 0.3 with full-window landmarks had the lowest pooled DER (20.25% versus 27.12% for the previous file-helper defaults) and the best count accuracy (16/24 versus 7/24). No held-out speaker identities, counts or boundaries reach inference.

`onespeech-eval short-diarization prepare-ls CORPUS OUTPUT` creates the repeatable check set outside the repository: LibriSpeech test-clean files ordered by SHA-256 of `1729:<filename>`, the first eligible 5–15 second file per distinct speaker, 24 speakers. Twelve single clips use the first twelve speakers; each is paired with the corresponding speaker in the second twelve. Odd-index pairs repeat the two utterances once, giving alternating-speaker files in the 10–60 second range. No silence is added. `sources.tsv` records source/derived hashes and CC-BY-4.0 attribution; `manifest.tsv` names WAV/RTTM pairs. Reference speaker identity covers each entire utterance, including its silence, so DER includes VAD misses at utterance edges. This is a constructed check, not a natural conversation accuracy claim. `tools/scripts/check-short-diarization.sh` prepares and scores it; the evaluator can also score saved CLI JSON (`json:DIRECTORY`) or `Speech.Transcribe` (`transcribe`, Whisper tiny, English).

The fixed 24-recording check produced the following speaker-count matches and pooled DER (no collar, overlap scored; reference includes utterance silence):

| Path | Single count matches | Single DER | Two-speaker count matches | Two-speaker DER |
|---|---:|---:|---:|---:|
| Previous low-level/CLI defaults | 0/12 | 76.12% | 9/12 | 19.59% |
| Previous file-helper profile | 0/12 | 74.04% | 9/12 | 14.81% |
| Shared short-recording policy | 12/12 | 5.15% | 11/12 | 6.08% |

`onespeech diarize` and `Speech.Transcribe` (Whisper tiny, English) both reproduced the final counts and DER. One two-speaker pair is merged; the threshold is not retuned on these checks. These are constructed-data correctness checks, not bootstrap population estimates or comparisons against a reference diarizer. Re-scoring the four full meeting holdouts preserved the previously measured DER at the shown precision: ES2004a 28.21%, ES2004b 20.92%, R8001 41.57%, R8003 26.79%.

### Memory and resegmentation

Only the embedding sequence and interval metadata grow with file duration. Temporary PCM is bounded by 30-second buffers (plus the decoder's bounded state); dense clustering is O(L²), capped at L=256. Retained sequence storage is O(ND+N), with float embeddings requiring 4ND bytes plus window/interval/collection overhead, labels and O(KD) centroids. This is **not** constant total memory independent of duration. Default `MaxWindows=100000` (configurable up to 1000000) refuses further windows explicitly instead of growing without a limit. At the default stride, continuous speech produces approximately 14400 windows/hour; dimension 192 needs approximately 11 MB/hour of vector payload, dimension 256 approximately 15 MB/hour, before metadata. Caller-owned in-memory PCM is outside this bound.

An optional `EmbeddingResegmenter` receives borrowed windows/initial labels synchronously and returns a label per window before turn construction. The output must use only initial labels and respect count hints. The low-level constructor installs none by default; file helpers use the cosine-centroid profile below. In particular **VBx is deferred**. Landini §§2.1–2.3 requires a compatible pretrained PLDA mean, whitening/LDA transform and between-speaker variances. None is fitted or substituted here. Embeddings are never written to disk or retained by the diarizer after the call.

### Development parameters and evaluation

CAM++ English (512 dimensions, float32) was fitted on **AMI Full-corpus-ASR dev ES2011a/ES2011b only**, with the original Mix-Headset signal and word-derived references described in the fetch scripts. Window defaults remain the paper-based 1.5 s / 0.25 s; count range is 1–8, with no oracle speaker count. The grid uses 64/128/256 landmarks and AHC thresholds −0.2, −0.1, 0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.50, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85. The initial high-threshold grid was extended using development results only. Selection minimizes pooled speaker-time DER, ties by reference-speaker-macro JER then ordinal profile ID. Embeddings and VAD outputs are replayed from RAM after input SHA checks while varying clustering parameters; no embeddings are saved or model weights altered.

| CAM++ en profile | Method | Landmarks | AHC threshold | Pooled dev DER / JER |
|---|---|---:|---:|---:|
| S128, selected primary | Spectral/NME |128|0.65 numerical fallback only|33.3520% / 42.6786%|
| A64-0.15, fitted alternative | Average-link AHC |64|0.15|57.0071% / 72.1743%|

These are model-specific recipes, not a claim that the generic options are optimal for every embedder. S128 coincides with the existing spectral defaults. To use the fitted AHC alternative, explicitly set `Method=Agglomerative`, `MaxLandmarks=64`, `AgglomerativeThreshold=0.15`; retain 1.5 s / 0.25 s windows and min/max 1/8. All selections were frozen before reading holdout scores.

Channel-layout calibration used AMI dev ES2011a/ES2011b Headset-0/1 only. Ten correlated proxies total (five per meeting) cover separated microphones, 1% statistical leakage, 25%/50% leakage negatives, and dual mono. Leakage is computed on frame energy/cross-energy statistics; no new speech waveform is rendered. Searching 10/15/20/25/30 dB and normalized-dot limits 0.2/0.3/0.4/0.5/0.6 retained **20 dB / 0.3**, with 4 true positives, 6 true negatives and no errors. Ties prefer the smallest change from the initial constants. The fixed three-frame and energy-floor rules are unchanged. This is not independent telephony accuracy or evidence that the heuristic proves separation.

### Initial holdout floors

The embedder is ModelScope `iic/speech_campplus_sv_en_voxceleb_16k` revision `032b8131a7ad812f87061955ca974c99060c5a03`; converted graph SHA-256 `f333a00d5d66b9bf3d7e7abf35f36ff4a01e6e8a0df235b89a125e9fbe3cc15c`. Linux x64, AMD Ryzen 9 7950X3D (32 logical CPUs, AVX2/FMA/AVX-512 available), .NET SDK 10.0.403/runtime 10.0.14, CPU/float32, eight inference threads, OneModel 1.0.0 and OneAudio 1.7.2; no GPU or speed measurement. Silero v6.2.3 uses threshold 0.5, negative 0.35, minimum speech 250 ms, minimum silence 100 ms, pad 30 ms, with the documented independent 30 s horizons. No collar; all overlap scored; no UEM exclusions. Figures are **first-measurement floors**, not reference parity or an absolute quality pass.

| Held-out meeting | S128 DER / JER | A64-0.15 DER / JER |
|---|---:|---:|
| AMI test ES2004a, Mix-Headset |33.9706% / 38.1994%|36.2530% / 39.3777%|
| AMI test ES2004b, Mix-Headset |27.0682% / 29.1370%|28.8334% / 29.0338%|
| AliMeeting Eval R8001_M8004_MS801, far channel 1 |66.1638% / 64.4226%|67.9955% / 65.5995%|
| AliMeeting Eval R8003_M8001_MS801, far channel 1 |45.3672% / 51.7859%|47.6219% / 53.7021%|

Pooled S128: AMI DER 29.0875%, JER 33.6682%; AliMeeting DER 55.2437%, JER 58.1043%. Descriptive 95% bootstrap intervals over the **two meetings** (1,000 resamples, seed 1729) are respectively DER [27.0682,33.9706]%, JER [29.1370,38.1994]%, and DER [45.3672,66.1638]%, JER [51.7859,64.4226]%. They do not establish generalization to a new speaker/meeting population. Individual-meeting intervals are unavailable. Missed speaker-time dominates these rows; this measurement does not separate VAD misses from the known inability to represent concurrent speakers.

### Language-appropriate evaluation and published bundle comparison

CAM++ Chinese (`iic/speech_campplus_sv_zh-cn_16k-common`, revision `013c18884dffb82846cf472dd06dd4a08d97e2cd`, converted SHA-256 `4b2f0fef5ec8cb855b53c6b5abae5dd4cd72c215137c52d32f5c477e4a792f49`) improves the Mandarin rows with the **same frozen settings**, without fitting on AliMeeting Eval:

| AliMeeting Eval meeting | Chinese S128 DER / JER | Chinese A64-0.15 DER / JER |
|---|---:|---:|
| R8001_M8004_MS801 |62.3775% / 59.4958%|72.9761% / 82.1277%|
| R8003_M8001_MS801 |39.6415% / 42.5590%|48.2732% / 59.3365%|

A released ModelScope 1.40.2 public `speaker-diarization` pipeline was run black-box with its published bundle `iic/speech_campplus_speaker-diarization_common` at `0b056318fde24067871436d60d020fa01f3096b4`. Unlike passing a speaker-verification model directly to a diarization task, this is the documented bundle. It uses its own FSMN VAD, Chinese CAM++ and speaker-change locator, with default settings and no speaker-count oracle. Only auxiliary configuration paths were changed to verified local copies; network access was disabled. Pins are in `tools/oracles/diarization-modelscope/diarization-bundle.tsv`; its driver is `bundle.py` and dependency lock is under `tools/oracles/diarization-modelscope`. CPU float32, four threads/affinity to four logical CPUs, Python 3.13.17 / Torch 2.11.0 CPU, same host and scoring conventions as above. These are **different systems**, not logit-parity measurements.

| Meeting | Our S128 DER / JER (English for AMI, Chinese for AliMeeting) | ModelScope bundle DER / JER |
|---|---:|---:|
| ES2004a |33.9706% / 38.1994%|26.5359% / 29.6337%|
| ES2004b |27.0682% / 29.1370%|18.1017% / 19.2563%|
| R8001_M8004_MS801 |62.3775% / 59.4958%|34.5196% / 34.0168%|
| R8003_M8001_MS801 |39.6415% / 42.5590%|26.8887% / 28.5758%|

Every row misses a 0.5-point difference bound. A single meeting cannot supply an utterance/file bootstrap interval; these fixed slices do not prove population quality. Missed speech is the largest component in all our S128 rows. Replacing the embedder alone does not address the VAD/overlap weakness; no cause beyond this loss breakdown is established here.

A real 300-second AMI ES2004a excerpt through `Speech.Transcribe` with Whisper small and the original 0.5-threshold / 30-ms-padding activity settings produced 344 words, WER **16.0584%** against chronological timed AMI words (`en/1`, 411 reference words), 28 Unassigned, zero Overlap and zero untimed words. Under an optimal one-to-one speaker mapping, 290/344 (**84.3023%**) predicted words had the correct reference speaker; 290/335 (**86.5672%**) among words with reference-time coverage. Each predicted word's reference speaker is the one covering most of its interval, with ordinal ties. This is temporal speaker assignment, not lexical correctness; Unassigned, untimed and no-reference words count against the all-word fraction. These are informational excerpt results, not a long-form quality bar. `onespeech-eval diarize speech-excerpt DATA_ROOT ES2004a OUTPUT openai/whisper-small` writes the transcript JSON and scoring summary. Holdout references were never used to choose settings.

See [evaluation](https://recursiveai.org/onespeech/docs/eval/) for no-collar, overlap-inclusive DER/JER and RTTM input.

## Diarization-specific speech activity

The file transcription helpers use **separate** Silero instances for ASR chunking and diarization. ASR and live chunking retain the published defaults (activation 0.5, deactivation 0.35, minimum speech 250 ms, minimum silence 100 ms, padding 30 ms). The diarization instance uses activation **0.2**, deactivation **0.2−0.15** (float32, approximately 0.05), minimum speech **250 ms**, minimum silence **250 ms**, padding **150 ms**. Independent 30-second horizons and the 100-ms minimum embedding segment remain unchanged. Low-level `ClusteringDiarizer` continues to use the caller-supplied detector; it does not override its options. No ASR VAD behavior or model weights change.

These activity settings were selected only on AMI development ES2011a/ES2011b. AliMeeting Train/dev was not present and AliMeeting Eval was not used for fitting. The activity grid contains activation {0.2,0.35,0.5}, deactivation activation−0.15, minimum speech {0,250} ms, minimum silence {100,250} ms, padding {30,150} ms (24 settings). Frame probabilities are computed once per independent horizon, then replayed through the production segmenter. The best three pooled development union-activity miss+false-alarm settings were shortlisted; full S128 clustering evaluated those, the original baseline, and a shorter-window diagnostic. The final choice minimizes pooled speaker-time DER, then speaker-macro JER, then ordinal profile ID. This is a bounded shortlist search, not a claim of global optimality.

Oracle activity uses the union of the development RTTM intervals, without reference speaker labels or counts, as the clustering input. It is diagnostic only and never used on holdouts. Clustering windows, S128 settings, embedders and the absence of overlap detection are held fixed. Short-segment diagnostics lower the internal embedding cutoff from 100 to 45 ms; the production cutoff stays at 100 ms.

| Development meeting | Original DER / JER | Selected activity DER / JER | Oracle activity DER / JER |
|---|---:|---:|---:|
| ES2011a |37.7930% / 53.8078%|33.1123% / 48.2213%|18.2216% / 41.1434%|
| ES2011b |30.4951% / 31.5493%|23.5827% / 25.3175%|15.4738% / 17.9560%|

Neither baseline development meeting lost an entire region to the 100-ms embedding cutoff: zero regions, zero seconds. Ownership spans the whole retained region even when a trailing embedding window is too short. With oracle activity only one 50-ms region (ES2011a) was omitted. Lowering the cutoff recovered that 50 ms but changed the sampled embedding sequence and increased confusion: oracle DER became 21.1118% / 15.5025%, and baseline DER 40.1308% / 30.8970%. Short-window dropping is not the main missed-speech cause on these development meetings, and the shorter cutoff was not adopted.

Reproduce the bounded development search with `onespeech-eval diarize activity grid`, `shortlist`, `cluster`, and `freeze`; the command rejects nondevelopment IDs for fitting. `activity holdout` requires the frozen setting file and writes its SHA-256 into every row. No evaluation speaker labels enter selection.

Pooled development DER/JER changes from **33.3520% / 42.6786%** to **27.3133% / 36.7694%**. Baseline miss/false-alarm/confusion totals are 660.880 / 20.873 / 117.625 seconds; selected totals are 427.738 / 96.369 / 130.535 seconds. Oracle-activity totals are 306.990 / approximately 0 / 89.667 seconds (16.5495% DER). Of oracle misses, **306.940 seconds** are the unavoidable extra speaker-time during reference overlap for an exclusive single-speaker output; only 0.050 seconds come from the short-region cutoff. The original avoidable activity miss is about 353.940 seconds, larger than oracle-activity confusion. Oracle activity changes embedding segmentation as well as coverage, so this is a diagnostic breakdown rather than an additive guarantee for a replacement VAD.

The selected setting `T20-M250-S250-P150-C100` was frozen before held-out scoring (selection SHA-256 `c4d50356d42f1c4b0bd82a953ab94c9a1cae60f9b2e841e5a11b3b01bb49fed0`). Each of the four held-out meetings was then run once, without an oracle activity pass or further tuning. All speech-activity, model, clustering, precision and scoring details remain as above except the selected activity profile. Values below supersede the earlier activity floors for the helpers:

| Meeting | Selected DER / JER | Selected miss / FA / confusion (s) | Published bundle DER / JER | Bundle miss / FA / confusion (s) |
|---|---:|---:|---:|---:|
| ES2004a |28.8213% / 31.0004%|180.754 / 42.453 / 42.938|26.5359% / 29.6337%|176.510 / 49.150 / 19.380|
| ES2004b |21.1057% / 19.9543%|298.314 / 90.640 / 82.346|18.1017% / 19.2563%|282.540 / 84.210 / 37.470|
| R8001_M8004_MS801 |42.4480% / 42.3801%|771.504 / 19.706 / 78.652|34.5196% / 34.0168%|616.960 / 46.150 / 44.280|
| R8003_M8001_MS801 |27.2741% / 29.8615%|418.452 / 117.852 / 81.658|26.8887% / 28.5758%|414.640 / 127.700 / 66.890|

All four DER/JER pairs improve versus their earlier same-embedder floors. R8003 DER is within 0.5 points of the bundle; its JER and both metrics on the other meetings remain outside that bound. Single-file intervals remain unavailable. On AMI, the remaining bundle gap is primarily confusion (excess 23.558 / 44.876 seconds), not activity misses alone. R8001 still has a substantial missed-speech gap. No new segmentation/speaker-change model or overlap detector is added, and these different-system comparisons do not establish numerical parity.

## Cosine-centroid resegmentation

`CosineResegmenter` is an optional embedding-only algorithm, not a learned
speaker-change model or VBx replacement. It uses the existing 1.5-second
windows at 0.25-second hop; it does not extract finer audio windows or claim
sub-hop precision. VAD coverage and the midpoint ownership rule stay unchanged.
No extra waveform pass, model, biometric persistence or overlap detection is
introduced.

For each initial cluster, average the unit embeddings of stable windows, then
normalize the sum. A window is stable when its initial label and the labels of
`StableRadius` neighbors on either side agree, with no activity gap in that
support. If a cluster has no stable windows, use all its windows. A near-zero
centroid norm (less than 1e-12) leaves the whole initial labeling unchanged.
Cosine distance to these fixed centroids is the per-window assignment cost.
An exact minimum-cost path minimizes the sum of assignment costs plus
`SwitchPenalty` for each label change. Costs are per window, not per second;
the penalty is soft smoothing, not a hard minimum-turn duration. The path is
reset at a positive gap between embedding windows. Ties prefer staying, then
the lowest original label. Speaker IDs are canonicalized by the enclosing
diarizer after refinement. No speaker outside the initial labels is created;
a speaker can disappear, and the enclosing diarizer still enforces explicit
count/min/max constraints. It refuses a violating result rather than ignoring
a count request.

This is our own deterministic centroid/path design. It does not implement the
publisher's acoustic transformer locator: its public API returned rounded
change-point text, while its published card and paper do not specify a complete
inference graph. That learned locator is not loaded by these helpers.

Public algorithm defaults are penalty 0.1 and radius 2; these are starting
values, not a universal calibration. Limits: up to 1,000,000 windows, 32 distinct
speakers and dimensions 1–4096, matching the clustering bounds. Path storage
uses NK bytes for byte predecessors plus 4N bytes for output labels; centroid
accumulators require at most 16KD bytes, plus small DP buffers and collection
overhead. There is no additional PCM storage. Existing embedding-sequence
storage remains O(ND), explicitly capped by `MaxWindows`, not constant total
memory for arbitrarily long input. At the default N cap of 100,000, additional
path payload is at most 3.6 MB. Cancellation is checked per window and during
traceback.

### Selected file-helper profile

The file helpers, CLI and low-level `ClusteringDiarizer` use
`CosineResegmenter(switchPenalty: 0.4, stableRadius: 2)` by default.
Set `ClusteringOptions.Resegment = false` to retain raw labels, or pass a custom
resegmenter (which takes precedence). The default smoother is also omitted
when an exact count or minimum above one is requested, because unconstrained
smoothing can remove a required label; custom backends still must honor hints.
The CLI and file helpers both use
`SileroVadOptions.Diarization`. The low-level API deliberately borrows its
caller-supplied detector unchanged; use that same profile when constructing
Silero for equivalent activity behavior. The development-only search compared no resegmentation with all
15 combinations of radius {0,2,4} and penalty {0,0.05,0.1,0.2,0.4}, using the
already selected activity profile and S128 clustering unchanged. All windows
were the original 1.5 s / 0.25 s windows. Selection minimizes pooled development
speaker-time DER, then macro JER and ordinal setting ID; no resegmentation wins
a tie against the baseline. No AliMeeting Eval or AMI test labels were used.

| AMI development | No resegmentation DER / JER | Selected DER / JER |
|---|---:|---:|
| ES2011a |33.1123% / 48.2213%|31.5613% / 47.1127%|
| ES2011b |23.5827% / 25.3175%|22.1844% / 23.0626%|
| Pooled |27.3133% / 36.7694%|25.8552% / 35.0877%|

Pooled development confusion falls from 130.535 to 95.589 seconds. Miss
427.738 seconds and false alarm 96.369 seconds are unchanged: this algorithm
cannot recover activity or represent overlap. Profile `R2-P400` was frozen
before holdout inference, SHA-256
`1d57269880fdccc38b635a53cf19776041c178a06e3419c5bcfd42bda7d53f13`.
`onespeech-eval diarize activity refine-dev`, `refine-freeze` and
`refine-holdout` reproduce the search and fixed holdouts; the holdout command
checks the freeze digest and refuses to overwrite scores. Development embeddings
are cached only in RAM and cleared after each meeting.

The four complete holdouts were each run once with that frozen profile.
Precision, machine, model pins, activity settings and no-collar/overlap-inclusive
scoring are the same as the preceding activity rows. CPU float32, eight graph
threads; OneModel 1.0.0, OneAudio 1.7.2, SDK 10.0.403/runtime 10.0.14. The
reference remains the pinned ModelScope 1.40.2 published bundle, not this
centroid algorithm. No new reference inference was needed.

| Meeting | Before DER / JER | Refined DER / JER | Refined miss / FA / confusion (s) | Published bundle DER / JER |
|---|---:|---:|---:|---:|
| ES2004a |28.8213% / 31.0004%|28.2116% / 29.6213%|180.754 / 42.453 / 37.308|26.5359% / 29.6337%|
| ES2004b |21.1057% / 19.9543%|20.9156% / 19.7743%|298.314 / 90.640 / 78.102|18.1017% / 19.2563%|
| R8001_M8004_MS801 |42.4480% / 42.3801%|41.5741% / 41.1256%|771.504 / 19.706 / 60.743|34.5196% / 34.0168%|
| R8003_M8001_MS801 |27.2741% / 29.8615%|26.7902% / 28.9546%|418.452 / 117.852 / 70.694|26.8887% / 28.5758%|

All four improve over the preceding same-activity results, but the confusion
gap is not closed. Using a 0.5-point single-file difference bound, only ES2004a
JER and both R8003 metrics agree with the reference; ES2004b JER differs by
0.518 points and also misses. Single-file confidence intervals are unavailable.
These fixed meetings do not establish population quality. This is not parity
with the learned locator or the reference's full system.


<!-- https://recursiveai.org/onespeech/docs/online-diarization/ -->

# Live speaker labels

`Speech.Listen` adds bounded online clustering to live recognition. The low-level
`OnlineDiarizationSession` decorates a **fresh** recognition-only `LiveSession` and
borrows its session and `SpeakerEmbedder`; dispose the decorator first, then its
borrowed resources. Push mono 16 kHz blocks of at most ten seconds. Complete both
recognition and speaker output by completing the decorator, not the inner session.
Do not push or complete the inner session separately.

At each recognized utterance closure, the retained PCM is partitioned into
nonoverlapping windows of at most 1.5 seconds. A residual tail below 45 ms shares
the preceding window's duration by splitting the last two windows evenly. A whole
utterance below 45 ms is Unassigned. Each window's normalized embedding is compared
to unit centroids by cosine similarity. The best qualifying centroid wins;
exact ties prefer the oldest ID. Otherwise a new ID is allocated. The default
minimum similarity is 0.3 (AMI development selection, frozen parameters in
`eval/manifests/online-clustering.json`). The default capacity is sixteen per
embedding space; an unmatched window at capacity stays
Unassigned instead of being forced into an existing speaker. Centroids use
bounded-weight updates (weight increases to 100, then update weight is 1/101).
No speaker count is inferred from future speech.

`SpeakerTurnEvent`s precede the utterance's `FinalEvent`. The final also carries `SpeakerTurns`, an immutable snapshot of the same labelled
activity, so a recognizer such as Moonshine can expose speaker labels on finals
even when it supplies no word timing. This is not a claim that an untimed word
belongs to a particular window. Its timed words
receive the speaker with the greatest positive interval overlap (earliest turn on
a tie). Untimed or uncovered words are Unassigned. Word times are not invented.
Partials remain unlabelled. Turns have empty text: this is activity attribution,
not a second transcription. Turn intervals cover the recognizer's retained
utterance, including its VAD padding, and are **not** precise word boundaries.
If recognition emits no final, it supplies no interval to diarize.

Finals and their word labels are immutable. IDs are never retrospectively merged,
renumbered, or reused. If two early IDs later appear to be the same speaker, both
remain; later windows can match either centroid. This intentionally sacrifices
offline global consistency. Do not interpret these session-local IDs as enrolled
identities. No embeddings or audio are persisted; disposal erases working arrays.

`Speech.Listen` uses CAM++ English for non-Chinese requested languages and CAM++
Chinese for Chinese (`zh`, `zh-*`, `Chinese`). With automatic language detection,
it loads both models and selects by each final's language. Their separate
embedding spaces have distinct ID prefixes; a language switch can split one
person into two IDs. Explicit language avoids loading the unused model. Downloads
follow the usual first-use cache rules.

## Latency and limits

No future audio beyond the recognizer's own utterance closure is used; clustering
adds zero audio lookahead, but adds synchronous embedding CPU work at the endpoint.
A window near the beginning of a long utterance therefore waits for that entire
utterance (by default at most about 30 seconds, plus endpoint silence and caller
chunk granularity). Input has to be fed at a sufficiently small chunk size for the
latency the application needs. Speaker updates are not low-latency change-point
detection. A duration-limited endpoint commits even if speech continues.

Storage is independent of recording length: 320 seconds of PCM, a 1.5-second
scratch window, bounded centroids and at most one utterance's output. Finals must
span at most 300 seconds and still be retained; unsupported intervals are refused,
never silently clipped. Audio-time stamps are relative to the first input sample.
The decorator forwards recognition events on that same clock and never reads wall
time. Model inference and immutable emitted snapshots allocate; centroid assignment
and non-emitting decorator pushes need no additional allocation. Endpoint scratch
and word/turn construction allocations are reported separately, not claimed zero.

## Limits versus offline diarization

This is a causal floor, not equivalent to the offline spectral clustering and
resegmentation pipeline. It cannot detect overlap, split simultaneous speakers,
use future context to repair an early decision, or reliably identify a short
speaker interjection. Fixed windows may straddle changes; VAD errors and empty ASR
output also affect coverage. Use offline diarization when final global consistency
matters. A tier-2 Nemotron end-to-end live upgrade is planned, **not currently
available**; it must be selected explicitly once supported, never silently used.

The evaluation tool `OneSpeech.OnlineDiarization` streams prepared mono WAV in
100 ms chunks through `UtteranceLiveSession` plus this same decorator. It uses an
activity-only recognizer to isolate diarization from ASR lexical failures, not
oracle speech intervals. Silero runs on the input; no reference boundaries or
speaker counts enter inference. It scores all emitted activity against the
prepared RTTM using no collar and scoring overlap. Fit similarity only on AMI
development, freeze the configuration/hash, then score holdouts. Allocation and
CPU diagnostics are indicative, not benchmark claims.


<!-- https://recursiveai.org/onespeech/docs/nemotron-diarization/ -->

# Nemotron 3 Diarization

Speech recognition for .NET. `OneSpeech.Diarization.NemotronDiarizer` is an explicitly selected,
overlap-aware, tier-2 file diarizer for up to eight anonymous speakers. It is never a default.
The [clustering documentation](https://recursiveai.org/onespeech/docs/diarization/#voice-clustering-and-overlap-limitation) shows the
one-call pipeline replacement. This is diarization, not source separation or identity recognition.

## Checkpoint and loading

Only `nvidia/Nemotron-3-Diarization` revision
`f667ed73aee57d40cc39428eb768b4fd87a0a29e` is accepted. The publisher's float32
`model.safetensors` is 396,954,592 bytes, SHA-256
`c074d86335b3b794f8fa5edc25594558f128bdb3914d27806a3a5a2e44963cb6`.
`LoadAsync` uses `ModelStore` to fetch and verify this file and build runtime graphs locally,
under the model-cache lock. `Load(ResolvedModel)` loads an already resolved cache only.
Local runtime hashes are fixed registrations, not download or mirror URLs. There is no
executable publisher serialization, remote Python, redistributed tier-2 graph, or default selection.
Publisher origin and the full OpenMDW-1.1 agreement remain in `ModelInfo`.

Instances borrow no audio beyond processing and own disposable CPU sessions. Do not process
concurrently on one instance. `DiarizeBlocks` accepts consecutive 16 kHz blocks and averages
channels; `AudioFileSource` supplies decoded/resampled file blocks. The transcription pipeline
makes a diarization pass and a recognition pass, so it does not keep the waveform in memory.

## Features and graph

The pinned card/config specifies a 31-layer, width-512, eight-head Transformer with pre-block
LayerNorm, exact-form GELU, RoPE and a final LayerNorm. It is not a FastConformer encoder.
Eight successive 128-bin mel frames are stacked and projected to width 512. Features use the
shared deterministic NeMo log-mel extractor without utterance normalization or dither: 16 kHz,
25 ms symmetric Hann window, 10 ms hop, 512-point centered, zero-padded STFT, preemphasis
0.97, Slaney filters and natural-log energies with an additive 2^-24 guard. The centered trailing
frame is retained but masked to zero. Subsampling padding is zero; an encoder group containing
only padding is excluded as an attention key. Inputs shorter than one complete 10 ms hop yield
no turns.

The classifier projects 512 to 192, convolves over time with kernel 3/padding 1 and 1536 output
channels, and rearranges factor-major channels to eight 10 ms frames. ReLU precedes the dense
192-to-192 projection and follows it; the last projection produces eight activity logits. Sigmoid
is applied independently, so simultaneous speakers remain present. Head ordering and activation
placement were checked through the released public bare-model API, not implementation source.

Parity probes use float32 and OneModel fast arithmetic, not its cross-platform bit-identical exact
mode. The targeted short feature/encoder/probability probes use max-absolute tolerances 1e-4.
Long-recording quality is checked by DER/JER rather than promising token- or probability-bit identity
through discrete cache selection.

## Bounded chunking and speaker cache

The offline-style card profile is fixed: speaker cache 264, FIFO 40, new chunk 340,
right context 40, cache update period 300, in 80 ms encoder-frame units. A model window is at
most 684 frames. RoPE positions restart at zero for each model window. The waveform buffer holds
30.4 s plus small STFT edge context; carried left-edge samples make features independent of caller
block boundaries. No VAD removes gaps before diarization: the original audio clock is preserved.

Following Streaming Sortformer §§3.1–3.3, the cache stores projected input frames, not Transformer
KV state. Activity is averaged over each group of eight output frames. Frame scores use
`log(max(0.25, p[s])) + sum(log(max(0.25, 1-p[j])))` for other speakers, with scores disabled below
p[s]=0.5. If a speaker has at least 16 frames whose joint exclusive activity exceeds 0.5,
its nonexclusive frames are excluded before bonuses. Recency adds 0.05 starting at the nominal
264-frame cache boundary, including the first compression when the cache was empty. These
boundary/filter semantics were established with crafted public-cache probes. Per-speaker representation boosts are 24 frames
by 2 log(2), and 48 frames by log(2). Strong selection precedes weak selection, which uses the already boosted scores. A partition-based
TopK includes one infinite-score learned-silence candidate per speaker in the global 264-slot
selection; its treatment of tied scores matches the released CPU oracle on crafted cache probes.
Selected non-silence slots are grouped by speaker and original frame order. Cached frames retain
their original activity scores rather than being relabelled by later chunks, as verified by a
two-update probe switching the active speaker. Learned silence follows represented speakers; unused/invalid slots sort to
the tail. This silence placement was checked with both crafted cache inputs and observed public
cache outputs; placing unused slots in front changes arrival-order identities.

Only bounded audio/features, FIFO/cache and two cached execution plans are retained. Returned
turn/change-point metadata necessarily grows with conversation activity; no duration-independent
bound is claimed for the returned document itself. Embeddings are temporary frame features and
are never persisted by the library.

## Turns, overlap and word assignment

A speaker is active at probability ≥0.5 on the 10 ms grid. Adjacent active frames coalesce per
speaker; intervals may overlap. Endpoints are clipped to actual audio duration. Empty audio
returns no turns. Nonfinite PCM is rejected and other rates require resampling.

`DiarizeActive`/`DiarizeActiveBlocks` additionally retain changes in the highest-probability active
speaker. Relative activity is 1 for that channel and 0.5 for other simultaneous channels, not a
calibrated probability. Exact ties choose the lower channel index. The pipeline uses this view to
apply its ordinary majority-covered-duration rule, `Overlap` flag and `Unassigned` gaps, without
changing words or timing. The returned transcript records the tier-2 model and licence provenance.

## Reference evaluation

The reference is the released Transformers **5.22.2** public Nemotron diarization API, float32,
CPU, batch size one, using the same publisher safetensors and offline-style profile. Oracle
runners are isolated under `tools/oracles/nemotron` with a locked `uv` environment. Their console
output is discarded; only bounded statuses and explicit output artifacts are exposed. NeMo 3.0.0
cannot load this publisher checkpoint's RoPE encoder; its sanitized repro is retained under
`tools/oracles/nemo`. No unreleased source build is substituted.

The four targeted comparisons use AMI ES2004a/ES2004b Mix-Headset and AliMeeting Eval
R8001_M8004_MS801/R8003_M8001_MS801 far-field channel 1, with the existing preparation/reference
labels, no collar, and overlap scored. Both hypotheses are scored by `DiarizationMetrics` in
`onespeech-eval`; a per-file comparison uses a 0.5 percentage-point bound, not an utterance bootstrap.
These labels differ from the publisher card's forced-aligned benchmark protocol, so card benchmark
numbers are not claimed as reproduced.


<!-- https://recursiveai.org/onespeech/docs/gated-segmentation/ -->

# Gated models and segmentation

Gated entries are tier 2, explicitly named, never defaults or mirrors. The
catalog pins publisher revisions and each file's SHA-256. `ModelStore.PullAsync`
takes an ephemeral API token, falling back to `ONESPEECH_MODEL_TOKEN` only when
the API argument is null. It checks gate credentials even for cached files.
A missing or whitespace token throws `ModelTokenRequiredException` before I/O.

The HTTP transport sends Bearer credentials only on the initial request to the
catalogued publisher file URL. Mirrors and every redirect target, including
same-host redirects, receive no credentials. Redirects must remain HTTPS, without
userinfo or custom ports, and stop after five hops. HTTP 401/403 responses throw
`ModelTokenRejectedException`, unless the publisher's `X-Error-Code` is
`GatedRepo`, which throws `ModelGateNotAcceptedException`. A server that does not
distinguish a gate refusal from a rejected credential cannot be classified more
precisely. Response bodies, error-message headers, credentials and signed URLs
never enter library exceptions, cache files or `ModelInfo`. No response body is
parsed to guess gate state. Custom `ModelSource` implementations are responsible
for the same confidentiality rules and are trusted host code.

## Local segmentation-3.0

After accepting the publisher's gate and obtaining the pinned snapshot, load it
without network access:

```csharp
using var segmentation = PyannoteSegmentation.LoadLocal(snapshotDirectory);
using var diarizer = new PyannoteDiarizer(segmentation, embedder);
var turns = diarizer.Diarize(audio);
```

These types are in `OneSpeech.Diarization`. The embedder is an independently
loaded 16 kHz `SpeakerEmbedder`, for example CAM++. The diarizer borrows both
models. Neither is a new default. The original checkpoint is read-only and its
SHA-256 must match the catalog's segmentation-3.0 revision. Conversion uses a
bounded inert checkpoint-data reader: no Python objects or code are executed.
Derived graphs and the MIT notice are written only to the specified user cache;
no publisher weights are copied there. `ModelInfo` includes upstream and derived
digests, the publisher attribution and full MIT notice. Derived tier-2 graphs
must not be mirrored, packaged or redistributed.

`GetFrameLogProbabilities` accepts exactly 160000 mono samples at 16000 Hz and
returns 589 × 7 row-major log-probabilities. Class order is silence, speaker 1,
speaker 2, speaker 3, speakers 1+2, 1+3, speakers 2+3, as in the model card. Frame
centers are `(495 + frame * 270) / 16000` seconds. Inference uses float32 OneModel
fast arithmetic, not a claim of bitwise exact mode. Compared with pyannote.audio
4.0.7 / CPU PyTorch 2.11.0, the probability-space max-absolute tolerance is 1e-5
on five fixed LibriSpeech clips and 5e-5 on crafted silence/tone/chirp inputs.
Log-probability errors are reported separately: very small probabilities can
have appreciable log error, especially on constant input after instance norm.
Powerset classes are selected by argmax, with ties going to the first class.

### Graph dataflow

The pinned checkpoint is a PyanNet-style network. Waveform instance
normalization uses the learned affine parameters and population variance with
`epsilon = 1e-5`. Its analytic SincNet bank has 40 real/imaginary pairs of
251-sample filters, stride 10. Low cutoffs are `50 + abs(low_hz_)`; high cutoffs
are `clamp(low + 50 + abs(band_hz_), 50, 8000)` Hz. The checkpoint supplies the
negative half-grid `n_` and half-window `window_`. For bandwidth `b = high-low`,
the real half-filter is `(sin(high*n)-sin(low*n))/(n/2) * window/(2*b)`;
the imaginary half-filter replaces the sine difference with
`cos(low*n)-cos(high*n)`. The real filter is symmetric, center 1; the imaginary
filter is antisymmetric, center 0. Real filters precede imaginary filters.

The first convolution is followed by absolute value. Each of the three
convolutional stages then applies max pooling (kernel/stride 3), affine instance
normalization (`epsilon = 1e-5`), and leaky ReLU (negative slope 0.01). The second
and third convolutions have 60 outputs, kernel 5 and stride 1, with checkpoint
biases. Four bidirectional LSTMs follow, 128 units per direction, concatenating
forward then backward features. Two 128-unit affine layers each use leaky ReLU
(0.01); the seven-unit classifier uses log-softmax. Dropout is inactive at
inference. LSTM gate weights are reordered from checkpoint i/f/g/o to the
OneModel/ONNX i/o/f/c layout without changing learned values.

## Overlapping turns

`PyannoteDiarizer` preserves separate local-speaker masks instead of reducing
them to a speech union. It uses independent 10 s windows, pads the final window,
and clips activity to real audio. Interior frame boundaries are the midpoints
between frame centers, rounded down to a sample; outer boundaries reach the
window edges. Embeddings use concatenated exclusive-speech samples where at
least 100 ms is available, otherwise the speaker's full activity mask, including
overlap. A local speaker with less than 100 ms total activity has no embedding
and produces no attributed turn.

The existing spectral/AHC algorithm clusters these embeddings globally. The
minimum cluster count is at least the largest local-speaker count; contradictory
count hints are refused. Per-window maximum-cosine injective assignment prevents
two local speakers from collapsing onto the same global label. Adjacent turns
of the same speaker are merged; simultaneous turns of different speakers are
preserved. This is our segmentation-plus-clustering policy, **not community-1
equivalence**: community-1 uses another embedder, its supplied PLDA and VBx.
No published PLDA is fitted or reused with incompatible CAM++ embeddings.

`PyannoteSegmenter` also implements `VoiceActivityDetector`. Passing it to
`ClusteringDiarizer` uses only the union of speech and does **not** detect overlap.
Use `PyannoteDiarizer` for overlapping turns.

## Network entry point and community-1

`PyannoteSegmentation.Load` retains its token refusals: without credentials it
throws `ModelTokenRequiredException`; with credentials it currently throws
`FeatureUnavailableException`. It never silently falls back to another model.
Use `LoadLocal` for an authorized segmentation-3.0 snapshot. Local loading does
not attest that a user accepted a gate; obtaining that snapshot is their
responsibility. Community-1 is catalogued and can be explicitly prefetched, but
has no native pipeline adapter. Its reference pipeline is evaluation-only.


<!-- https://recursiveai.org/onespeech/docs/speaker-recognition/ -->

# Speaker recognition

Diarization asks **who spoke when**, using anonymous labels. Recognition asks whether speech matches an enrolled identity. `InMemorySpeakerRegistry` borrows a `SpeakerEmbedder`, holds enrollment vectors only in memory and clears them on disposal. Keep the embedder alive until the registry is disposed. It does not persist a voice database.

## File and stream helpers

```csharp
OneSpeech.Speech.Enroll("alice", "enrollment.wav");
OneSpeech.SpeakerMatch match = OneSpeech.Speech.Identify("query.wav");
double cosine = OneSpeech.Speech.Verify("first.wav", "second.wav");
OneSpeech.Speech.ClearSpeakers();
```

`SpeakerMatch.SpeakerId` is the best enrolled name, or null for unknown. `Score` is raw cosine, not a probability; an empty registry returns negative infinity. `Verify` scores two samples without enrollment and does not make a security decision. Enrollment replaces a case-sensitive name only after successful extraction. All helpers have explicit `CancellationToken` overloads. The only optional selectors are `language` and `model`: English CAM++ by default, Chinese CAM++ for `zh`, `zh-*` or `Chinese`, or either embedder's catalog ID explicitly. Enrollment spaces are separate per model. Other model IDs are refused with `CapabilityNotSupportedException`.

Samples are downmixed and resampled to mono 16 kHz, must contain at least 45 ms and at most 60 seconds, and should contain one consenting speaker's speech. These helpers do not automatically select speech or separate multiple speakers. Silent or extremely short samples cannot establish a reliable identity. Missing models download on first use; loaded speaker models remain in memory until `ClearSpeakers`. Operations are serialized so replacement and clearing are atomic with respect to speaker queries. Clearing is best effort, not a guarantee against memory dumps or runtime copies.

`EnrollAsync(name, stream)`, `IdentifyAsync(stream)` and `VerifyAsync(firstStream, secondStream)` consume readable encoded-audio streams from their current positions and leave them open. Encoded input is temporarily spooled to disk for decoding and deleted on success, cancellation or error. Enrollment embeddings are never persisted. CPU model execution stays synchronous; only stream and model I/O is awaited.

`Speech.Transcribe` / `TranscribeAsync` identify each diarization cluster's normalized mean window embedding against compatible enrollments. Accepted names populate `SpeakerLabel.Name` on turns and assigned words. IDs remain `speaker-1`, etc.; unassigned words and rejected clusters are unchanged. Different clusters may match the same name; this does not merge clusters or alter timings. Streaming segment and live event helpers do not retrospectively name speakers. A later clear or reenrollment never mutates a returned transcript.

## Low-level registry

The following method accepts an already loaded embedder and consented mono PCM at its required sample rate. Choose the identification threshold on disjoint development trials; 0.3 here is illustrative, not a universal biometric operating point.

```csharp
static void RecognizeSpeaker(OneSpeech.SpeakerEmbedder embedder,
    OneSpeech.AudioBuffer enrollment, OneSpeech.AudioBuffer query,
    CancellationToken cancellationToken)
{
    using var registry = new OneSpeech.Speakers.InMemorySpeakerRegistry(embedder, threshold: 0.3);
    registry.Enroll("alice", enrollment, cancellationToken);
    var match = registry.Identify(query, cancellationToken);
    double score = registry.Verify(enrollment, query, cancellationToken);
    Console.WriteLine(match);
    Console.WriteLine(score);
}
```

Enrollment replaces the named identity. Multiple utterances are normalized and averaged with equal weight. Identification may return unknown below the threshold; verification returns a similarity score, **not a probability or calibrated identity decision**. See [embedding and scoring details](https://recursiveai.org/onespeech/docs/speakers/), and [results](https://recursiveai.org/onespeech/docs/results/#speaker-embeddings) for the limited trial population. File-based `enroll`, `identify` and `verify` are also available in the [CLI](https://recursiveai.org/onespeech/docs/cli-and-model-packages/). The low-level registry retains explicit caller-controlled thresholds and multi-utterance enrollment.

Embeddings are biometric data. Obtain informed consent and check local law, including Illinois BIPA and GDPR Article 9. Apply appropriate access controls and deletion policies to enrollment audio, transcripts and application logs. Not persisting embeddings in this library does not remove the host's legal obligations. Never identify a person merely from a document-local diarization label. This is not legal advice.


<!-- https://recursiveai.org/onespeech/docs/speakers/ -->

# Speaker identity and scores

`SpeakerEmbedder` is the synchronous CPU extraction role. Its model metadata identifies the
checkpoint and licence; `Dimension` fixes output length and `SampleRate` fixes mono input Hz.
`Embed` writes into caller-owned storage and accepts cancellation. Input must be nonempty,
mono, and already resampled. Implementations return finite, nonzero vectors and reject unsupported
input rather than silently changing channels. Callers dispose model resources after consumers stop.
`CampPlusEmbedder` loads a locally converted CAM++ graph; see below.

## Arithmetic

`EmbeddingMath.Cosine` accumulates in double precision and clamps rounding error to [-1,1].
Both vectors must be finite, nonempty, equal-dimensional and nonzero. `Normalize` supports
in-place normalization. `Average` first unit-normalizes each vector, computes an equal-weight
mean, and unit-normalizes that mean. Opposite vectors whose mean is zero are an error, not an
invented speaker identity. The functions neither retain nor persist their inputs.

`AdaptiveScoreNormalizer` is optional symmetric adaptive cohort normalization (AS-norm).
It copies a cohort of equal-dimensional vectors. For each side of a trial, take its highest
`topCount` cosine scores against the cohort (default 100, capped at cohort size, minimum two).
Compute their mean and population standard deviation. The returned score is
`0.5 * ((cosine - mean1) / std1 + (cosine - mean2) / std2)`.
Standard deviations are floored at 1e-6 by default. Scores are unbounded, not probabilities.
Use a disjoint, consented development cohort; evaluation identities must not enter the cohort.
The library does not select, fetch or persist one.

`LogisticScoreCalibration` fits an affine sigmoid to labeled **development scores**, not
speech-model weights. Labels indicate same-speaker trials. The objective is mean binary
cross-entropy plus half the specified L2 penalty times the sum of squared coefficients.
Both coefficients are penalized; the default penalty is 1e-4. Scores are standardized with
population standard deviation floored at 1e-6. Zero-initialized, fixed-step gradient descent
runs 100 iterations by default, with step `1 / (0.5 + penalty)`; there is no randomness.
Both classes must occur. Output probabilities reflect the development set's empirical class
prior, not a deployment population prior. Changing trial balance changes their meaning.
Choose a deployment threshold on held-out data, never on the reported evaluation trials.

## In-memory registry

`InMemorySpeakerRegistry` implements the existing `SpeakerRegistry` contract. It borrows a
`SpeakerEmbedder`, serializes its own extraction calls, and stores only unit-normalized
averages keyed by case-sensitive IDs. The caller must coordinate any embedder use outside
this registry. Enrollment replaces an ID atomically after all utterances succeed; the overload
accepting multiple utterances uses the arithmetic above. A failed or cancelled extraction leaves
the previous enrollment intact. Very short or silent audio remains the embedder's responsibility.

The constructor requires an explicit, inclusive acceptance threshold; no universal threshold is
claimed. Identification returns the highest score, breaking exact ties by ordinal ID. Below the
threshold it returns null ID with the best score. An empty registry returns null ID and negative
infinity. Verification scores a pair without modifying enrollment. Raw scoring is cosine unless
an optional AS-norm cohort is supplied. Calibration is separate and is never silently applied.
`Remove` clears an enrollment and returns whether it existed. `Dispose` clears all enrollments,
but does not dispose the borrowed embedder. Clearing managed arrays is best effort: it is not
a memory-erasure guarantee against runtime copies, crash dumps or a hostile host.

There is no embedding serialization, file storage or network enrollment. Hosts can share one
registry instance in their process; the separate `Speech` helpers manage a process-wide registry
per supported CAM++ model, with calibrated identification thresholds described below.

**Privacy:** speaker embeddings and enrollment audio are biometric data. Obtain appropriate
consent and determine applicable local-law requirements, including Illinois BIPA and GDPR
Article 9 where relevant. Avoid logging embeddings, restrict access to process memory, and
remove enrollments when no longer needed. The library never persists embeddings or audio.
Arithmetic tests use algebraic vectors. Speaker integration tests use public LibriSpeech speech only; embeddings are never committed.

## Checkpoint inspection

The development converter can fetch and SHA-256 verify four publisher checkpoints:

```sh
dotnet run --project tools/OneSpeech.Convert -- speaker-fetch /caller/selected/model-cache
dotnet run --project tools/OneSpeech.Convert -- speaker-inspect /caller/selected/checkpoint
```

The fetch command verifies existing files too; a mismatch fails rather than overwriting an
existing checkpoint. Downloads are bounded to 128 MiB, verified before rename, and temporary
files are removed on failure. Do not run concurrent fetches against the same directory.
Inspection prints only tensor names and dimensions, not a runtime graph or inferred architecture.
These commands do not register a model or select a default embedder.

Inspection supports a deliberately restricted tensor-only ZIP/pickle data language. It never
executes Python or invokes globals. Only inert ordered-map, storage and tensor-view constructors
are accepted; all other global names, reductions and object state are refused. Float32, float64,
float16 and int64 storage can be converted to float32 arrays, including bounded noncontiguous
views. Optional ordered-map metadata is parsed under the same bounds and discarded. No archive
entry is extracted to disk. All index text is strict UTF-8.

Limits: 256 MiB archive, expanded entries, and output arrays (each budget independently);
8 MiB central directory and pickle index; 20,000 ZIP entries; rank at most 8;
dimensions at most 10,000,000; no negative strides or views outside storage; two million parser
operations and aggregate collection slots; 200,000 stack and memo entries; 65,536-byte strings.
Multi-disk and ZIP64-only directory records are not supported. Invalid data throws
`InvalidDataException`; ordinary stream I/O failures remain I/O errors. Inspection is a developer
tool, not the shipped model-loading API. No embedding model implementation or parity claim is
provided by checkpoint inspection.

## CAM++ inference

`OneSpeech.Embedders.CampPlusEmbedder` implements `SpeakerEmbedder` over a locally converted
OneModel graph. Construct it with an explicit graph path, truthful checkpoint `ModelInfo`, and
embedding dimension (512 for the English VoxCeleb checkpoint; 192 for the Chinese common
checkpoint). The default is one CPU thread. It never downloads, trains, saves embeddings, or
selects a default model. Input is mono 16 kHz float PCM, 45 ms through 60 s; destination storage belongs to the caller.
Inference calls and disposal are serialized per instance. Cancellation is checked during feature
extraction and before graph inference; OneModel's synchronous graph call itself is not interruptible.

`SpeakerFbank` computes 80 natural-log mel energies on complete 400-sample windows at 160-sample
hops. It uses unscaled float PCM, removes each window's DC component, applies 0.97 preemphasis
(with the first sample repeated), a symmetric Povey window, and a zero-padded 512-point power FFT.
Mel triangles span 20 Hz through 8 kHz on the mel scale. Energies are floored at float machine
epsilon before taking natural logs. Each feature dimension has its utterance mean removed.
FFT twiddles are cached at construction-independent fixed sizes. Mel accumulation visits only each triangle's nonzero support, in the original bin order; there is no SIMD reduction or change of rounding order. The optimized path is bit-identical to the dense arithmetic on five fixed LibriSpeech probes and the synthetic test inputs. All power bins, including zero-weight bins, are checked for overflow.
Output is planar `[mel, frame]`, not interleaved frames. These settings were observed by a
pass-through capture at the documented public fbank API invoked by ModelScope 1.40.2. The wrapper
returned all values unchanged; no implementation source was inspected. The extractor is deterministic,
requires finite samples, and accepts 25 ms through 60 s. A partial final window is not padded.

The converter implements Wang et al. (2023), Sections 2.1–2.4: frequency-downsampling residual
FCM, densely connected TDNN blocks of 12/24/16 layers with growth 32 and dilations 1/2/2, and
context-aware masks combining global and consecutive 100-frame segment means. An incomplete last
segment is divided by its actual frame count. All nonlinear normalization stages use BN then ReLU.
Statistics pooling concatenates the mean and unbiased standard deviation before the final projection
and non-affine batch normalization. Architecture details left unstated by the paper were settled by
black-box numerical probes with identical input features. All neural operators run in OneModel;
there is no native inference dependency.

```sh
dotnet run --project tools/OneSpeech.Convert -- speaker-convert-camp checkpoint.bin camp.onemodel
```

The checkpoint must already have been verified against its publisher pin. Conversion does not
execute pickle objects or load Python. The graph and weights stay outside the repository.
No corpus-quality or default-selection claim follows from successful conversion alone.

## Verification slice

`tools/scripts/generate-speaker-trials.sh TEST_CLEAN TEST_OTHER trials.tsv 1729` constructs ID-only
trials from separately fetched LibriSpeech test-clean/test-other data. Sort IDs ordinally, group by
speaker, and retain the first 20 utterances per speaker. A seeded .NET 10 PRNG selects speakers
uniformly and utterances uniformly within speakers; reject identical utterances and duplicate unordered
pairs. Same-speaker and different-speaker classes contain 10,000 unique pairs each. Trials are finally
sorted by their two IDs. No audio or embeddings are included in the trial file. This is an open-data
verification slice, not VoxCeleb1-O and not an estimate of a deployment prior.

`speaker-score trials.tsv OUR_EMBEDDING_DIR REFERENCE_EMBEDDING_DIR result.json` applies the same
cosine scorer to both embeddings, grouping equal scores at ROC thresholds. EER linearly interpolates
at the false-accept/miss crossing. Minimum DCF uses target prior 0.01, miss and false-alarm costs 1,
and normalization by the always-reject cost (0.01); it includes the always-reject operating point.
The paired bootstrap uses 1,000 resamples, seed 1729, independently sampling trials with replacement
within each same/different class while keeping our/reference scores paired. Percentile endpoints use
linear quantiles at 0.025 and 0.975. Confidence intervals describe the score difference conditional on
the fixed utterance/speaker set, not uncertainty over independent unseen speakers; trials sharing
speakers are not independent samples of a speaker population. No normalization or score calibration
is applied to these evaluation rows.

The English VoxCeleb checkpoint at revision `032b8131a7ad812f87061955ca974c99060c5a03` was checked
against ModelScope 1.40.2 (torch/torchaudio 2.11.0 CPU, float32) on the 50 IDs in
`eval/manifests/librispeech-speaker-parity50.txt`. Feature maximum absolute error was 0.000597 and
mean absolute error 0.00000624; the documented float32 feature tolerance is max 0.001 on these
probes. Embedding minimum/mean cosine were 0.999999999957/0.999999999985, above the 0.999 target.
This is numeric parity on a fixed probe set, not a speaker-verification quality claim. The converted
English graph is 29,414,032 bytes, SHA-256
`f333a00d5d66b9bf3d7e7abf35f36ff4a01e6e8a0df235b89a125e9fbe3cc15c`.

On the committed 20,000-trial LibriSpeech verification slice, both this graph and the pinned
ModelScope reference scored **0.75% EER** and **0.0379 normalized minDCF**. The paired bootstrap
95% intervals for our-minus-reference differences were [0,0] percentage points and [0,0], respectively:
the relevant ROC operating points were identical. These are conditional trial-resampling intervals,
not confidence intervals over a new population of speakers. The same feature extractor was also
checked on deterministic silence, tone, impulse, DC-offset, clipped, seeded-noise, exact-window and
two-window probes; maximum absolute error across them was 0.000851 (within the 0.001 tolerance).

These CPU/float32 checks used Linux x64, an AMD Ryzen 9 7950X3D (32 logical CPUs, AVX2 and AVX-512
available), eight inference threads, .NET SDK 10.0.403/runtime 10.0.14, OneModel 1.0.0 and
OneAudio 1.7.2. No GPU was used. Oracle dependencies and wheel hashes are pinned in the speaker
oracle's `uv.lock`; model and identifier pins are given above. Conversion is deterministic: its
recorded graph SHA-256 was reproduced from the same checkpoint independently.

## ECAPA-TDNN

`EcapaTdnnEmbedder` accepts the converted VoxCeleb checkpoint and truthful `ModelInfo`, returning
192 raw embedding values at mono 16 kHz. Calls are serialized; inputs must be 40 ms through 60 s.
The minimum ensures reflected temporal convolution pads are smaller than the frame dimension.
The graph follows Desplanques et al. (2020), Sections 3.1–3.3 and the pinned checkpoint config:
1024-channel TDNN, three scale-8 SE-Res2Blocks (dilations 2/3/4), multi-layer aggregation,
channel-dependent attentive statistics pooling and the final 192-dimensional projection. Convolution
uses reflected padding; TDNN activation is Conv → ReLU → BN. The first Res2 split bypasses its
convolution; subsequent splits add the preceding processed split. Global and weighted pooling
standard deviations use population statistics with a 1e-12 variance floor. These details were
confirmed against numeric black-box outputs, not implementation source.

`EcapaFbank` is a distinct public feature profile, not the CAM++ filterbank: centered zero-padded
400-sample periodic Hamming windows, 160-sample hops, 400-point power FFT, no preemphasis/DC removal
or amplitude scaling. There are 80 symmetric Hz-domain triangles with centers evenly spaced in
HTK mel from 0 through 8 kHz; each triangle's half-width is its center's distance from the preceding
center. Energies are floored at 1e-10, converted by 10 log10, clipped at the utterance maximum minus
80 dB, and mean-normalized by feature dimension. `ComputeRaw` exposes the decibel energies before
mean subtraction. Numeric public feature probes settle this profile; the model paper's MFCC input
description does not replace its released checkpoint's Fbank configuration.

The 50 committed LibriSpeech probe IDs, using SpeechBrain 1.1.2 with torch/torchaudio 2.11.0 CPU,
produce max/mean normalized-feature absolute errors 0.001542/0.00002079 dB, and minimum/mean embedding
cosine 0.999999999983/0.999999999990. Feature tolerance is max 0.002 dB (about 0.000461 natural-log
energy units); embedding target is cosine >=0.999. The mixed-radix 400-point transform uses managed
16-point FFTs and 25-way combination, not a native frontend. Feature-only probes also cover
silence, tone, DC-offset, impulse, seeded noise and sub-window audio. The feature extractor accepts
1 sample through 60 s; the model's stricter 40 ms limit still applies.

```sh
dotnet run --project tools/OneSpeech.Convert -- speaker-convert-ecapa embedding_model.ckpt ecapa.onemodel
```

The converted graph is 83,294,800 bytes, SHA-256
`f107ead463783cab70914fd0470cc6b42e2ec74b7a717bb42c1360879076c669`. As with CAM++, its catalog
release URI is reserved/unpublished; the model store converts verified publisher files locally. ECAPA is available by name; CAM++ English is the speaker default.

On the same 20,000-trial LibriSpeech slice, ECAPA and its pinned reference both score **1.10% EER**
and **0.0548 normalized minDCF**, using the same raw cosine scorer, prior/costs and conditional
paired-trial bootstrap described above. Our-minus-reference intervals are [0,0] for both metrics.
The ECAPA oracle uses the publisher's exact `/resolve/<commit>` URL as its source and pretrained
path, because the released `from_hparams` interface does not accept a `revision` constructor keyword.
Auxiliary classifier/global-normalization files loaded by the oracle are recorded in notices but are
not part of the converted graph; embedding normalization is explicitly disabled on the reference call.

## CAM++ Chinese common checkpoint

The same converter/frontend and `CampPlusEmbedder` (dimension 192) support
`iic/speech_campplus_sv_zh-cn_16k-common` revision `013c18884dffb82846cf472dd06dd4a08d97e2cd`.
On the same 50 IDs, feature max/mean errors are 0.000597/0.00000624 and embedding minimum/mean
cosines are 0.999999999966/0.999999999984. This meets the numeric parity target.
The converted graph is 28,098,192 bytes, SHA-256
`4b2f0fef5ec8cb855b53c6b5abae5dd4cd72c215137c52d32f5c477e4a792f49`, registered under the same
unpublished-artifact convention. No default selection is implied.

**Information only:** this checkpoint is trained on Chinese speech, while these trials contain
English read speech. Both our graph and the reference score 2.87% EER and 0.1876 normalized minDCF
on the shared 20,000-trial English slice, with paired difference intervals [0,0]. The same prior,
costs and conditional trial bootstrap apply. This is an implementation comparison and not a
language-appropriate accuracy bar or a new-speaker population confidence interval.

## Convenience identification operating points

The file helpers use raw cosine without AS-norm or a fitted affine map. Threshold fitting uses
`eval/manifests/librispeech-speaker-trials.tsv` (SHA-256
`8a7e63167cd625bd0633a29bd5f517ceb37e9a32fccec062328a52fd44aa43ae`, seed 1729).
Sort its 73 distinct speaker IDs ordinally; the first floor(73/3) = 24 identities are development,
the remaining 49 are holdout. Retain a trial only when both identities belong to the same partition;
discard cross-partition trials. This is a partition of the existing LibriSpeech test-clean/test-other
trial manifest, not the official LibriSpeech dev split and not a new independent benchmark.
Neither holdout scores nor query examples select the threshold.

Development IDs: 1089, 1188, 121, 1221, 1284, 1320, 1580, 1688, 1995, 1998, 2033, 2094,
2300, 237, 2414, 260, 2609, 2830, 2961, 3005, 3080, 3331, 3528, 3538.
There are 3,296 positive and 1,036 negative development trials. Choose the observed score minimizing
normalized DCF = miss rate + 99 × false-accept rate (target prior .01, unit miss/false-accept costs).
Acceptance is inclusive; cost ties prefer the larger threshold. Also report interpolated EER and
the observed threshold nearest equal false-reject/false-accept rates (stricter on ties).
`OneSpeech.Convert speaker-calibrate <manifest> <embedding-directory>` reproduces the selection
from the existing parity-checked embeddings without changing any model weight.

| Model | Dev EER % | EER operating point | Dev minDCF | Identification threshold | Holdout miss % | Holdout false accept % | Holdout DCF |
|---|---:|---:|---:|---:|---:|---:|---:|
| CAM++ English | 0.6068 | 0.41554184019080986 | 0.01638 | **0.4929986257373565** | 3.1921 | 0.02227 | 0.05397 |
| CAM++ Chinese | 2.7002 | 0.5890692253248851 | 0.16292 | **0.7229075438038025** | 15.5131 | 0.06682 | 0.22128 |

Holdout contains 11,194 retained trials. These are conditional trial rates, not population
confidence intervals or an authentication guarantee. Chinese CAM++ is measured on English read
speech here; this does **not** establish a Chinese-speaking population operating point. Open-set
identification with many enrolled names and cluster-average embeddings has different error rates
from pair verification. Deployments must validate their own population, recording conditions,
consent and risk policy using the low-level registry's explicit threshold. The convenience
operating point is a conservative read-speech default, not a liveness check or anti-spoofing system.


<!-- https://recursiveai.org/onespeech/docs/whisper-features/ -->

# Whisper features

`OneSpeech.Audio.WhisperLogMelSpectrogram` consumes at most 480,000 samples of 16 kHz mono float PCM. Resampling and channel mixing belong to the audio input layer. Samples are not clipped. Nonfinite samples and signals whose power exceeds float32 range are rejected.

The default 80-bin variant covers tiny through large-v2. Large-v3 and large-v3-turbo use 128 bins, as specified by their published `preprocessor_config.json`. Do not infer the bin count from model size alone.

## Transform

- Periodic Hann, 400 samples (25 ms), hop 160 samples (10 ms).
- Zero-pad short signals to 480,000 samples **before** centered reflection padding of 200 samples at each side. Reflection excludes the boundary sample itself. An empty padded signal is silence.
- Unnormalized forward Fourier transform, squared complex magnitudes at 201 nonnegative frequency bins. Drop the final STFT frame.
- Slaney's piecewise mel scale: linear below 1 kHz with spacing 200/3 Hz per mel unit; logarithmic above it, with 27 units from 1 to 6.4 kHz. Equally spaced mel edges from 0 to 8 kHz. Triangular filters, area-normalized by 2/(right edge − left edge) in Hz.
- Floor mel power at 1e-10, take log10, floor again at the maximum log power over the entire window minus 8. Return `(log power + 4) / 4`.
- Output is a flat mel-major float array: element `(mel, frame)` is `mel * frames + frame`. Padded windows always have 3000 frames.

`padToWindow: false` supports feature probes with their natural length, not model inference: frame count is `floor(samples / 160)`. Input must exceed 200 samples so centered reflection is defined. Longer files must be windowed by the caller; this extractor neither silently truncates nor allocates in proportion to file length. Cancellation is checked at every frame.

The algorithm follows Radford et al., *Robust Speech Recognition via Large-Scale Weak Supervision* (2022), §2.2. Padding, normalization and edge behavior were confirmed with black-box calls to the feature APIs below, not implementation source.

## Numerical checks

The feature acceptance tolerance is maximum absolute error **3e-5**, mean absolute error **1e-6** against float32 CPU feature outputs. This is a feature tolerance, not OneModel's graph exact-mode contract. The managed mixed-radix Fourier transform uses double complex accumulation, then float32 powers and mel sums; reference FFT and matrix-reduction orders need not be bit-identical.

`tools/oracles/whisper-features/features.py` calls only the public `whisper.log_mel_spectrogram` API (openai-whisper 20250625) and `WhisperFeatureExtractor` (transformers 5.22.2). Its locked environment pins torch 2.15.0 CPU, numpy 2.5.4 and soundfile 0.14.0. `prepare` decodes four LibriSpeech test-clean clips to shared float arrays and generates deterministic synthetic inputs. `whisper` and `transformers` write reference features; `tools/OneSpeech.WhisperProbe` compares the managed extractor to both. Model weights are not required.

Example, from the repository root, with the corpus and scratch paths supplied by the caller:

```sh
export UV_PROJECT_ENVIRONMENT="${ORACLE_ENV:?external environment directory required}"
uv sync --project tools/oracles/whisper-features --locked
PY="$UV_PROJECT_ENVIRONMENT/bin/python"
$PY tools/oracles/whisper-features/features.py prepare "$OUTPUT" --librispeech "$TEST_CLEAN"
$PY tools/oracles/whisper-features/features.py whisper "$OUTPUT"
$PY tools/oracles/whisper-features/features.py transformers "$OUTPUT"
dotnet run --project tools/OneSpeech.WhisperProbe -- "$OUTPUT"
```

Use `UV_PROJECT_ENVIRONMENT` to put the environment outside the checkout. Inputs and feature outputs also stay outside the checkout. The preparation manifest records clip IDs and hashes. It chooses test-clean IDs 1089-134686-0000 through 1089-134686-0003 in sorted order, plus a 440 Hz tone, empty input, a single-sample impulse, exactly 400 samples, DC and alternating clipped samples. No evaluation audio is committed. Small tone-value fixtures in the tests record observed output of openai-whisper 20250625.

## Graph conversion and recognition

`tools/OneSpeech.Convert fetch <tiny|small|large-v3-turbo> <directory>` fetches the explicitly pinned upstream data and verifies the safetensors SHA-256. `whisper <directory>` verifies the weight digest again and writes independently constructed encoder and explicit-cache decoder graphs in OneModel format. The encoder also computes cross-attention keys and values once per window. Because OneModel 1.0.0 limits each model file to 2 GiB minus one byte, turbo's encoder is partitioned after its first 16 blocks into `encoder-part0.onemodel` and `encoder.onemodel`.

`WhisperRecognizer.Load(directory, options)` loads local graphs and data without network access. Its offline path uses timestamp tokens to advance through 30-second windows, bounded previous-text conditioning, per-window cancellation and optional audio-time progress. The default temperature schedule is 0/.2/.4/.6/.8/1, with compression ratio 2.4, average log probability -1 and no-speech probability .6. Temperature zero is greedy, not the CLI's beam-search configuration. Positive temperatures sample one candidate with an explicit seed. Sampling uses a local xorshift32/inverse-CDF stream; the same numeric seed is not expected to reproduce another runtime's random stream. Set `Temperatures = [0]` for deterministic greedy-only comparisons. Timestamp-only, whitespace-only and zero-duration slices are omitted from both the transcript and previous-text conditioning. Keeping their tokens would shift the bounded prompt for the next window, even when the visible text stayed unchanged. Native word timestamps are not advertised; optional cross-attention DTW supplies word timing provenance.

Reference inference and evaluation drivers under `tools/oracles/whisper-features` write per-utterance files for resumption rather than hiding incomplete runs. Fixed 200-utterance LibriSpeech slices agree within paired-bootstrap noise for small on test-clean/test-other and large-v3-turbo on test-clean. The tokenizer matches 412 crafted/transcript cases exactly on tiny, small and turbo. These checks do not establish full-corpus or long-form parity: disabling stochastic fallback on the Earnings-22 probe exposes different repeated-text trajectories, and the long-form agreement check remains unmet. Encoder fp32 accumulation differences are amplified by trained residual outliers and the final normalization; the graph path is not tensor-identical to the reference.

For a continuous file, the recognizer preserves the 200-sample STFT halo across window boundaries rather than reflecting each artificial boundary. It makes a bounded-memory feature pass to find the file-wide dynamic-range floor, then applies that floor to each decoding window. This matches the public full-file log-mel operation without retaining a full-file spectrogram. The standalone extractor's documented padding behavior remains unchanged.

DTW operates on the exact generated text-token IDs, concatenated across all segments in a window. It normalizes the selected heads across token rows, applies a seven-frame median filter, averages the heads and solves a monotone minimum-cost path at 20 ms resolution. Punctuation is attached to written words without extending their lexical timing interval across a following pause, as observed in reference output. Failed alignment leaves endpoints and confidence null. Word confidence is the mean teacher-forced token probability for the recognized written form, not a calibrated acoustic likelihood.

`WhisperOptions.BeamSize` optionally enables cache-aware beam decoding at temperature zero. Null preserves the public reference API's greedy default; positive-temperature fallback still samples one candidate. Generic `Decoding.BeamSearch` owns explicit independent decoder states, so beam branches can reuse a parent's KV cache without invalidating siblings. Beam scores use mean filtered natural log probability, deterministic score/ID ties, and an EOS-completion budget equal to the beam width. Width one follows greedy selection.


<!-- https://recursiveai.org/onespeech/docs/qwen3/ -->

# Qwen3-ASR and forced alignment

Qwen3-ASR-0.6B and Qwen3-ASR-1.7B run on OneModel, including their AuT audio
encoder/projector and Qwen language decoder with a past/present KV cache.
The published ungated checkpoints are Apache-2.0. Model revisions, publisher
file SHA-256s and independently reproduced runtime-file SHA-256s are pinned
in the catalog. Weights are not embedded in the core package.

```csharp
var document = Speech.Transcribe("speech.flac", model: "Qwen/Qwen3-ASR-0.6B");
```

This file path uses VAD-first chunks, language identification, clustering
speaker attribution and Qwen3-ForcedAligner for its eleven languages. The
Qwen entries do not change the default ASR model. Missing tier-1 publisher
files download on first use; verified tensors are converted locally to
OneModel's format. No Python, native neural runtime or executable model-repo
code is needed by the library.

## Low-level recognition

```csharp
using var recognizer = Qwen3Recognizer.Load(modelDirectory, new Qwen3Options
{
    Context = "Vocabulary: Recursive AI, log-mel, transcription.",
    Precision = Qwen3Precision.Binary32,
    Threads = 4,
}, cancellationToken);
Transcript result = recognizer.Transcribe(audio, "en", cancellationToken);
```

Input is 16 kHz mono float PCM. File decoding/downmixing/resampling belongs to
OneAudio and `AudioFileSource`. More than 30 seconds uses VAD-first chunks;
each chunk has fresh decoder state. Empty audio returns an empty document;
unpadded nonempty audio must exceed 200 samples. Nonfinite samples and inputs
whose power exceeds float32 range are rejected.

Recognition is greedy. `MaxNewTokens` defaults to 512 per chunk (1–4096);
reaching the limit throws rather than returning silently truncated text.
`Context` is the system-context prompt for biasing, at most 4096 tokenizer
IDs. Special tokens in user context are refused. A language tag such as
`en`, `zh`, `fr` or `yue` forces that language; null requests the model's own
`language ...<asr_text>` output. Unsupported tags are refused. Translation
and true live partials are not declared by this family.

Low-level recognition does not download a timing model. Its word timings
remain null unless a timing stage supplies them. File transcription uses the
shared timing dispatcher, selecting Qwen3-ForcedAligner before CTC/DTW for its
eleven supported languages. Models are loaded only when needed, not attached
eagerly to the recognizer.

## Forced alignment

```csharp
using var aligner = Qwen3ForcedAligner.Load(alignmentDirectory, threads: 4, cancellationToken: cancellationToken);
Transcript aligned = aligner.Align(audio, text, "en", cancellationToken);
```

Supported tags are `zh`, `en`, `yue`, `fr`, `de`, `it`, `ja`, `ko`, `pt`, `ru`
and `es`. Language is required. Audio must exceed 200 samples and not exceed
300 seconds of 16 kHz mono PCM; text is limited to 1 MiB UTF-8 and 4096 units.
The released config defines 5000 timestamp classes with an 80-ms class step.
This config, rather than the report's illustrative class count, defines the
loaded head. The model predicts start/end slots in one forward pass, not by
autoregressive generation.

Whitespace delimits written units outside Chinese/Cantonese/Japanese, where
Unicode characters delimit units. Letters, numbers and ASCII apostrophes
form the spoken input; other punctuation is removed without inserting spaces.
Written whitespace-delimited forms are retained in words, and the document's
text is preserved. Combining marks removed by this cleaning do not reappear
through the tokenizer's later NFC normalization. These English unit rules
are checked against public toolkit output, including apostrophes and hyphens.
Character units do not claim the reference toolkit's language-specific
morphological segmentation.

Valid predictions carry `TimingMethod.ForcedAligner`. Predictions with reversed
bounds or an end beyond the audio duration retain null times. Confidence is
the mean of the two slot-class probabilities, not a calibrated probability
that the written word was spoken. The transcript overload instead preserves
existing word confidence and other non-timing fields, projecting character slots
back to the caller's written-word boundaries. No boundaries are interpolated or invented.
Independent slot predictions are not repaired for monotonicity across words;
valid intervals may overlap. These direct model predictions do not promise
identity with the toolkit's postprocessed boundaries.

## Front end and graph numerics

The pinned preprocessors request 128-bin Whisper features: 16 kHz mono,
400-sample periodic Hann windows, 160-sample hops, centered reflection, Slaney
mel filters, power floor 1e-10, log10 peak-minus-eight clipping, then
`(x + 4) / 4`. Mel-major single-input output has `sample_count / 160` frames,
dropping the final STFT frame. Maximum absolute feature tolerance is 3e-5
against the published float32 feature extractor. The public Whisper extractor
retains its 30-second bound; the shared internal alignment path allows the
aligner's bounded 300-second input.

Convolution chunks have 100 mel frames, padded independently, and yield up to
13 audio tokens each. Sinusoidal positions reset at each convolution chunk.
Learned activation arithmetic, including positional addition and audio-token
embedding substitution, is performed in OneModel graphs. The released CPU
toolkit emits attention-window metadata, but changing frames beyond an
eight-second boundary changes earlier encoder outputs. The CPU graphs
therefore use full-input encoder attention within each bounded input. This
is not a claim about a FlashAttention backend's window policy.

First pull converts and verifies only the Binary32 runtime graphs. BFloat16 files
remain catalogued but are not required or included in the resolved FP32 file list.
Loading `Qwen3Recognizer` with `Precision = Qwen3Precision.BFloat16` verifies any
existing BF16 graphs; if any are missing, it verifies the pinned publisher files,
converts BF16 into a staging directory, checks every output size and SHA-256, then
installs them in the same cache directory. This load never downloads. Corruption
throws `ModelIntegrityException`; remove the named file and retry. Concurrent
verification or conversion of that directory throws `IOException`; retry after it completes.
FP32 remains the default and never converts or validates unrelated BF16 files.

`Binary32` expands the original bfloat16 tensors exactly and retains float32
activations. `BFloat16` stores published tensors as bfloat16 and rounds neural
operator boundaries to nearest-even bfloat16; OneModel accumulates in float32.
Attention score/softmax intermediates remain float32, with the attended output
rounded to bfloat16. This does not promise bit-identical fused-kernel output
across runtimes. Decoder files contain four layers each; embedding and output
heads are separate files so every artifact fits OneModel's file-size limit.

## Tokenizer

`Qwen3Tokenizer` loads `vocab.json`, `merges.txt` and `tokenizer_config.json`
without executing model-repository code. Each file is limited to 16 MiB;
vocabulary IDs and merge counts are below 200,000. Input text is limited to
1 MiB UTF-8; invalid UTF-16 is rejected. It adds no implicit task tokens.

Text undergoes Unicode NFC normalization before byte-level BPE. Splitting is
Unicode-scalar-aware, including astral letters, individual numeric scalars,
case-insensitive contractions and significant whitespace/newline grouping.
Merge rank ties go left-to-right. Added tokens are atomic. Encoding tokens
marked special by model data requires permission. Omission during decoding
matches the released toolkit's Transformers 4.57.6: the named special-token
list plus BOS/EOS/padding/unknown tokens, rather than every added-token flag.
Thus `<tts_text_eod>`, blank tokens, `<asr_text>` and the aligner's `<timestamp>`
remain when special tokens are omitted. Transformers 5.22.2 differs on blank
and `<tts_text_eod>` omission; it is not the decoding reference for this API.
Unknown IDs are rejected; incomplete UTF-8 decoding uses replacement characters.

## Reproduction

`tools/oracles/qwen3/` contains the locked tokenizer/feature observations.
`tools/oracles/qwen3-asr/` uses released `qwen-asr` 0.0.6 and its required
Transformers 4.57.6 through public APIs, with CPU-only Torch. Public tensor
hooks export parity observations, never implementation source. Fixed slices
are the first 200 sorted IDs of each LibriSpeech test split. Generated audio,
features, hypotheses and transcript-derived observations stay outside git;
the committed tokenizer fixture contains only synthetic strings.

`tools/OneSpeech.Convert` provides `qwen-fetch`, `qwen-convert`,
`qwen-convert-bf16`, `qwen-reproduce`, probe and fixed-slice commands. Fetching,
weight expansion and graph construction are C#. The runtime uses the same
converter and checks converted sizes/digests before installing any graphs.

## Offline Qwen memory lifetime

Qwen file transcription recognizes all bounded VAD chunks first, recording only
chunk boundaries and written-word ranges with the transcript. It releases the
recognizer graphs before loading timing graphs, then replays decoded audio
through a fresh VAD with the same settings. Replay boundaries must match exactly;
a mismatch is an error, never a shifted word assignment. Timing is projected
back onto the original words and segments on the same audio clock. This adds a
bounded decoding/VAD pass, not a second recognition pass. Encoded-stream input
replays its existing temporary spool. No full-file PCM or KV history is retained.
Live and file-segment streaming retain their one-pass session policy; they
cannot defer finals until the end of the file. Other whole-file recognizers
without native timing use the same lifetime separation. Decoder shape caches retain
one plan: an older KV length is not reused by autoregressive generation.


<!-- https://recursiveai.org/onespeech/docs/moonshine/ -->

# Moonshine

The registered English checkpoints are `moonshine-ai/moonshine-tiny`,
`moonshine-base`, `moonshine-streaming-tiny`, `moonshine-streaming-small` and
`moonshine-streaming-medium` (all IDs use the `moonshine-ai/` publisher prefix).
They are pinned, ungated MIT checkpoints. Other checkpoints, including the
legacy Community License models, are refused. The catalog records upstream
and converted file hashes; local conversion uses only verified publisher data.
Models load as CPU fp32 OneModel graphs. No Python or native inference runtime
is required by the library.

```csharp
await foreach (Segment segment in Speech.TranscribeStream(
    "lecture.flac", model: "moonshine-ai/moonshine-base"))
    Console.WriteLine(segment.Text);

// pcm is an application-owned IAsyncEnumerable<AudioBuffer>, mono 16000 Hz.
await foreach (LiveEvent update in Speech.Listen(
    pcm, model: "moonshine-ai/moonshine-streaming-tiny"))
    if (update is PartialEvent partial) Console.Write(partial.Segment.Text);
```

`Speech.Listen` defaults to streaming-tiny. Offline tiny/base use VAD-endpointed
utterance mode and emit finals only. The file-transcription default is unchanged;
select a Moonshine model explicitly for file recognition. The file-streaming
API awaits model I/O and yields segments asynchronously; decoding/inference use
the synchronous CPU APIs, without background-task wrappers.

## Recognition contract

`MoonshineRecognizer.Load(directory, options)` opens verified converted graphs,
`tokenizer.json`, `config.json` and three-line `upstream.txt` provenance. Loading
checks file hashes against the catalog. The low-level recognizer accepts finite
mono 16 kHz PCM and utterances no longer than 60 s; use the bounded VAD pipeline
for longer recordings. Empty audio returns an empty transcript. Nonempty inputs
shorter than the frontend minimum (895 samples offline, 80 streaming) are
refused. Other rates/channels and nonfinite samples are refused with
`UnsupportedAudioException`.

Decoding is fp32 greedy, starting with BOS ID 1 and stopping at EOS ID 2 or the
new-token budget. Defaults are one inference thread and 6.5 new tokens per
second of real input, excluding BOS. A null language hint selects the fixed
English task; it does not run language identification. Other languages,
translation, contextual biasing and language detection are not provided.

Words preserve unknown timing and confidence as null, with `TimingMethod.None`.
VAD segment bounds are not word boundaries. These checkpoints do not produce
native word timestamps. Speaker assignment is a separate operation.

## Frontends and numerics

The offline architecture follows Jeffries et al., *Moonshine*, arXiv
2410.15608v2, §3.1: 16 kHz raw waveform, valid convolution kernels 127/7/3 with
strides 64/3/2, tanh and group normalization after the first convolution, then
GELU after each remaining convolution. Encoder attention and decoder
self-attention use rotary embeddings. Decoder feed-forward blocks use a
value half followed by a SiLU-gated half.

The streaming architecture follows the publisher's *Moonshine Streaming*
paper, §§3.1–3.4: 80-sample frames, per-frame population CMVN with variance
regularizer 1e-6, learned asinh compression, linear/SiLU, two causal
kernel-5/stride-2 convolutions. Only the first convolution has a following SiLU.
For an incomplete 80-sample frame, the released feature extractor pads the
waveform and the public model masks that entire frame before convolution.
We retain the padded frame count and zero the incomplete frame, as confirmed
by frontend and first-logit comparisons on 321 samples; silently reducing
the encoder frame count would give different output.
The encoder is position-free; learned positions and an optional projection
adapt its output to the decoder.

Details checked against released Transformers 5.22.2 public outputs include
adjacent-pair rotary embeddings, the decoder gate ordering, LayerNorm's
`1 + gamma` encoder scale and epsilon 1e-5, and local-window endpoints. For
window `(16,4)`, keys satisfy `query - 15 <= key <= query + 3`; `(16,0)` includes
the current key with no future keys. First/last two layers use lookahead;
intermediate layers are causal.

The fp32 comparison criteria are maximum scaled error
`abs(actual - reference) / max(1, abs(reference)) <= 1e-4` for the offline
frontend, final encoder output and decoder logits, and maximum absolute error
1e-5 for streaming frontend features. These are fast-arithmetic criteria, not
bitwise tensor equality. Tokenizer IDs and decoded strings must match exactly.

## Live state

Streaming checkpoints emit replacing partials no more often than once per
200 ms of accepted audio, after minimum speech is established. Finalization
uses the shared VAD endpoint policy in [Live sessions](https://recursiveai.org/onespeech/docs/live-sessions/).

Encoder prefixes whose right context is complete are retained. Only a bounded
suffix is re-encoded: its left context covers every layer's 15 past frames and
three frontend frames, and its provisional suffix covers 12 lookahead frames.
This does not repeatedly encode the growing whole utterance. A new endpoint
resets session-local caches. Each new audio hypothesis starts at BOS, because changed encoder memory can
change earlier decoder states. Within that hypothesis, the decoder evaluates
one token per call and retains each layer's self-attention keys and values.
Cross-attention keys/values are projected once per audio hypothesis, not once
per token. Cache planes are packed into one tensor to keep graph bindings small.

Live sessions preallocate and warm their working state before accepting PCM.
The encoder uses a fixed receptive-field window; decoder cross-attention uses
masked frame buckets of 128/256/512/1024/1536 frames. Self-attention cache capacity
is a power of two covering the configured token budget over the shared 30-second
VAD horizon. Unused keys are masked; padded encoder queries remain finite and
cannot affect real frames. These bounds do not change offline recognition's
60-second limit. Session creation includes the warm-up work.

Immutable emitted events, their text and word payloads belong to the caller and
are never pooled or mutated. They are measured separately from working-state
allocation. At the validated 100 ms push size, an emitting push also needs no
new working buffers. Backend runtime allocations are reported separately; zero
working-state allocation is not a claim of zero total process allocation while
the backend still allocates.

References: [offline paper](https://arxiv.org/abs/2410.15608),
[streaming paper](https://download.moonshine.ai/docs/moonshine_streaming_paper.pdf).
The exact checkpoint revisions and model-card URLs are in the catalog.


<!-- https://recursiveai.org/onespeech/docs/nemo-and-transducers/ -->

# NeMo features and transducer decoding

## Features

`NemoLogMelSpectrogram` consumes mono 16 kHz float PCM. It implements the
Parakeet v3 preprocessor parameters: 128 Slaney-area-normalized mel bands,
400-sample symmetric Hann window centered in a 512-point FFT, 160-sample hop,
preemphasis 0.97, power spectrum and natural log with additive `2^-24` guard.
80 mel bands are also supported. Dithering and augmentation are disabled.
Input is limited to 320–960,000 samples; callers must resample, downmix and
chunk outside this extractor.

Centered STFT uses **zero** padding. This edge behavior was confirmed against
NeMo Toolkit 3.0.0's public preprocessor API, not inferred from other STFT
implementations. The valid frame count is `floor(sample_count / 160)`. The
physical time axis has one extra centered frame, masked to zero; optional
`padTo` rounds that physical axis up to a multiple. The default zero disables
rounding, matching the Parakeet v3 model configuration. Padding follows
normalization, never participates in statistics, and is zero.

Per-feature normalization uses sample variance (denominator `valid_frames-1`)
and adds `1e-5` to the standard deviation. Float32 reduction roundoff can
produce a small nonzero normalized value even on digital silence. The
implementation retains float32 mean arithmetic rather than forcing silent
channels to zero. Extraction checks cancellation per STFT frame and channel;
instances are immutable and may be shared between threads.

Correctness probes use the first five lexicographically sorted LibriSpeech
test-clean clips. Against NeMo Toolkit 3.0.0, Torch 2.11.0 CPU, with dithering
zero, the maximum normalized absolute error is below `1e-4`; raw log-energy
error is below `1.1e-4`. Synthetic silence and 400/512-sample edges also meet
`1e-4` normalized error. Constant DC, whose variance is nearly zero in some
channels, uses `1e-3` normalized tolerance (`1.1e-4` before normalization).
These are float32 tolerances, not an exact-mode or whole-corpus parity claim.
`tools/oracles/nemo` records the released oracle pins and creates external
fixtures; `OneSpeech.NemoProbe` checks their dimensions and tolerances.

## Decoding

- `CtcGreedy` picks the smallest token ID on ties, merges consecutive equal
  IDs, then removes blanks. A blank separates repeated tokens. Token intervals
  cover the complete winning frame run; confidence is the mean conditional
  token probability over that run.
- `CtcPrefixBeamSearch` tracks distinct blank/nonblank probability sums for
  each prefix and merges paths before pruning. Scores are natural log
  probabilities without length normalization, language models or phrase bias.
  Ties use lexicographic token order. The result has no inferred timing.
- `RnntGreedy` follows Xu et al. 2023 Algorithm 1: nonblank updates the predictor
  prefix without advancing acoustic time; blank advances one frame. Repeated
  nonblanks are retained. Its reported token interval is one encoder frame,
  not a measured phonetic duration.
- `TdtGreedy` follows Algorithm 2: token and duration heads have independent
  argmax selections. Nonblank updates the prefix, and the selected duration
  advances acoustic time. Zero-duration tokens retain zero-length intervals;
  no minimum word duration is invented. A blank with zero duration advances
  one frame defensively, preventing an infinite loop. Positive duration
  intervals are clipped to the encoder-frame count.

Transducer callbacks receive the complete accepted nonblank prefix and must
fill both output spans; they may cache predictor state by prefix length.
NaN, positive infinity, all-masked rows and incomplete callback outputs are
errors. Symbol/token bounds throw instead of silently losing tokens.
All decoding is synchronous, deterministic and cancellation-aware.

`NativeWordTiming.Map` takes explicit tokenizer-produced word/token ranges.
It unions only those token intervals, averages token confidences, applies the
encoder-frame duration and chunk offset, and marks timed words `Native`.
Zero-token words remain untimed. It does not split text heuristically, fill
blank gaps, estimate missing boundaries, or assign speakers. At 8x encoder
subsampling, one 10 ms mel hop corresponds to 80 ms of audio time.

## Graph construction

The internal FastConformer graph builder emits OneModel-format graphs from an
abstract float tensor source. Its canonical tensor names are a **converter
schema**, not publisher tensor names. A checkpoint-specific mapping and
numerical parity are required before connecting it to a released model.
`ParakeetTensorSource` maps the pinned publisher header into this schema.

The encoder supports 8x depthwise-separable subsampling, relative-position
multihead attention, the Conformer convolution module and two half-step
feed-forwards. Single-batch graphs have no padding within the valid time axis.
They can be split at block boundaries, with each file required to stay below
2 GiB. Weights are copied at save time rather than materialized together.
Predictor graphs use an explicit packed input/output layout:
`[embedding_or_predictor_vector, h0, c0, h1, c1, ...]`. Each LSTM layer uses
input/forget/candidate/output gate order. Joint graphs accept concatenated
encoder and predictor vectors and return token logits (including blank),
followed by duration logits; zero duration columns selects the RNN-T shape.

Small deterministic random-weight tests exercise encoder, split-vs-unsplit,
predictor and joint graph loading/running on OneModel. Publisher-weight encoder
probes use the same five oracle feature arrays as the feature probes. Their
fp32 maximum absolute tolerance is 2e-5 (fast arithmetic, not bitwise exact
mode). Subsampling kernels operate on time then frequency. The masked final
STFT frame is cropped before subsampling; retaining it changes edge outputs
on odd frame lengths, as observed through NeMo's public encoder API.

## Parakeet-TDT-v3 recognition

Select this model explicitly:

```csharp
var transcript = Speech.Transcribe("recording.flac", model: "nvidia/parakeet-tdt-0.6b-v3");
```

The checkpoint is CC-BY-4.0, ungated **tier 2**: never a default, a package or a
mirror asset. First use fetches the pinned publisher safetensors and tokenizer,
verifies SHA-256, then converts locally into the per-user cache. The two
12-layer encoder graphs are each below 2 GiB, so OneModel 1.0.0 can load them
by path. Predictor, joint and embedding artifacts are local outputs too.
Their sizes and digests are catalog metadata, not download routes. Nothing
converted is redistributed. `ModelInfo` retains NVIDIA attribution, licence,
publisher revision, upstream digests and local output digests; format conversion
is marked and weights are unchanged.

`ParakeetRecognizer.Load` is the lower-level CPU entry point for a verified
`ResolvedModel`. It accepts mono 16 kHz PCM, uses greedy independent token and
duration argmax with durations [0,1,2,3,4], blank ID 8192, and a 10-symbol bound
per encoder frame. The initial predictor input is zero with zero LSTM state;
accepted tokens update the predictor exactly once. Its state is local to each
chunk. There is no beam, contextual biasing or translation in this adapter.
The output vocabulary uses the publisher's Metaspace decoder: leading ▁ starts
a word and punctuation remains attached to its preceding token range. Native
80 ms token-duration intervals are unioned into word intervals; zero durations
remain zero and final intervals are clipped to the input audio horizon.
Punctuation-only emissions retain their text and acoustic-clock advance but
contribute a zero-duration token timing interval. Word endpoints and confidence
stop at the last lexical emission, excluding trailing punctuation even when
it is emitted after a blank gap. This matches NeMo's observed timestamp output. The tokenizer's unknown surface is spaced U+2047, not the literal
`<unk>` vocabulary spelling; this too is checked through the public oracle.

Input above 30 seconds and all file helpers use VAD-first bounded chunks. Shorter
than 20 ms produces no words. Immutable final segments share one audio clock.
A supplied supported language labels the document but does not condition this
automatic multilingual checkpoint. Without it the language field remains null:
there is no independent language-ID output, and `DetectLanguage` refuses rather
than guessing from text. All other unsupported requests are refused through
the capability layer. Instances are not concurrent; cancellation is checked at
front-end frames, graph boundaries and decoding steps.


<!-- https://recursiveai.org/onespeech/docs/cache-aware-conformer/ -->

# Cache-aware FastConformer recognition

Nemotron 3.5 ASR loads only by explicit model name,
`nvidia/nemotron-3.5-asr-streaming-0.6b`. It is an upstream-only OpenMDW-1.1
model, never a default or a redistributed weight package. `ModelStore` verifies
the pinned publisher safetensors, configuration and tokenizer files, converts
locally, and verifies every converted output. `ModelInfo` retains the full
agreement, publisher origin and all file digests.

```csharp
var resolved = await ModelStore.Default.PullAsync(
    "nvidia/nemotron-3.5-asr-streaming-0.6b");
using var recognizer = NemotronAsrRecognizer.Load(resolved,
    new NemotronAsrOptions { ChunkMilliseconds = 80, Threads = 1 });
using var vad = new SileroVad();
using var live = recognizer.CreateLiveSession(vad, language: "en");
```

Types are in `OneSpeech.Catalog`, `OneSpeech.FastConformer` and `OneSpeech.Vad`.
`Speech.Listen` and `Speech.TranscribeStream` accept the same explicit model ID.
Their default attention chunk is 320 ms, matching the publisher configuration.
The low-level options also accept 80, 560 and 1120 ms. The released reference
processor rejects 160 ms; the library refuses that option rather than silently
changing the reference's supported settings. Language tags cover the publisher's
transcription-ready and broad-coverage locales and their base-language aliases;
adaptation-only locales are not advertised. Null language selects the automatic
prompt and preserves its emitted language tag. Automatic live prompting resets
for every utterance. Translation and phrase biasing are not supported.

## Features, padding and cache state

`NemoMelStream` emits raw, unnormalized 128-bin mel frames with the same arithmetic
as `NemoLogMelSpectrogram.Extract(normalize: false)`. The fixed ring preserves
preemphasis across pushes. Centered windows use zero extension only at stream
edges; there are no whole-utterance statistics. Callbacks borrow one reused
frame. The valid offline frame count is `floor(samples/160)` at 16 kHz. Crop the
masked final STFT frame before subsampling.

The three stride-two subsampling stages pad frequency two positions on the left
and one on the right. Offline time padding is likewise two past positions and
one end position at each stage. Thus, with `f(L) = floor(L/2) + 1`, the encoded
length is `f(f(f(L)))`, not `ceil(L/8)`. An exact-size offline tail graph applies
padding at every stage; padding mel input alone would wrongly evaluate biases
beyond the real tail.

Cache execution follows Noroozi et al., *Stateful Conformer with Cache-based
Inference*, §3.3. Each attention layer carries projected keys/values for 56 past
frames; each causal depthwise convolution carries its previous eight GLU
activations. Subsampling history preserves the stride phase. State is ordinary
caller-owned tensor input/output data. The initial and steady shapes are warmed
and reused, and state resets between utterances.

The documented streaming reference requires full steady chunks, including EOF.
Live recognition therefore zero-extends the final PCM context to the same chunk
geometry. This never advances the caller's real audio clock. Offline recognition
instead uses exact valid lengths. The emitted words use the native RNN-T 80 ms
frame intervals, with softmax token confidence averaged over a word. A partially
covered final frame is clipped to the real horizon; words wholly beyond it remain
untimed (`TimingMethod.None`), not assigned invented boundary times.

## Live events and bounds

The predictor/joint network accepts at most ten nonblank symbols per encoder
frame and keeps its recurrent state across chunks. RNN-T repeats are preserved;
only declared special/language tokens are stripped from text. A preferred partial
cadence follows the selected attention chunk. The shared controller retains its
200 ms default for other families. The actual delivery clock also depends on the
caller's PCM push size and VAD endpoint policy.

The VAD controller bounds utterances at 30 seconds by default, emits revisioned
immutable partial snapshots and one immutable final before each endpoint, and
uses one integer-sample audio clock. File streaming uses the same live controller.
In-memory offline input above 30 seconds uses VAD-first chunks. Models are
borrowed by live sessions and must outlive them; recognizer/session instances are
not thread-safe. Cancellation is checked at feature, graph and decoding steps.

## Small-probe evidence

On five fixed clips, final encoder max absolute error was below `3e-6` versus
Transformers 5.22.2 and below `2e-6` versus NeMo 3.0.0 using the same raw features
and valid lengths (check tolerance `1e-5`, float32 fast arithmetic, not exact-mode
equivalence). Prompt/predictor/joint decoding matched all 207 nonblank reference
IDs, including special tokens. End-to-end decoding on those clips was
lexical-token-identical at 80, 320 and 560 ms. These probes are not full-corpus
quality claims. The sanitized reference harnesses share the locked Transformers
5.22.2 CPU environment under `tools/oracles/moonshine`.

## Streaming measurement method

The streaming comparison uses fp32 greedy RNN-T with at most ten symbols per
frame, the English prompt, and identical publisher weights. Each input push
contains the selected attention chunk's duration. Native rows exercise the live
controller's actual partial publication on known utterances with speech activity
assumed present; the reference uses its documented streaming generator and
streamer. Both complete at the real input horizon. This isolates recognition
from endpoint detection, which the reference API does not provide.

Word finalisation latency here means stable-word publication: a word closes when
the next word begins, or at EOF for the last word. Match words with the same
English scoring normalizer and report matched/unmatched counts. Measure both
implementations against the same offline native word ends; additionally report
available MFA word ends as a separately named clock. Signed negative differences
are retained because native endpoints have 80 ms resolution. EOF Final delay is
reported separately; this method is not a claim about a vendor endpoint policy
or wall-clock network latency.

`nemotron-asr-metrics <converted> <clip> <output> <chunk-ms> [threads]` measures
three warmed live repetitions; threads defaults to one. RTF includes `Push` and
`Complete` processing, not model load or session startup (reported separately).
Allocation accounting separates immutable emitted payloads and VAD-runtime
allocations from working storage. Non-emitting pushes allocate no working bytes,
but partial snapshot preparation and EOF currently do: an overall zero-working-
allocation claim would be incorrect. Indicative shared-load measurements do not
substitute for isolated release benchmarks.


<!-- https://recursiveai.org/onespeech/docs/custom-models/ -->

# Custom speech graphs

`OneSpeech.Custom.GraphSpeechRecognizer` is a low-level `SpeechRecognizer` for
caller-supplied float32 ONNX or OneModel files. It never downloads or converts
weights. The caller supplies lawful provenance in `ModelInfo` and an immutable
`TokenVocabulary`, including a blank ID. Built-in family loaders remain the
simpler choice for supported checkpoints.

## Encoder contract

Each `EncoderGraph` has exactly one float32 input and output, batch one. Inputs
are `[1, channels, frames]`. Intermediate outputs and transducer encoder outputs
are channel-major. The final CTC output is `[1, frames, vocabularySize]`, raw
logits, including blank. Stages can split large encoders at layer boundaries.
Declared batch, ranks and channel dimensions are checked at load; output sizes
and temporal stride are checked at execution. Violations throw
`InvalidModelShapeException`. File/format errors remain OneModel's load errors.

Front ends accept finite mono 16000 Hz PCM:

- `RawWaveform`: unchanged `[1,1,samples]`; no normalization is inferred.
- `Whisper80` / `Whisper128`: unpadded log-mel, `samples / 160` frames.
- `Nemo128`: NeMo normalized mel, with the masked trailing STFT frame cropped.

For each stage, frames become `floor((frames - receptiveField) / stride) + 1`,
or `ceil(frames / stride)` when `SamePadding` is true. The caller specifies the
final `frameDuration` in audio time. For omniASR, use raw waveform, receptive
field 400, stride 320, 9812 output channels, blank 0 and 20 ms. For the split
Parakeet encoder, use NeMo128, 1024 channels at both stages, first-stage stride 8
with same padding, then stride 1, blank 8192 and 80 ms.

Calls below 400 samples produce no words. Calls over 30 seconds use Silero
VAD-first chunks; `TranscribeStream` yields immutable final segments from those
chunks. Language hints are output labels, not acoustic conditioning or language
identification. No translation or phrase-biasing head is inferred.

## Search and vocabulary

CTC uses greedy collapse by default; `beamSize` 1–64 enables prefix beam for
vocabularies up to 4096 IDs. Beam tokens are timed with our CTC Viterbi trellis;
greedy tokens retain their occupied frame intervals. A failed beam alignment
throws rather than inventing word timings. Confidence is the mean conditional
token probability over its occupied frames, not a calibrated word posterior.

Vocabulary pieces concatenate without normalization. Optional metaspace decoding
replaces U+2581 with space and removes one leading space. The caller supplies
special-token surfaces (including an empty blank surface), and any unknown-token
surface; byte-fallback decoding is not inferred. Pieces must not encode multiple
whitespace-separated words in a single token: disjoint token ranges are required
for word timing. Optional trailing-punctuation exclusion matches native
transducer timing without changing written text. All words carry `Native`
timing provenance, null confidence/times for a zero-token range, and initially
`Unassigned` speaker. Times are clipped to the input horizon.

## Packed transducer contract

`TransducerGraphs` declares separate predictor and joint files. The predictor
maps `[1,stateSize]` to the same shape. Its zero state is run once to prime the
blank input. On each accepted token, the first `embeddingSize` state values are
replaced by that token's row in the caller-supplied embedding table, then run
again. The first `embeddingSize` output values feed the joint after the current
encoder frame. Remaining state values carry recurrent state.

The joint maps `[1,encoderChannels + embeddingSize]` to
`[1,vocabularySize + durationCount]`. Empty durations select RNN-T greedy;
strictly increasing durations beginning at zero select TDT greedy. Token logits
precede duration logits. Default packed widths (640/3200), duration table
`[0,1,2,3,4]`, and 8193 embedding rows describe the converted Parakeet graph.
Other predictor/state layouts need an adapted graph, not an implicit shim.
All calls are synchronous, cancellation-aware at graph/search steps, and not
thread-safe; dispose the recognizer to release its owned sessions.

Language hints are never an implicit capability: declare the supported language labels with the `languages` constructor argument on `GraphSpeechRecognizer`, for example `languages: ["en", "fr"]`. The default is empty (no hints accepted). Hints normalize through the shared rule and label output; this adapter has no language-conditioning or detection head. An undeclared hint throws `CapabilityNotSupportedException` instead of being silently treated as supported.


<!-- https://recursiveai.org/onespeech/docs/int8/ -->

# Integer inference

The fp32 defaults are unchanged. The following explicit catalog IDs select locally
converted, mixed-int8 graphs:

- `openai/whisper-large-v3-turbo-int8`
- `openai/whisper-small-int8`
- `moonshine-ai/moonshine-streaming-tiny-int8`

For example:

```csharp
var transcript = Speech.Transcribe("recording.wav", language: "en",
    model: "openai/whisper-large-v3-turbo-int8");
```

The variants use the same pinned publisher weights, configuration and tokenizer
files as their fp32 counterparts. Conversion is deterministic and runs in the
local model cache. Each converted file has a pinned size and SHA-256 in the
catalog. The model ID in `ModelInfo` identifies the int8 variant; its source and
revision continue to identify the original publisher checkpoint. No public API
or transcript JSON schema changes are required.

## Numerical scheme

Whisper constant-weight linear projections, including vocabulary projection, use
signed int8 weights with one scale per output channel. Moonshine quantizes only
the decoder MLPs and vocabulary projection; its acoustic encoder, attention and
cross-attention projections remain fp32 to protect recognition quality. For each channel, the scale is
`max(abs(weight)) / 127`, or 1 for an all-zero channel. Values are divided by the
scale, rounded to nearest with ties to even, and clamped to [-127, 127]. The stored
weight is transposed to the matrix product's input/output layout. No development
or evaluation data is used to choose scales.

Activations at quantized linear products use ONNX
`DynamicQuantizeLinear`: the range includes zero, uses 255 intervals, and an
all-zero input has scale 1 and zero point 0.

`MatMulInteger` accumulates into int32. Its fp32 accumulator is multiplied by the
product of activation and weight scales before adding any fp32 bias. No collected
activation statistics are involved. The supported checkpoints' linear input
widths are below OneModel's integer accumulation bound.

Convolutions, embedding lookup, normalization, attention score/value products,
residuals and nonlinearities remain fp32. These are mixed-int8 models, not
all-integer networks. Quantization changes logits and can change decoded tokens;
fp32 token identity is not promised. Integer inference also does not guarantee a
speed improvement on every CPU or thread count.


<!-- https://recursiveai.org/onespeech/docs/model-downloads/ -->

# First-use models and offline preparation

`model` selects the requested role, not an isolated inference graph. Present files are verified and reused. Nothing is downloaded on `Transcript` export. Tier-2 models must be explicitly named and always come from their publisher; prefetch never accepts a gate for you. Silero VAD (`silero/vad-v6.2.3`, 2.3 MB) is embedded in core and never needs a runtime download.

| Verb | Selected model | Companions that can be acquired |
|---|---|---|
| `Transcribe` / `TranscribeAsync` | `openai/whisper-large-v3-turbo` by default, or the named recognizer | CAM++ English or Chinese based on the requested/detected language. Without native word timing: Qwen3-ForcedAligner for its covered languages, then omniASR-CTC when words remain untimed or Qwen cannot handle the chunk; existing DTW is last fallback. |
| `TranscribeStream` (path or encoded stream) | Same recognizer default | No diarization embedder. Qwen3-ASR uses Qwen3-ForcedAligner; other families use native/DTW word output. |
| `Listen` | `moonshine-ai/moonshine-streaming-tiny` by default | CAM++ English for non-Chinese hints, Chinese for `zh`; both when no hint is supplied. Qwen3-ASR additionally uses its forced aligner. |
| `Align` / `AlignAsync` | Ordered timing by default | Qwen3-ForcedAligner for a covered hint, then omniASR-CTC if needed; without a hint only omniASR-CTC. An explicit aligner ID loads only that model. No speaker models. |
| `DetectLanguage` / `DetectLanguageAsync` | Same recognizer default as `Transcribe` | No timing or speaker companion. |
| `Enroll`, `Identify`, `Verify` and their async forms | CAM++ English by default; Chinese for `zh`, or either explicit CAM++ ID | No other downloadable model. The process-wide registry retains the selected embedder until `ClearSpeakers`. |

The timing ID is `Qwen/Qwen3-ForcedAligner-0.6B` (1,835.5 MB publisher weights, approximately 3,671 MB converted fp32). CTC is `facebook/omniASR-CTC-300M` (1,302.1 MB publisher weights, 1,302.2 MB converted). Qwen supports `zh`, `en`, `yue`, `fr`, `de`, `it`, `ja`, `ko`, `pt`, `ru`, `es` and needs more than 200 input samples. Standalone alignment is limited to 60 s.

English CAM++ is `iic/speech_campplus_sv_en_voxceleb_16k` (29.4 MB publisher and 29.4 MB converted); Chinese is `iic/speech_campplus_sv_zh-cn_16k-common` (28.0 MB publisher and 28.1 MB converted). Whisper tiny publisher weights are 75.6 MB, turbo 1,617.8 MB. Moonshine streaming tiny publisher weights are 176.2 MB, approximately 315 MB converted. Sizes are decimal MB, rounded; publisher and runtime files both occupy cache space. Catalog file sizes and digests are authoritative.

```csharp
await Speech.PrefetchAsync(language: "en", model: "openai/whisper-tiny",
    cancellationToken: cancellationToken);
```

`PrefetchAsync` defaults to the transcription model and prepares a **superset** of the selected recognizer's file/live needs: the recognizer, applicable Qwen timing, CTC fallback, and the chosen CAM++ embedder(s). With no language it includes both embedders and Qwen timing for eventual language detection. This deliberately includes companions a particular silent or already-timed input might not use, so a subsequent call can stay offline. Pass the live default ID to prepare `Listen`. Pass an explicit aligner or embedder ID to prefetch only that role; for a default ordered alignment, prefetch its possible aligners individually. `ModelStore.PullAsync(id)` remains the precise single-model operation. All prefetch calls are cancellable and load no inference sessions.


<!-- https://recursiveai.org/onespeech/docs/models/ -->

# Model files and pre-fetch

`ModelCatalog.Default` reads a versioned embedded data catalog without network access. `ModelStore.PullAsync(id)` resolves an **explicit** model ID to a `ResolvedModel` containing an absolute directory and `ModelInfo`. The returned directory contains verified publisher files and registered local runtime conversions, including configuration and tokenizer data. Graph execution itself is separate from fetching.

```csharp
var store = new OneSpeech.Catalog.ModelStore(cacheDirectory: "model-cache");
var model = await store.PullAsync("openai/whisper-tiny", cancellationToken: cancellationToken);
// model.Directory: verified publisher files; model.Info: revision, digests and licence provenance.
```

The tool provides `onespeech models list` and `onespeech models pull <id>`. Ctrl+C cancels a pull. List never downloads. The current file default is Whisper large-v3-turbo; the live default is Moonshine streaming-tiny. `GetDefault(family)` refuses when that family has no declared default.

## Cache and trust

Cache location precedence is constructor argument, `ONESPEECH_MODEL_CACHE`, then:

- Windows: LocalApplicationData/OneSpeech/models.
- macOS: `~/Library/Caches/OneSpeech/models`.
- Other platforms: `$XDG_CACHE_HOME/onespeech/models`, falling back to `~/.cache/onespeech/models`.

Use a **trusted per-user directory**, not a shared writable directory. Hosts must not replace cache directories or files with symbolic links. Model subdirectories use a digest of the ID, revision and upstream filenames/digests, so different revisions cannot silently reuse old pins. A `.lock` file opened with exclusive OS sharing serializes pulls across stores and processes. It is intentionally retained after releasing the handle; deleting lock files can split locks between processes. Cancellation also interrupts lock waits. A sharing/I/O failure is retried at 50 ms intervals up to 12,000 times (about ten minutes), then reported as an I/O error.

Each file is downloaded to `.download` under that lock, size-bounded while copying, SHA-256 checked, closed, then atomically renamed to its final name. A cancelled or failed transfer removes its temporary file. An interrupted process can leave `.download`; the next pull restarts it rather than trusting it. Already-complete files survive interruption. There is no byte-range resume.

Present files never cause network requests. On first open **per store instance**, present files are rehashed. The verified marker lives only in memory, keyed by path, length and modification time; a new store/process always rehashes disk bytes. There is no writable disk marker that can authorize unverified weights. A corrupt cache file is removed and produces `ModelIntegrityException`; retry explicitly to fetch it again. The trusted-cache requirement excludes a hostile local process changing bytes while preserving their metadata or holding a file open. This is verification of publisher bytes, not a local filesystem sandbox.

`PullAsync` supports cancellation and `IProgress<ModelDownloadProgress>` with bytes received/expected per file, emitted only for downloads. Errors include `ModelNotFoundException`, `ModelIntegrityException`, `ModelTokenRequiredException`, `ModelNetworkException`, and `InvalidModelCatalogException`. Filesystem permission and storage failures remain ordinary I/O errors. Fetching is asynchronous; there is no synchronous wrapper around network I/O.

## Tiers and sources

Tier 1 is ungated MIT/Apache-2.0/BSD, eligible for defaults. Its transport selects a declared mirror if present, otherwise the publisher. A mirror must be an HTTPS asset beneath `https://github.com/recursive-ai-oss/onespeech-csharp/releases/download/<tag>/<asset>`. All embedded mirror URLs are currently null: **not yet mirrored**. Nothing silently guesses a release URL.

Tier 2 is never a default, has no mirror, and has only pinned publisher URLs. The catalog rejects tier-2 defaults, mirrors, and redistributed converted artifacts before any source is called. Registered tier-2 converters may produce cache-only local artifacts from already verified publisher inputs; those artifacts are never fetched, mirrored or redistributed. Tier 2 may use the declared permissive licences, CC-BY-4.0, or OpenMDW-1.1; unknown, noncommercial and research-only licences are refused. Gate status is independent of licence: any gated entry must be tier 2. Parakeet-TDT-v3 is available by explicit name. Nemotron 3 Diarization is a named tier-2 entry. Its pinned publisher safetensors are read as bounded data and converted under the per-model lock into the user's cache. Local graph/silence artifact digests are registered in code, not as catalog `convertedFiles` URLs. Outputs are hash-checked in a staging directory before installation, never mirrored or packaged; the loader also verifies them. `ModelInfo` retains the OpenMDW agreement and publisher origin.

A gated request requires an explicit `token` argument or `ONESPEECH_MODEL_TOKEN`, even if its bytes are cached or a custom source is used. Gate acceptance is the user's responsibility. Tokens are never logged or persisted. The HTTP source sends authorization only on the original upstream request, not to CDN redirect targets. It follows at most five HTTPS redirects, allowing publisher delivery through signed CDN URLs without adding those URLs to the catalog. Transport failures expose no signed URL or token. Requests time out after 30 minutes; transfers are restartable by calling again.

A host may replace HTTP entirely:

```csharp
var source = new OneSpeech.Catalog.DirectoryModelSource("publisher-files");
var store = new OneSpeech.Catalog.ModelStore(source: source);
```

Directory source layout is `root/<model-id>/<revision>/<filename>`; the root is trusted and must not contain symbolic links. A custom `ModelSource` supplies the pinned publisher bytes. The store still enforces catalog policy, gate authorization, file size and SHA-256. The token-aware virtual overload is additive: older host hooks implementing the tokenless overload still work. A source override does not authorize redistribution of tier-2 files.

## Catalog schema 1

Optional `defaultRoles` is an array of up to 16 unique lower-case ASCII role names
(letters and hyphens, at most 64 characters). It is empty when omitted. Each role
has at most one default across the catalog, independently of `isDefault` family
picks; only ungated tier-1 entries qualify. `GetRoleDefault("transcription")`
selects the file recognizer for `Speech.Transcribe` when no model is specified.
The built-in pick remains `openai/whisper-large-v3-turbo`. Changing that catalog
role assignment changes the pick without changing recognition code. A missing
role default throws `ModelNotFoundException`, never a hidden family fallback.

The envelope is `{ "schemaVersion": 1, "models": [...] }`. Model properties are the camel-case `CatalogEntry` fields: ID, family, source repository, 40-hex revision, files, convertedFiles, tier, licence, exact attribution, gated, isDefault and capabilities. `files` contains the publisher file set. `convertedFiles` is a separate, possibly empty array of local runtime outputs, with SHA-256 and size. Tier-1 output URLs describe OneSpeech release assets. Tier-2 output URLs must exactly match one of the pinned publisher input URLs: they identify provenance, not a downloadable output. `PullAsync` fetches **only publisher files** and produces registered missing converted files locally, then verifies their sizes and hashes before atomic installation. Silero uses publisher ONNX directly. Upstream and converted bytes must never be assigned the same name in one entry.

File metadata is name, size, sha256, upstreamUrl and optional mirrorUrl. Relative names use slash-separated ASCII letters/digits, hyphens, underscores and dots, with an alphanumeric first character per component; Windows device names are also refused; traversal, absolute paths and hidden components are refused. Names are unique ignoring case. Maximum file size is 64 GiB. URLs require HTTPS, no credentials or custom port, and upstream paths pinned to the declared source repository and revision. The publisher forms supported are `<repository>/resolve/<revision>/<file>` and GitHub's `raw.githubusercontent.com/<owner>/<repo>/<revision>/<file>`.

The reader bounds input **before** generated deserialization: 8 MiB UTF-8, depth 32, 64 KiB encoded strings, 500,000 tokens, 100,000 containers, 256 elements per general array, and 64 per file-set array. It accepts at most 256 model entries and 64 files in each file set, rejecting duplicate IDs, duplicate properties, unknown properties, invalid UTF-8, unsupported versions and semantic policy violations. `capabilities`, its `languages` array, and `convertedFiles` are explicit, required fields in schema 1 (use empty arrays where nothing is declared). Empty languages does not mean unrestricted language support. JSON uses `CatalogJsonContext`, generated at compile time, not reflection.

## CAM++ English developer artifact

The CAM++ English entry records the verified publisher checkpoint and the SHA-256/size of its
converted float32 graph. The converted file's content-addressed `models-v1` release URL is a
**reserved, unpublished artifact location**, not a claim that bytes are downloadable. Nothing has
been mirrored or published. Current `ModelStore.PullAsync` fetches publisher `files`, not
`convertedFiles`; registered conversions run locally in the cache.
CAM++ English is the speaker default; Chinese speech selects CAM++ Chinese. See [speaker inference](https://recursiveai.org/onespeech/docs/speakers/).

ModelScope publisher files use its HTTPS data API, with exactly `Revision=<pinned commit>` and
`FilePath=<catalog filename>` query fields in that order. The catalog checks the API host,
repository path, revision, filename, absence of extra query keys/fragments, and SHA-256/size.
This narrow query exception does not apply to other publishers, mirrors, or converted artifacts.


<!-- https://recursiveai.org/onespeech/docs/loading/ -->

# Loading and composing roles

Every downloadable family has the same canonical loader pair:

- `Load(ResolvedModel model, options/threads, cancellationToken)` opens already-pulled local files without network access. Obtain the directory/provenance pair from `ModelStore.PullAsync`, not by guessing cache filenames.
- `LoadAsync(ModelStore? store = null, string? model = null, options/threads, CancellationToken cancellationToken = default)` pulls, verifies and locally converts missing files before loading. Defaults are the family's declared checkpoint, not a tier-2 default for `Speech`. Calling a named tier-2 family loader is explicit selection of its documented publisher checkpoint.

Whisper, Moonshine, Qwen3-ASR and Nemotron ASR use typed options; other loaders accept `threads`, default 1. All family options call this knob `Threads`, default 1; families permitting 0 document it as runtime-auto. String-directory overloads and graph-path constructors remain for manually converted/custom graphs and require the documented family graph format. They are not needed for catalog loads. Embedded Silero uses its constructor and never downloads.

## Rebuild the file pipeline

```csharp
using SpeechRecognizer recognizer = await WhisperRecognizer.LoadAsync(
    model: "openai/whisper-tiny", cancellationToken: cancellationToken);
using VoiceActivityDetector recognitionVad = new SileroVad(cancellationToken: cancellationToken);
using SpeakerEmbedder embedder = await CampPlusEmbedder.LoadAsync(cancellationToken: cancellationToken);
using VoiceActivityDetector speakerVad = new SileroVad(SileroVadOptions.Diarization, cancellationToken);
using Diarizer diarizer = new ClusteringDiarizer(embedder, speakerVad);
using Aligner qwen = await Qwen3ForcedAligner.LoadAsync(cancellationToken: cancellationToken);
using Aligner ctc = await OmniAsrAligner.LoadAsync(cancellationToken: cancellationToken);
var pipeline = new VadTranscriptionPipeline(recognizer, recognitionVad)
{
    Diarizer = diarizer,
    TimingAligners = [qwen, ctc]
};
Transcript result = pipeline.Transcribe("meeting.wav", "en", cancellationToken);
```

Namespaces: `OneSpeech`, `.Whisper`, `.Vad`, `.Embedders`, `.Diarization`, `.Qwen3`, `.Alignment`, `.Pipeline`. `CampPlusEmbedder.LoadAsync(model: "iic/speech_campplus_sv_zh-cn_16k-common")` derives the Chinese dimension itself. The example uses explicit English timing/embedding; automatic per-recording language selection and deferred model acquisition are provided by `Speech.Transcribe`. It eagerly loads both timing candidates, unlike the lazy happy path. It uses default clustering and resegmentation with the diarization-specific VAD profile; no overlap detection is implied.

Roles are disposable owners of **their own** graph resources. Pipelines, clustering diarizers, live sessions and recognizer decorators borrow their collaborators; disposing a wrapper/session never disposes a borrowed recognizer, detector, aligner or embedder. Dispose sessions before their models and models after all processing has stopped. `Dispose` is idempotent. Custom subclasses override `Dispose(bool)` and call the base. Progress uses `IProgress<TimeSpan>` with processed audio time, never wall time.

`SpeechRecognizer.CreateLiveSession(VoiceActivityDetector detector, language)` accepts any detector. Silero sessions own independent recurrent state; custom detectors receive one zero-padded 32 ms mono frame at a time. They should be stateless or dedicated to that session. A custom VAD returns intervals, not calibrated probabilities; any positive interval counts as active for endpointing.


<!-- https://recursiveai.org/onespeech/docs/tier-2-models/ -->

# Tier-2 models, gates and credentials

Tier 2 includes CC-BY-4.0, OpenMDW-1.1 and every gated model even if its licence is MIT. Select these models explicitly by catalog ID. They are never defaults, mirrored weights or offline model packages. They load pinned publisher files with SHA-256 verification and may convert into the user's cache; converted tier-2 artifacts must not be redistributed. `ModelInfo` preserves licence, source, revision, file digests and exact attribution.

A publisher gate requires **your own account's acceptance** and an authorized read token. The library does not accept terms for you. Supply the explicit `token` parameter on `ModelStore.PullAsync`, or inject `ONESPEECH_MODEL_TOKEN` through your host's secret manager. Do not put credentials in command-line arguments, source, JSON transcripts, shared shell history or logs. The CLI reads the environment variable. Gate authorization is required even for cached files and custom sources.

HTTP authorization is sent only on the original exact publisher request, never to redirects or mirrors. Token material is not persisted by the downloader. Distinguish `ModelTokenRequiredException`, `ModelTokenRejectedException` and `ModelGateNotAcceptedException`; fix account access rather than retrying indefinitely. Network failures and digest mismatches have separate typed errors. Never bypass a gate with an unofficial mirror.

The [segmentation slot](https://recursiveai.org/onespeech/docs/gated-segmentation/) supports authorized local segmentation-3.0 snapshots and overlap-aware turns with our clustering. Its network entry point still refuses inference, and community-1 has no native pipeline adapter. Successful pre-fetch alone does not imply that a family has a working inference adapter. Nemotron 3 Diarization and Parakeet are ungated tier-2 examples with published inference support. See [model loading](https://recursiveai.org/onespeech/docs/models/) for cache precedence, trust boundaries and custom `ModelSource` hooks.


<!-- https://recursiveai.org/onespeech/docs/cli-and-model-packages/ -->

# Command line and offline model packages

Speech recognition for .NET. `onespeech --help` lists commands, positional arguments,
options and exit codes. The tool accepts standalone audio and OneMedia containers.
`transcribe` and `align` emit text, schema-v1 JSON, SRT or VTT (`--format`);
`--output` selects a UTF-8 file instead of stdout. `align` reads its second argument
as a UTF-8 text file. Model and language options are passed to the library, with
unsupported requests reported as typed errors rather than ignored.

`diarize` uses the low-level clustering defaults and returns turns without speech
recognition. Its text output is start seconds, end seconds and speaker ID, separated
by tabs. JSON uses the same Transcript schema; SRT/VTT contain speaker-label cues,
not recognized words. `--model nvidia/Nemotron-3-Diarization` selects the overlap-aware
alternative. The default clustering method does not detect overlap.

`identify query.wav alice enrollment.wav bob another.wav` holds all enrollments
only until that invocation exits. `verify first.wav second.wav` prints a cosine
score, not a probability or a calibrated decision. `enroll alice enrollment.wav`
checks enrollment and discards it on exit; it does not create a database.
Speaker samples are limited to 60 seconds per file. These commands use the `Speech`
speaker helpers by default; identification uses their [development-fitted operating points](https://recursiveai.org/onespeech/docs/speakers/#convenience-identification-operating-points).
An explicit `--threshold` uses the low-level registry instead and accepts a finite cosine in [-1,1].
Neither setting is a population authentication guarantee. Obtain informed consent and check local law (including BIPA and GDPR
Art. 9) before processing voices.

`models list` is offline. `models pull <id>` prefetches a pinned model; gate tokens
are read from `ONESPEECH_MODEL_TOKEN`, never a command-line argument or saved file.
Exit codes are 0 (success), 1 (model/audio/I/O failure), 2 (usage), 130 (cancelled).
Errors print their typed name and message to stderr; inference never writes logs
or progress to stdout. Ctrl+C cancels through the library's cancellation points.

## Offline packages

Reference `RecursiveAI.OneSpeech.Models.CamPlusEnglish`, `.MoonshineTiny`,
`.MoonshineBase`, or `.MoonshineStreamingTiny` alongside core. Each contains only
approved, pinned tier-1 publisher files, its notices and an AOT-safe transitive
build target. No reflection or assembly registration is used. Build and publish
copy the files into `onespeech-models/<catalog SHA-256>` beside the executable.
Keep that directory, including when deploying NativeAOT or a single-file app.

The loader tries these files before its configured ModelSource, verifies their
size and SHA-256, copies them atomically into the per-user cache, then performs
normal local conversion and verifies converted hashes. Corrupt package content
fails rather than falling back to network. Tier-2 models never read package
content. A package covers only its own model, not all roles in `Speech.Transcribe`
(which may also need an aligner and diarizer). `tests/OfflinePackageSmoke` demonstrates
low-level transcription with network access explicitly refused by its ModelSource.
NuGet restore itself needs the packages and their dependencies in a local feed/cache.

Maintainers prepare and pack via `tools/models/pack-model-packages.sh`. Set
`ONESPEECH_ARTIFACTS`, `MODEL_PACKAGE_CACHE` and `MODEL_PACKAGE_FEED` to external
directories. Optionally set `MODEL_PUBLISHER_CACHE` to an existing publisher-file
cache. The C# preparer fetches missing files, validates each against the embedded
catalog, and revalidates immediately before pack. No weights are stored in git.
All packages share the repository version and metadata; each archive must remain
below 250 MB. Nothing is published by the script.

Unaffiliated with other products or companies named OneSpeech.

## Command examples

After installing the tool from a feed containing its package and dependencies:

```sh
onespeech --help
onespeech models list
onespeech models pull openai/whisper-tiny
onespeech transcribe meeting.m4a --language en --format json --output meeting.json
onespeech align short.wav words.txt --language en --format vtt --output short.vtt
onespeech diarize meeting.wav --model nvidia/Nemotron-3-Diarization
onespeech identify query.wav alice enrollment.wav --threshold 0.3
onespeech verify first.wav second.wav
```

Model IDs select roles, not filenames. An ASR model passed to `diarize` is not a speaker embedder. First use may download additional models for alignment and diarization; pre-fetch those as well for a disconnected host. Gate tokens must be injected through the environment, never added to these command lines. Enrollment and query files must be obtained with appropriate consent.


<!-- https://recursiveai.org/onespeech/docs/media/ -->

# Container audio and captions

`RecursiveAI.OneSpeech.Media` references core, OneMedia and OneMedia.Audio. Call `OneSpeech.Media.MediaInput.Register()` once during startup, before inference. Registration sets core's explicit `AudioFileSource.AdditionalDecoder` hook; it does not scan assemblies, download models or change the document schema. OneAudio and the core G.711 reader continue handling their formats first. On an unsupported format the stream is rewound and OneMedia opens it. Containers without a decodable audio track fail rather than returning silence. OneMedia exceptions retain their typed details.

## Audio input

After registration:

```csharp
using OneSpeech;
using OneSpeech.Media;

MediaInput.Register();
var transcript = Speech.Transcribe("meeting.mkv");
var alignment = Speech.Align("meeting.mp4", "the supplied words", language: "en");
await foreach (var segment in Speech.TranscribeStream("lecture.webm"))
    Console.WriteLine(segment.Text);
await using var encoded = File.OpenRead("meeting.mkv");
var streamedInput = await Speech.TranscribeAsync(encoded, cancellationToken: CancellationToken.None);
```

The normal transcription pass includes diarization; core's low-level audio source exposes the same demuxed/resampled PCM to custom pipelines. The adapter uses OneMedia's default audio track and codecs, not video decoding. Supported containers include MP4/MOV, Matroska/WebM, AVI, transport streams, ASF and FLV, subject to OneMedia.Audio codec support. No claim is made that every codec inside those containers is decodable. Core downmixing, resampling, duration limits, timing and cancellation policies are unchanged. Disposing the adapter decoder releases its media file, not a caller-owned stream.

For nonseekable encoded input, `MediaInput.TranscribeStream(stream, language: "en", cancellationToken: token)` asynchronously copies at most 512 MiB to a temporary disk file, then yields core streaming final segments. The copy uses a 64 KiB buffer. It does not produce live partials while copying, nor run whole-file diarization. Cancellation, errors and early enumeration disposal remove the temporary file; the caller's stream is never closed. For larger files use a local path.

## Caption policy

`transcript.ToSubtitles()` returns OneMedia's cue document. Segments define the caption groups, or root words if no segments exist. With fully timed words a group splits at speaker changes, 84 UTF-16 characters or six seconds; lines wrap at 42 characters without splitting a word. A single long word or indivisible long interval can exceed those targets. A segment with any untimed word stays one wrapped cue: there is no evidence for extra split boundaries. Text without words stays a segment cue.

Every word becomes a UTF-16-range `SubtitleSpan`, carrying its measured absolute `WordStart`/`WordEnd`, language and speaker ID as `Voice`. Null bounds remain null. Cue bounds use the segment when untimed words are present, otherwise measured word bounds. Completely untimed cues remain `MediaTime.Unknown`; track encoders drop them and report the loss. They are not silently given an arbitrary duration. The four string export extensions refuse unknown cue bounds with CapabilityNotSupportedException; align those segments first. Untimed words inside an otherwise timed segment can still be exported.

When turns exist, core's majority-overlap assignment determines word speakers (equal activity, ordinal speaker-ID tie break); uncovered words remain unassigned. Cue `Speaker` is the first word's assigned speaker. Display cues are sorted by start; overlaps end at the next cue's start and fully covered display cues are omitted. This is the exclusive caption view, not a mutation of the original Transcript. Absolute word measurements are retained even when a display interval is clipped. Untimed cues are kept after timed cues in the in-memory document.

`document.ToTranscript()` imports cue text and bounds as segments, non-raw spans as words and cue speakers as turns. It never infers word times from cue bounds. Subtitle formats can merge spans, omit untimed span boundaries, round times and drop provenance. Imported timed words use `TimingMethod.Native` and null confidence; untimed spans use `None`. This is a subtitle interchange, not lossless Transcript serialization: use `ToJson()` for that.

```csharp
var cues = transcript.ToSubtitles();
var output = RecursiveAI.OneMedia.Captions.SubtitleFormats.Ttml.Write(cues);
Console.WriteLine(output.Report); // fidelity losses, if any
File.WriteAllText("meeting.ttml", output.Text);
```

`ToSrt`, `ToVtt`, `ToAss` and `ToTtml` are convenience extensions over the same writers. SRT carries neither speakers nor word times. WebVTT carries voices and word starts (ends are reconstructed by the reader); ASS uses cue-level speaker and centisecond karaoke; TTML carries agents and timed spans. Consult the OneMedia conversion report when fidelity matters.

## Caption muxing

`transcript.MuxCaptions(input, output)` copies compatible audio/video/subtitle tracks from MP4/MOV/3GP to a new moov-last MP4 and adds a WebVTT track. It copies packets in each track's decode order; it does not re-encode, repackage unsupported codecs, copy container metadata or promise optimal interleaving. Non-ISO input is refused to avoid copying incompatible packet framing; use OneMedia remuxing first. OneMedia's typed mux refusals are preserved. Output must not exist. Failure removes the incomplete file created by the call, never an existing destination.

For a caller-managed muxer, `EncodeCaptionTrack(TextTrackFormat.WebVtt)` returns the Matroska track declaration, packets and losses. Declare all tracks first, then write each caption packet using the index returned by `AddTrack`, then `Finish`. Use `WebVttIso` (the default) for MP4. OneMedia owns the format mapping; this package contains no duplicate container reader, subtitle parser or muxer.


<!-- https://recursiveai.org/onespeech/docs/results/ -->

# Measured results and limits

The release revalidation table distinguishes full-split measurements from the earlier fixed-slice evidence below. The tables disclose incomplete datasets, qualified references and remaining gaps; these results are not a cross-hardware guarantee. Lower is better for all errors below. Scores are percentages unless otherwise stated. Unless noted, recognition quality rows use float32. Models and publisher revisions are pinned in the [catalog](https://recursiveai.org/onespeech/docs/models/); see [notices](https://recursiveai.org/onespeech/docs/third-party-notices/) for attribution. No speed comparison with a native implementation is made here.

**CI convention:** the release table labels absolute-score and paired-difference 95% intervals separately. The older slice tables give only intervals for **our score minus the reference score**, in percentage points. Recognition uses 1,000 utterance resamples, seed 1729, and the same `en/1` scoring normalizer on both sides. “Within noise” means the difference interval includes zero. With no computable CI, the comparison uses a 0.5-point bound. Slices must not be interpreted as full-test results.

## Release quality revalidation (2026-11-08–11)

The inference measurements started from core source `dd63517` through the public recognizer, live-session and embedding APIs. Later reader hardening changes invalid-input handling, not valid-model inference. Moonshine uses its default greedy decoding, 6.5 generated tokens per audio second, no biasing, and two CPU threads. The reference uses the pinned publisher weights, Transformers 5.22.2, float32, eager attention and deterministic generation. Speaker extraction uses eight threads on each side. Whisper int8 rows use temperature-zero greedy decoding without word timestamps on both sides; the reference uses beam/best-of one, no VAD filter and previous-text conditioning. Native CPU reservations changed from eight to two threads during the resumable run; precision and decoding controls did not change. No previously saved hypotheses or embeddings were substituted for the new inference runs.

| Model / task | Dataset | Metric | Ours [absolute 95% CI] | Reference / version | Paired difference CI | Verdict |
|---|---|---|---|---|---|---|
| Whisper large-v3-turbo fp32 | Full LibriSpeech test-clean, 2,620 utterances | WER % | 2.267 [2.093, 2.432] | 2.289 / openai-whisper 20250625 | [-0.136, +0.072] pp | Within noise |
| Whisper large-v3-turbo fp32 | Full LibriSpeech test-other, 2,939 utterances | WER % | 4.275 [4.032, 4.538] | 4.331 / openai-whisper 20250625 | [-0.139, +0.019] pp | Within noise |
| Whisper small fp32 | Full LibriSpeech test-clean, 2,620 utterances | WER % | 3.650 [3.452, 3.877] | 3.669 / openai-whisper 20250625 | [-0.083, +0.046] pp | Within noise |
| Whisper small fp32 | Full LibriSpeech test-other, 2,939 utterances | WER % | 7.852 [7.486, 8.236] | 7.924 / openai-whisper 20250625 | [-0.171, +0.031] pp | Within noise |
| Whisper large-v3-turbo mixed-int8 | Full LibriSpeech test-clean, 2,620 utterances | WER % | 2.291 [2.118, 2.461] | 2.355 / faster-whisper 1.2.1, CT2 4.8.5 int8 | [-0.367, +0.118] pp | Within noise |
| Whisper large-v3-turbo mixed-int8 | Full LibriSpeech test-other, 2,939 utterances | WER % | 4.479 [4.227, 4.734] | 4.503 / faster-whisper 1.2.1, CT2 4.8.5 int8 | [-0.514, +0.264] pp | Within noise |
| Whisper small mixed-int8 | Full LibriSpeech test-clean, 2,620 utterances | WER % | 3.731 [3.517, 3.965] | 3.927 / faster-whisper 1.2.1, CT2 4.8.5 int8 | [-0.645, +0.074] pp | Within noise |
| Whisper small mixed-int8 | Full LibriSpeech test-other, 2,939 utterances | WER % | 8.038 [7.658, 8.449] | 8.007 / faster-whisper 1.2.1, CT2 4.8.5 int8 | [-0.145, +0.185] pp | Within noise |
| Moonshine streaming-tiny, offline | Full LibriSpeech test-clean, 2,620 utterances | WER % | 4.811 [4.505, 5.138] | 4.811 / Transformers 5.22.2 | [0, 0] pp | Within noise |
| Moonshine streaming-tiny, offline | Full LibriSpeech test-other, 2,939 utterances | WER % | 12.616 [12.089, 13.133] | 12.620 / Transformers 5.22.2 | [-0.012, 0] pp | Within noise |
| Moonshine streaming-tiny mixed-int8, offline | Full LibriSpeech test-clean, 2,620 utterances | WER % | 4.771 [4.474, 5.090] | 4.811 / our fp32 | [-0.097, +0.011] pp | Within 0.5 pp of fp32 |
| Moonshine streaming-tiny mixed-int8, offline | Full LibriSpeech test-other, 2,939 utterances | WER % | 12.641 [12.107, 13.176] | 12.616 / our fp32 | [-0.103, +0.152] pp | Within 0.5 pp of fp32 |
| Moonshine streaming-tiny, live, 100 ms pushes | LibriSpeech test-clean, first 200 | WER % | 5.070 [4.016, 6.222] | No equivalent streaming-policy oracle | — | Floor |
| Moonshine streaming-tiny, live, 100 ms pushes | LibriSpeech test-other, first 200 | WER % | 18.185 [16.110, 20.255] | No equivalent streaming-policy oracle | — | Floor |
| Moonshine streaming-tiny, live, 100 ms pushes | LibriSpeech test-clean, first 200 | Mean finalisation latency, ms | 2254.6 [2000.6, 2559.8] | Whisper tiny DTW word-end estimates, openai-whisper 20250625 | — | Floor, not independent truth |
| Moonshine streaming-tiny, live, 100 ms pushes | LibriSpeech test-other, first 200 | Mean finalisation latency, ms | 1535.6 [1416.3, 1649.1] | Whisper tiny DTW word-end estimates, openai-whisper 20250625 | — | Floor, not independent truth |
| CAM++ English | LibriSpeech fixed 20,000 trials | EER % | 0.750 [not computed] | 0.750 / ModelScope 1.40.2 | [0, 0] pp | Within noise |
| CAM++ English | LibriSpeech fixed 20,000 trials | minDCF | 0.0379 [not computed] | 0.0379 / ModelScope 1.40.2 | [0, 0] | Within noise |
| ECAPA-TDNN | LibriSpeech fixed 20,000 trials | EER % | 1.100 [not computed] | 1.100 / SpeechBrain 1.1.2 | [0, 0] pp | Within noise |
| ECAPA-TDNN | LibriSpeech fixed 20,000 trials | minDCF | 0.0548 [not computed] | 0.0548 / SpeechBrain 1.1.2 | [0, 0] | Within noise |
| Whisper large-v3-turbo, language ID | FLEURS, five each English/French/German | Accuracy | 15/15 [not computed] | 15/15 / openai-whisper 20250625 | — | Information only |

Full-split Moonshine token/text identity is 5,558/5,559 sequences. The differing test-other recording is `6070-86744-0018` (33.645 seconds); it contains punctuation and word differences, not merely a different special-token representation. The within-noise WER result does **not** imply token-identical decoding on every recording.

Live finalisation latency means audio-time `Final` availability minus an edit-matched word's estimated end, not CPU inference latency. There are 4,427 matched tokens out of 4,685 reference tokens on clean, and 2,524 out of 3,086 on other. The calculation retains 35/32 negative estimates rather than clamping them. Latency intervals resample recordings 1,000 times with seed 1729. The 200-recording floors and the earlier 20-recording floors below have different populations and must not be compared as a regression test. Scoring the retained first-20 subset separately reproduces the earlier rounded floors: WER 1.70/8.33% and mean latency 3057/1617 ms.

Speaker trials contain 10,000 same-speaker and 10,000 different-speaker pairs, 73 speakers and 1,460 distinct trial utterances. Extraction covered 1,490 utterances, including additional parity probes. Paired bootstrap resamples trials within same/different strata, conditional on this fixed population; minDCF uses prior 0.01, unit costs and normalization. These rows do not replace the unavailable VoxCeleb1-O evaluation. Model revisions are Moonshine `f8e9dfd8c562c257c151a907b7b7f2fe8ff8511a`, CAM++ English `032b8131a7ad812f87061955ca974c99060c5a03`, and ECAPA `0f99f2d0ebe89ac095bcc5903c4dd8f72b367286`.

Environment: CPU-only x64, AMD Ryzen 9 7950X3D (16 physical / 32 logical CPUs; FMA, AVX2 and AVX-512 available), .NET SDK 10.0.403 / runtime 10.0.14, OneModel 1.2.0 and OneAudio 1.7.2. Language ID uses eight threads. Runs used a shared CPU queue; these are correctness measurements, not publishable speed rows.

**Coverage:** full Whisper turbo/small fp32 and int8, and full Moonshine fp32/int8 rows are complete. Each LibriSpeech comparison contains all 2,620 clean or 2,939 other utterances. Parakeet full-set, Qwen first-500 CPU-limited slices, five-recording long-form comparisons and timing revalidation are in the additional ASR section below. Full-set diarization and Nemotron streaming are reported in their separate section below. The fixed shortest-duration Earnings-22 selection is `4474955`, `4483046`, `4468919`, `4479524`, `4481221` (875.020, 1193.180, 1214.487, 1226.213, 1236.904 seconds). Its measured long-form rows and both temperature policies are reported below. Earlier misses below remain visible and are not superseded by unmeasured rows.

### Reader mutation campaigns

Each listed target completed a 600-second mutation budget (libFuzzer reported 601 seconds), with one fuzzer running at a time and four CPU cores reserved for publishing and mutation. SharpFuzz 2.2.0 instruments a disposable copy of the assembly containing the reader; libfuzzer-dotnet is release v2025.05.02.0904. Mutation input limits are 64 KiB, except SentencePiece (128 KiB) and Parakeet vocabulary (256 KiB). These bounded campaigns are not an exhaustive proof for all input sizes or dependency readers.

| Target | Mutation budget, s | Executions | Coverage features | Crashes |
|---|---|---|---|---|
| Transcript JSON | 600 | 28,843,861 | 1,596 | 0 |
| Model catalog | 600 | 24,003,517 | 865 | 0 |
| WAV / G.711 entry | 600 | 21,043,276 | 398 | 0 |
| General audio decode entry | 600 | 17,547,248 | 282 | 0 |
| Checkpoint ZIP | 600 | 27,408,863 | 1,045 | 0 |
| Checkpoint pickle index | 600 | 16,476,079 | 2,166 | 0 |
| Safetensors | 600 | 32,004,710 | 507 | 0 |
| SentencePiece | 600 | 27,466,894 | 1,081 | 0 |
| Moonshine tokenizer / configuration | 600 | 19,052,083 | 580 | 0 |
| Parakeet vocabulary / normalization | 600 | 2,599,800 | 251 | 0 |
| RTTM (evaluation tool) | 600 | 27,514,840 | 236 | 0 |
| Whisper byte-BPE files | 600 | 2,280,339 | 221 | 0 |
| Qwen byte-BPE files | 600 | 1,455,773 | 265 | 0 |
| Whisper configuration | 600 | 2,060,381 | 69 | 0 |
| Qwen configuration | 600 | 3,929,405 | 85 | 0 |
| Converter provenance (Whisper/Moonshine/Qwen) | 600 | 1,485,318 | 1,073 | 0 |
| Nemotron ASR processor configuration | 600 | 2,166,332 | 174 | 0 |
| Whisper generation configuration | 600 | 1,946,348 | 157 | 0 |

The checkpoint campaigns instrument the core assembly where the reader now resides, not the converter assembly. General audio decoding is exercised without the WAV-only prefilter; OneAudio itself is not instrumented. RTTM is an evaluation-tool reader, not a shipped core reader. No mutated-input crash was found. New hostile-input tests also cover malformed byte-BPE/configuration shapes, empty special tokens and invalid generation IDs/heads. The complete [reader inventory](https://recursiveai.org/onespeech/docs/model-data-readers/) distinguishes owned readers from dependency parsers. One new harness startup failed because it touched an instrumented static initializer before fuzzer initialization; setup was corrected before the clean campaigns, with no input crash to minimize.

## Offline recognition

LibriSpeech rows are the first 200 sorted utterance IDs of each named split. Each row is separate; results are not averaged across splits or models.

| Model | Dataset slice | Metric / ours | Reference score | Black-box reference | Difference CI | Verdict |
|---|---|---|---|---|---|---|
| Whisper small | test-clean 200 | WER 2.503 | 2.674 | openai-whisper 20250625 | [-0.493, +0.066] | Within noise |
| Whisper small | test-other 200 | WER 10.280 | 10.312 | openai-whisper 20250625 | [-0.556, +0.554] | Within noise |
| Whisper large-v3-turbo | test-clean 200 | WER 1.476 | 1.561 | openai-whisper 20250625 | [-0.321, +0.129] | Within noise |
| Qwen3-ASR 0.6B | test-clean 200 | WER 1.754 | 1.711 | qwen-asr 0.0.6 | [0, +0.155] | Within noise |
| Qwen3-ASR 0.6B | test-other 200 | WER 5.628 | 5.628 | qwen-asr 0.0.6 | [0, 0] | Within noise |
| Qwen3-ASR 1.7B | test-clean 200 | WER 1.433 | 1.369 | qwen-asr 0.0.6 | [0, +0.233] | Within noise |
| Qwen3-ASR 1.7B | test-other 200 | WER 4.131 | 4.131 | qwen-asr 0.0.6 | [0, 0] | Within noise |
| Moonshine tiny | test-clean 200 | WER 3.72 | 3.72 | Transformers 5.22.2 | [0, 0] | Within noise |
| Moonshine tiny | test-other 200 | WER 14.54 | 14.54 | Transformers 5.22.2 | [0, 0] | Within noise |
| Moonshine base | test-clean 200 | WER 2.74 | 2.74 | Transformers 5.22.2 | [0, 0] | Within noise |
| Moonshine base | test-other 200 | WER 11.39 | 11.39 | Transformers 5.22.2 | [0, 0] | Within noise |
| Moonshine streaming-tiny | test-clean 200 | WER 3.64 | 3.64 | Transformers 5.22.2 | [0, 0] | Within noise |
| Moonshine streaming-tiny | test-other 200 | WER 14.96 | 14.96 | Transformers 5.22.2 | [0, 0] | Within noise |
| Moonshine streaming-small | test-clean 200 | WER 2.10 | 2.10 | Transformers 5.22.2 | [0, 0] | Within noise |
| Moonshine streaming-small | test-other 200 | WER 9.43 | 9.43 | Transformers 5.22.2 | [0, 0] | Within noise |
| Parakeet-TDT-0.6B-v3 | test-clean 200 | WER 2.053 | 2.053 | NeMo 3.0.0, greedy TDT | [0, 0] | Within noise |
| Parakeet-TDT-0.6B-v3 | test-other 200 | WER 4.424 | 4.424 | NeMo 3.0.0, greedy TDT | [0, 0] | Within noise |
| omniASR-CTC-300M | test-clean 200 | CER 1.2879 | 1.2879 | omnilingual-asr 0.2.0 | [0, 0] | Within noise |

The omniASR row tests the aligner's acoustic decoding, not general-purpose ASR coverage. Moonshine produced 1,600/1,600 identical token sequences across its eight rows; Parakeet produced identical tokens and frames across the two slices (14,365 tokens).

## Timing

Boundary errors are mean absolute start/end error in milliseconds. Confidence intervals were not supplied for these rows. A timing floor is a first measurement against independent truth, **not** a claim of equivalence to that truth.

| Method | Slice / paired words | Reference | Start / end ms | Within 50 ms start / end | Within 100 ms start / end | Verdict |
|---|---|---|---|---|---|---|
| Whisper-small DTW | test-clean 200 / 4,625 | openai-whisper 20250625 timestamps | 11.9 / 8.6 | 97.0 / 98.1% | 98.7 / 99.8% | Measured implementation difference |
| Qwen3-ForcedAligner | test-clean 200 | qwen-asr 0.0.6 toolkit | 28.7 / 18.9 | 90.8 / 98.1% | 94.4 / 98.1% | Post-processing parity gap; 67/4,634 untimed |
| Parakeet native | test-clean / 4,633 | NeMo 3.0.0 | 0 / 0 | 100 / 100% | 100 / 100% | Identical boundaries |
| omniASR CTC | test-clean 200 / 4,531 common words | MFA 3.4.3 | 52.35 / 50.33 | Not supplied | Not supplied | Floor |
| omniASR CTC | test-other 100 / 1,465 common words | MFA 3.4.3 | 49.77 / 53.00 | Not supplied | Not supplied | Floor |
| Our Whisper-small DTW | test-clean 200 / same common words | MFA 3.4.3 | 94.16 / 79.23 | Not supplied | Not supplied | Floor |
| Our Whisper-small DTW | test-other 100 / same common words | MFA 3.4.3 | 104.27 / 75.52 | Not supplied | Not supplied | Floor |
| Reference Whisper DTW | test-clean 200 / same common words | MFA 3.4.3 | 92.87 / 77.98 | Not supplied | Not supplied | Information only |
| Reference Whisper DTW | test-other 100 / same common words | MFA 3.4.3 | 103.96 / 75.80 | Not supplied | Not supplied | Information only |

MFA truth uses English MFA acoustic model/dictionary v3.0.0 with speaker adaptation disabled. The CTC aligner left 0/4,634 clean words and 9/1,562 other words untimed. Common-word comparisons exclude words not timed by every compared method; they do not erase these failures.

## Diarization

DER uses **no collar and scores overlap**. Values below are DER / JER percentages. Single-recording rows have no bootstrap CI. AMI uses Mix-Headset test recordings ES2004a/b; AliMeeting uses Eval far-field channel 1. The tier-1 method is Silero + CAM++ English (AMI) or Chinese (AliMeeting), spectral clustering and cosine resegmentation; its settings were selected on AMI development ES2011a/b, not these test recordings.

| Diarizer | Recording | Ours DER / JER | Reference DER / JER | Black-box reference | Verdict (DER / JER) |
|---|---|---|---|---|---|
| Tier-1 clustering | AMI ES2004a | 28.21 / 29.62 | 26.54 / 29.63 | ModelScope 1.40.2 CAM++ diarization bundle | Miss / within 0.5 |
| Tier-1 clustering | AMI ES2004b | 20.92 / 19.77 | 18.10 / 19.26 | ModelScope 1.40.2 CAM++ diarization bundle | Miss / miss |
| Tier-1 clustering | AliMeeting R8001_M8004_MS801 | 41.57 / 41.13 | 34.52 / 34.02 | ModelScope 1.40.2 CAM++ diarization bundle | Miss / miss |
| Tier-1 clustering | AliMeeting R8003_M8001_MS801 | 26.79 / 28.95 | 26.89 / 28.58 | ModelScope 1.40.2 CAM++ diarization bundle | Within 0.5 / within 0.5 |
| Nemotron 3 Diarization | AMI ES2004a | 26.90 / 29.08 | 26.90 / 29.09 | Transformers 5.22.2 | Within 0.5 / within 0.5 |
| Nemotron 3 Diarization | AMI ES2004b | 21.62 / 21.53 | 21.62 / 21.53 | Transformers 5.22.2 | Within 0.5 / within 0.5 |
| Nemotron 3 Diarization | AliMeeting R8001_M8004_MS801 | 18.64 / 18.03 | 18.64 / 18.03 | Transformers 5.22.2 | Within 0.5 / within 0.5 |
| Nemotron 3 Diarization | AliMeeting R8003_M8001_MS801 | 18.30 / 18.54 | 18.30 / 18.54 | Transformers 5.22.2 | Within 0.5 / within 0.5 |

Tier-1 overlap detection is absent. These misses are real limitations, not rounded-away successes. Short recordings (at most 60 seconds) now use cosine AHC: a separate seeded LibriSpeech short-recording check improved speaker-count accuracy from 9/24 to 23/24 (singles 12/12, pairs 11/12), with DER 5.85%. It is not an AMI/VoxConverse population estimate.

Online clustering floors with 100 ms pushes: ES2004a 47.08/54.66, ES2004b 26.88/27.71, R8001 56.39/53.94, R8003 39.41/42.64 DER/JER. Turn availability delay averaged 1.32–1.71 seconds. No equivalent reference-policy CI is available; live and whole-recording clustering are not interchangeable.

## Speaker embeddings

LibriSpeech test-clean + test-other, 20,000 balanced trials, 73 speakers, seed 1729. These are **identity-only trials on this fixed population**, not open-set VoxCeleb1-O results. The paired-trial CI is conditional on these speakers. minDCF uses prior 0.01, miss/false-alarm costs 1, normalized; scores use raw cosine.

| Model | Metric / ours | Reference | Reference version | Difference CI | Verdict |
|---|---|---|---|---|---|
| CAM++ English | EER 0.75%; minDCF 0.0379 | 0.75%; 0.0379 | ModelScope 1.40.2 | [0,0] both | Within noise |
| ECAPA-TDNN | EER 1.10%; minDCF 0.0548 | 1.10%; 0.0548 | SpeechBrain 1.1.2 | [0,0] both | Within noise |
| CAM++ Chinese | EER 2.87%; minDCF 0.1876 | 2.87%; 0.1876 | ModelScope 1.40.2 | [0,0] both | Information on English trials |

## Authorized local segmentation

The local segmentation-3.0 adapter uses the pinned publisher checkpoint with
float32 OneModel 1.2.0. On the first five ordinal LibriSpeech test-clean clips
(1089-134686-0000 through 0004), padded/truncated to 10 seconds, probability-space
max absolute error against pyannote.audio 4.0.7 / CPU Torch 2.11.0 is 9.19e-6;
the largest per-clip mean error is 2.64e-7. All 2,945 powerset frame decisions
match. This probe row uses four threads on x64 (AMD Ryzen 9 7950X3D);
it does not establish a bitwise or all-ISA guarantee. Raw log-probability errors and crafted low-variance cases are described in
[the segmentation contract](https://recursiveai.org/onespeech/docs/gated-segmentation/); this is not bitwise parity.

The following are **floors for our segmentation-plus-CAM++ clustering policy**,
not community-1 equivalence. Both sides use fp32, no collar, overlap scored, and
the same scorer/truth. AMI v1.6.2 uses Mix-Headset audio; AliMeeting Eval uses
far-field channel 1. Each row is one recording, without a bootstrap CI.
Our side uses four CPU threads, independent 10 s chunks and the existing
spectral/AHC defaults; CAM++ English on AMI and Chinese on AliMeeting.
Community-1 is the publisher's complete pipeline (different embedder and PLDA/VBx),
run offline through pyannote.audio 4.0.7, for information only. Its ES2004a
reference uses four CPU threads; the remaining reference shards use eight to
fit the per-recording run bound, with unchanged model/pipeline parameters.

| Recording | Our DER / JER % | Community-1 DER / JER % | Status |
|---|---|---|---|
| AMI ES2004a | 20.62 / 25.99 | 18.76 / 22.74 | Floor; reference information |
| AMI ES2004b | 13.40 / 15.83 | 12.61 / 14.28 | Floor; reference information |
| AliMeeting R8001_M8004_MS801 | 26.46 / 30.41 | 22.02 / 24.55 | Floor; reference information |
| AliMeeting R8003_M8001_MS801 | 20.40 / 24.11 | 17.38 / 19.14 | Floor; reference information |

Community-1's native WeSpeaker/PLDA/VBx adapter is not implemented. Its published
PLDA does not require fitting, but its exact transform/dataflow contract has not
yet been established for an independent implementation. Its PLDA is never
silently substituted for a CAM++ backend. The tier-1 default is unchanged.


## Additional ASR, long-form and timing results

All WER rows: fp32, `en/1` normalizer, paired utterance (or file) bootstrap, 1000 resamples, seed 1729, scored by `onespeech-eval score`. "Delta" is ours minus reference; CIs are of the paired difference. Machine: AMD Ryzen 9 7950X3D (16C/32T, AVX2/FMA/AVX-512), shared worker at load 22-25, CPU only. .NET SDK 10.0.403, OneModel 1.2.0, OneAudio 1.7.2. Jobs ran as queue shards of 4-6 cores; no timing claim is made from them.

| Row | Data | Ours (95% CI) | Reference (pin) | Delta [CI] | Verdict |
|---|---|---|---|---|---|
| Parakeet-TDT-0.6B-v3 (tier 2), greedy TDT | LibriSpeech test-clean, all 2620 | 2.088% [1.931, 2.241] | 2.088% (NeMo 3.0.0, fp32) | 0.000 [0, 0] | within noise |
| same | test-other, all 2939 | 3.791% [3.575, 4.027] | 3.791% | 0.000 [0, 0] | within noise |
| Qwen3-ASR-1.7B | test-clean, first 500 sorted IDs | 1.326% [1.026, 1.677] | 1.260% (qwen-asr 0.0.6 on Transformers 4.57.6, fp32) | +0.066 [0, +0.175] | within noise |
| same | test-other, first 500 sorted IDs | 5.815% [5.077, 6.624] | 5.815% | 0.000 [0, 0] | within noise |
| Whisper large-v3-turbo long-form, temperature 0 on both sides | Earnings-22, 5 files (4474955, 4483046, 4468919, 4479524, 4481221) | 13.427% [9.701, 17.827] | 85.101% [49.0, 125.3] (openai-whisper 20250625 `transcribe()`) | -71.674 [-115.1, -33.6] | better than the reference pipeline; see below |
| same, default fallback temperatures, both sides (information) | same 5 files | 13.427% [9.701, 17.827] | 19.524% [17.243, 21.736] | -6.097 [-10.047, -2.409] | better than the reference pipeline |

Parakeet: all 99,653 (clean) and 97,717 (other) reference tokens are identical to NeMo's, and so are all emission frames (2620/2620 and 2939/2939 utterances). The raw text strings differ in 6 and 5 utterances only by spacing; the normalized scores are identical.

Qwen3-ASR-1.7B: the first 500 sorted IDs per split, not the full splits. The CPU reference costs about 115 utterances per hour on a shared host, so the full 5559 utterances did not fit. Raw text differs in 2 of 500 (clean) and 1 of 500 (other) utterances.

Long-form, per file (ours; reference at temperature 0; reference with default fallback), WER %:

| File | Ours | Reference T=0 | Reference default |
|---|---:|---:|---:|
| 4474955 | 8.473 | 58.386 | 21.960 |
| 4483046 | 21.119 | 58.837 | 20.711 |
| 4468919 | 13.790 | 37.137 | 22.137 |
| 4479524 | 10.070 | 140.140 | 15.944 |
| 4481221 | 10.869 | 136.665 | 16.805 |

Why the reference rows are large: at temperature 0, `transcribe()` with its default previous-text conditioning falls into repetition loops on these long recordings (reference output for 4479524 has 4000 words against 2122 reference words; for 4474955 one segment text repeats 346 times, for 4481221 1013 times). Our pipeline chunks at speech boundaries and does not condition across chunks, so it does not loop. Ours is identical with and without fallback on this set. These rows are better than the reference pipeline, not an equality or within-noise claim; the comparison is against the reference's own pipeline. The 4483046 default-fallback row is the one where the reference is better than ours-at-T=0 in per-file terms (20.711 vs 21.119, paired, no CI at n=1).

### Timing revalidation on current code

Mean absolute boundary error (ms) and share within 50/100 ms, pooled over timed words, against MFA 3.4.3 (English v3.0.0, speaker adaptation off) alignments of the same LibriSpeech slices (test-clean first 200, test-other first 100). Floors are the values recorded earlier; no row regressed.

| Method | Slice | Matched/timed | Start mean | End mean | Start within 50/100 % | End within 50/100 % | Recorded floor |
|---|---|---:|---:|---:|---|---|---|
| Timing order via `Speech.Transcribe` (turbo + Qwen3-ForcedAligner) | clean 200 | 4563/4563 | 26.17 | 26.95 | 91.41 / 98.79 | 90.88 / 98.53 | 26.2 / 27.0 |
| same | other 100 | 1499/1499 | 27.72 | 26.98 | 90.59 / 98.13 | 90.46 / 98.33 | none |
| omniASR-CTC via `Speech.Align` | clean 200 | 4634/4634 | 52.70 | 50.94 | 53.28 / 91.11 | 60.25 / 88.82 | 52.70 / 50.94 (identical) |
| same | other 100 | 1571/1562 | 49.69 | 53.37 | 57.11 / 92.25 | 59.73 / 86.81 | 49.69 / 53.37 (identical) |
| Whisper small DTW (ours) | clean 200 | 4540/4540 | 94.24 | 79.28 | 49.07 / 71.15 | 46.34 / 69.25 | identical |
| same | other 100 | 1470/1470 | 104.07 | 75.49 | 49.80 / 72.86 | 47.89 / 71.77 | identical |
| openai-whisper DTW (information, unchanged archived run) | clean / other | 4531 / 1469 | 92.87 / 103.89 | 77.98 / 75.80 | | | |

Qwen3-ForcedAligner vs the toolkit, clean 200 (4634 words): start/end mean 28.73/18.90 ms; start within 50/100 ms 90.80/94.37 %, end 98.07/98.10 %; 67 words untimed by ours, 0 by the toolkit. Identical to the recorded row (post-processing parity remains a documented gap).

Numbers are word-pooled and cover the matched words of each method's own transcript, so the turbo row (its own recognized words) and the CTC row (reference text) have different word sets, as in the earlier floors.

## Diarization and cache-aware streaming — full-set evaluation

Float32 CPU evaluation on an AMD Ryzen 9 7950X3D (x64; AVX2/FMA/AVX512 available), .NET SDK 10.0.403/runtime 10.0.14, OneModel 1.2.0 and OneAudio 1.7.2. These are quality and audio-clock availability measurements, not throughput benchmarks.

### Data and scoring

- **AMI:** all 16 Full-corpus-ASR test meetings: EN2002a–d, ES2004a–d, IS1009a–d and TS3003a–d. Original Mix-Headset audio, 16 kHz mono, word annotations v1.6.2; positive silence gaps retained. CC-BY-4.0.
- **AliMeeting:** all eight Eval far-field recordings, first input channel, 16 kHz mono, nonempty TextGrid intervals. CC-BY-SA-4.0, evaluation only.
- DER has **no collar and scores overlap**. Dataset DER pools reference speaker-seconds; JER averages reference-speaker errors. Paired 95% intervals use 1,000 meeting resamples, seed 1729. Meetings within an AMI series share participants; these are meeting, not independent-speaker, resamples.
- **default:** the unchanged file-helper policy, Silero VAD + CAM++ English for AMI/Chinese for AliMeeting + spectral/AHC clustering and cosine resegmentation. It does **not detect overlap**. Reference: released ModelScope 1.40.2 CAM++ bundle, pinned local auxiliaries, float32 CPU, four threads. No test-set fitting.
- **nemotron:** Nemotron 3 Diarization, explicitly selected tier 2, compared with Transformers 5.22.2 at the same publisher pin and precision; four threads on both sides.
- **pyannote:** the segmentation-3.0/CAM++ slot, four threads, a **floor**, not a community-1 reimplementation. The pyannote.audio 4.0.7 community-1 pipeline is information only: its embedding, stitching and PLDA/VBx backend differ. Authorized local snapshots, offline; eight CPU threads, or sixteen on five recordings that exceeded the bounded eight-thread reservation. Neither tier-2 path is a default.

### Raw and qualified reference results

The ModelScope public output for EN2002c contains one reversed interval, `[2043.28, 2043.10, speaker 1]`, among 513 returned turns. The raw output and strict error are preserved. A negative-duration interval has no valid RTTM representation; a numeric raw reference score is therefore unavailable. An explicit qualified projection omits **only this interval** and retains the other 512 turns. It never swaps endpoints or invents timing.

| EN2002c reference treatment | Our DER / JER % | Reference DER / JER % | Interpretation |
|---|---:|---:|---|
| Raw public output, strict serialization | 32.8114 / 32.7066 | Unscorable: reversed interval | Unqualified full-AMI reference score/CI unavailable |
| Explicitly qualified projection | 32.8114 / 32.7066 | 30.5655 / 32.7222 | Full-AMI comparison below is conditional on this one omission |

All other ModelScope recordings were valid. Dataset-level tier-1 differences include zero in their paired intervals, but that does not hide substantial individual-meeting differences: for example EN2002b is +6.63 DER/+15.59 JER points. Every meeting is listed below. Nemotron is within paired noise on both complete sets. The pyannote slot remains below community-1 quality: AMI differences are +3.68 DER/+8.12 JER points and AliMeeting +4.65/+5.67 points, with positive paired intervals; those are disclosed information-baseline gaps, not equivalence passes.

### Nemotron 3.5 ASR streaming 0.6B

First 200 ordinal LibriSpeech test-clean/test-other utterances, float32 greedy English decoding. Reference: Transformers 5.22.2 CPU, two threads; OneSpeech four threads, actual published live-event policy. Known utterance boundaries isolate recognition from endpoint-detector differences. Normalizer **en/1**, 1,000 paired utterance resamples, seed 1729.

All **800** native/reference utterance/chunk pairs have identical lexical token sequences. Both sides were freshly generated. Paired WER differences and their 95% intervals are **[0,0]** on every row.

| Chunk ms | Split / utterances | WER ours = reference % | Each WER 95% CI % | Token-identical pairs |
|---:|---|---:|---|---:|
|80|test-clean / 200|2.888|[2.264, 3.589]|200/200|
|80|test-other / 200|9.824|[8.331, 11.312]|200/200|
|560|test-clean / 200|2.802|[2.223, 3.493]|200/200|
|560|test-other / 200|8.295|[6.983, 9.672]|200/200|

**Stable-word availability** closes a word at the next word boundary or EOF. Both engines use the same saved offline 80-ms RNN-T acoustic endpoints as a timing proxy, not forced-alignment truth. Signed negative delays are retained. Latency means are over matched words; unmatched counts are shown. This is audio-time availability, not CPU speed or a learned endpointing-policy comparison. Publication remains approximately **55–58 ms later** in OneSpeech, so latency parity is not claimed.

| Chunk ms | Split | Ours mean / median ms | Reference mean / median ms | Mean gap ms | Matched / unmatched words |
|---:|---|---|---|---:|---|
|80|test-clean|258.541 / 240.000|200.100 / 179.000|+58.441|4576 / 64|
|80|test-other|217.315 / 160.000|160.448 / 99.000|+56.867|2871 / 143|
|560|test-clean|405.200 / 400.000|347.623 / 339.000|+57.577|4594 / 48|
|560|test-other|348.760 / 320.000|293.566 / 259.000|+55.194|2895 / 123|

Actual EOF Final availability relative to the last common offline word end is identical for both engines and chunk sizes: test-clean mean/median **33.875/0 ms**, test-other **12.925/0 ms**. Utterance EOF is known in this experiment. These descriptive latency statistics have no bootstrap interval.

### Diarized transcript sanity check

Whisper small through `Speech.Transcribe`, English, AMI ES2004a's first 300 seconds: en/1 WER **16.0584%** on 411 reference words. After optimal one-to-one word-count speaker mapping, **287/344 = 83.4302%** of all Word objects have the correct maximum-overlap reference speaker, or **287/321 = 89.4081%** among eligible timed/reference-covered words. There are **29 Unassigned**, zero untimed words and zero overlap flags. This speaker measure is temporal attribution, not lexical correctness.

Timing provenance is 333 ForcedAligner, five CtcAligner and six Dtw words. The earlier all-word sanity figure was 84.3%; the current value is 83.43%, while WER remains 16.06%. The timing chain has changed since the earlier DTW-only measurement; no improvement claim is made.

### Complete-set and per-meeting diarization rows

All rates are percentages. Difference intervals and single-meeting differences are percentage points. The single-meeting 0.5-point diagnostic is descriptive; complete-set verdicts use paired meeting intervals. Pyannote/community-1 rows are information, not native-equivalence bars.

### ami — complete-set comparisons

| System | DER ours / reference % | DER difference [95% CI] pp | JER ours / reference % | JER difference [95% CI] pp | Interpretation |
|---|---|---|---|---|---|
| default (N=16) | 28.605698 / 26.996987 | +1.608711 [-0.645170, +3.608149] | 32.731323 / 31.691698 | +1.039624 [-2.895311, +5.636265] | within paired noise; qualified reference |
| nemotron (N=16) | 25.956223 / 25.956130 | +0.000093 [-0.000632, +0.001013] | 28.229055 / 28.228935 | +0.000120 [-0.000768, +0.001258] | within paired noise |
| pyannote (N=16) | 20.727578 / 17.049233 | +3.678345 [+2.154713, +5.615498] | 30.307162 / 22.187481 | +8.119681 [+4.497820, +12.028704] | floor; community-1 information only |

### ami — every meeting

| System | Recording | DER ours / reference % | JER ours / reference % | Delta DER / JER pp | Single-meeting diagnostic |
|---|---|---|---|---|---|
| default | EN2002a | 38.1571 / 37.9171 | 38.7891 / 39.5974 | +0.2400 / -0.8083 | outside 0.5 pp: JER |
| default | EN2002b | 41.1616 / 34.5341 | 53.2373 / 37.6481 | +6.6275 / +15.5893 | outside 0.5 pp: DER, JER |
| default | EN2002c | 32.8114 / 30.5655 | 32.7066 / 32.7222 | +2.2460 / -0.0156 | outside 0.5 pp: DER; one malformed oracle turn omitted |
| default | EN2002d | 36.4656 / 43.8972 | 38.4239 / 54.0351 | -7.4316 / -15.6112 | outside 0.5 pp: DER, JER |
| default | ES2004a | 28.2116 / 26.5359 | 29.6213 / 29.6337 | +1.6757 / -0.0125 | outside 0.5 pp: DER |
| default | ES2004b | 20.9156 / 18.1098 | 19.7743 / 19.3180 | +2.8059 / +0.4564 | outside 0.5 pp: DER |
| default | ES2004c | 19.2390 / 19.2674 | 19.7516 / 20.3014 | -0.0283 / -0.5498 | outside 0.5 pp: JER |
| default | ES2004d | 29.8934 / 28.3500 | 30.8881 / 32.7533 | +1.5434 / -1.8652 | outside 0.5 pp: DER, JER |
| default | IS1009a | 28.8166 / 27.4062 | 38.1760 / 41.7104 | +1.4104 / -3.5344 | outside 0.5 pp: DER, JER |
| default | IS1009b | 16.3341 / 16.6866 | 17.3990 / 17.9862 | -0.3525 / -0.5871 | outside 0.5 pp: JER |
| default | IS1009c | 15.4235 / 14.5912 | 14.2788 / 15.2765 | +0.8323 / -0.9978 | outside 0.5 pp: DER, JER |
| default | IS1009d | 24.0613 / 24.9293 | 28.7483 / 31.4794 | -0.8680 / -2.7311 | outside 0.5 pp: DER, JER |
| default | TS3003a | 30.9360 / 24.5841 | 65.1388 / 65.9223 | +6.3519 / -0.7835 | outside 0.5 pp: DER, JER |
| default | TS3003b | 29.5800 / 19.3441 | 42.8439 / 17.2337 | +10.2358 / +25.6102 | outside 0.5 pp: DER, JER |
| default | TS3003c | 24.2966 / 20.0232 | 21.6058 / 18.7358 | +4.2734 / +2.8700 | outside 0.5 pp: DER, JER |
| default | TS3003d | 33.0900 / 31.1031 | 32.3121 / 32.9713 | +1.9869 / -0.6592 | outside 0.5 pp: DER, JER |
| nemotron | EN2002a | 29.8965 / 29.8961 | 31.4741 / 31.4739 | +0.0004 / +0.0003 | within 0.5 pp |
| nemotron | EN2002b | 30.1470 / 30.1465 | 30.7041 / 30.7037 | +0.0004 / +0.0004 | within 0.5 pp |
| nemotron | EN2002c | 29.1778 / 29.1778 | 29.3575 / 29.3575 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | EN2002d | 33.2286 / 33.2286 | 34.7778 / 34.7778 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | ES2004a | 26.8965 / 26.9008 | 29.0843 / 29.0874 | -0.0043 / -0.0031 | within 0.5 pp |
| nemotron | ES2004b | 21.6238 / 21.6238 | 21.5331 / 21.5331 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | ES2004c | 20.3478 / 20.3500 | 20.2420 / 20.2445 | -0.0022 / -0.0026 | within 0.5 pp |
| nemotron | ES2004d | 22.4500 / 22.4500 | 23.5946 / 23.5946 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | IS1009a | 27.6462 / 27.6462 | 46.7131 / 46.7131 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | IS1009b | 14.9801 / 14.9801 | 15.5914 / 15.5914 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | IS1009c | 15.5322 / 15.5322 | 15.5646 / 15.5646 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | IS1009d | 19.7187 / 19.7187 | 22.7378 / 22.7378 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | TS3003a | 34.9082 / 34.9082 | 44.7680 / 44.7680 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | TS3003b | 25.8803 / 25.8742 | 25.7980 / 25.7907 | +0.0060 / +0.0073 | within 0.5 pp |
| nemotron | TS3003c | 29.9417 / 29.9422 | 29.5242 / 29.5245 | -0.0005 / -0.0003 | within 0.5 pp |
| nemotron | TS3003d | 31.4813 / 31.4813 | 30.4825 / 30.4825 | +0.0000 / +0.0000 | within 0.5 pp |
| pyannote | EN2002a | 25.2994 / 21.9055 | 31.1313 / 26.5933 | +3.3940 / +4.5380 | information, not parity |
| pyannote | EN2002b | 25.1858 / 20.7423 | 32.2769 / 26.1589 | +4.4435 / +6.1180 | information, not parity |
| pyannote | EN2002c | 19.7641 / 17.5378 | 22.7784 / 19.9288 | +2.2263 / +2.8497 | information, not parity |
| pyannote | EN2002d | 34.8360 / 23.1721 | 50.5422 / 27.7051 | +11.6639 / +22.8371 | information, not parity |
| pyannote | ES2004a | 20.6223 / 18.7569 | 25.9899 / 22.7352 | +1.8654 / +3.2547 | information, not parity |
| pyannote | ES2004b | 13.3961 / 12.6053 | 15.8318 / 14.2777 | +0.7908 / +1.5541 | information, not parity |
| pyannote | ES2004c | 13.5521 / 12.3476 | 15.7594 / 13.6303 | +1.2045 / +2.1292 | information, not parity |
| pyannote | ES2004d | 20.7726 / 18.6732 | 27.6010 / 24.3477 | +2.0994 / +3.2533 | information, not parity |
| pyannote | IS1009a | 28.0381 / 19.6477 | 53.5455 / 33.2918 | +8.3903 / +20.2537 | information, not parity |
| pyannote | IS1009b | 15.4046 / 15.1910 | 19.2789 / 15.9595 | +0.2136 / +3.3194 | information, not parity |
| pyannote | IS1009c | 12.4533 / 10.0505 | 15.2045 / 11.7915 | +2.4028 / +3.4130 | information, not parity |
| pyannote | IS1009d | 25.5250 / 16.4544 | 47.5072 / 25.4566 | +9.0705 / +22.0506 | information, not parity |
| pyannote | TS3003a | 21.4821 / 17.1439 | 59.8290 / 39.2217 | +4.3382 / +20.6074 | information, not parity |
| pyannote | TS3003b | 13.6168 / 11.9684 | 16.3609 / 13.8352 | +1.6484 / +2.5257 | information, not parity |
| pyannote | TS3003c | 15.2336 / 13.3405 | 16.7263 / 13.5044 | +1.8931 / +3.2219 | information, not parity |
| pyannote | TS3003d | 25.8165 / 21.0753 | 32.6690 / 25.9974 | +4.7412 / +6.6716 | information, not parity |

### alimeeting — complete-set comparisons

| System | DER ours / reference % | DER difference [95% CI] pp | JER ours / reference % | JER difference [95% CI] pp | Interpretation |
|---|---|---|---|---|---|
| default (N=8) | 35.487475 / 37.264385 | -1.776910 [-7.743750, +2.860047] | 39.678194 / 42.152640 | -2.474446 [-13.124693, +4.145233] | within paired noise |
| nemotron (N=8) | 17.520012 / 17.522146 | -0.002135 [-0.005671, +0.000577] | 18.115182 / 18.116630 | -0.001449 [-0.004958, +0.000871] | within paired noise |
| pyannote (N=8) | 25.079101 / 20.431073 | +4.648028 [+1.130799, +8.780060] | 31.380495 / 25.710263 | +5.670231 [+0.666037, +11.195585] | floor; community-1 information only |

### alimeeting — every meeting

| System | Recording | DER ours / reference % | JER ours / reference % | Delta DER / JER pp | Single-meeting diagnostic |
|---|---|---|---|---|---|
| default | R8001_M8004_MS801 | 41.5741 / 34.5196 | 41.1256 / 34.0168 | +7.0545 / +7.1088 | outside 0.5 pp: DER, JER |
| default | R8003_M8001_MS801 | 26.7902 / 26.8887 | 28.9546 / 28.5758 | -0.0985 / +0.3788 | within 0.5 pp |
| default | R8007_M8010_MS803 | 48.2493 / 64.5159 | 54.9748 / 83.9990 | -16.2667 / -29.0242 | outside 0.5 pp: DER, JER |
| default | R8007_M8011_MS806 | 34.7856 / 33.3411 | 48.5413 / 42.2086 | +1.4445 / +6.3327 | outside 0.5 pp: DER, JER |
| default | R8008_M8013_MS807 | 62.1732 / 60.0237 | 61.9760 / 61.6746 | +2.1495 / +0.3013 | outside 0.5 pp: DER |
| default | R8009_M8018_MS809 | 16.9636 / 17.0693 | 17.4936 / 17.9012 | -0.1057 / -0.4077 | within 0.5 pp |
| default | R8009_M8019_MS810 | 20.5389 / 20.8256 | 23.1732 / 22.2419 | -0.2868 / +0.9313 | outside 0.5 pp: JER |
| default | R8009_M8020_MS810 | 15.6915 / 15.8842 | 15.1541 / 16.6524 | -0.1927 / -1.4983 | outside 0.5 pp: JER |
| nemotron | R8001_M8004_MS801 | 18.6372 / 18.6372 | 18.0251 / 18.0251 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | R8003_M8001_MS801 | 18.3003 / 18.3003 | 18.5405 / 18.5405 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | R8007_M8010_MS803 | 21.8159 / 21.8159 | 24.4054 / 24.4054 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | R8007_M8011_MS806 | 13.1157 / 13.1136 | 14.3962 / 14.3934 | +0.0021 / +0.0027 | within 0.5 pp |
| nemotron | R8008_M8013_MS807 | 18.2221 / 18.2221 | 18.6807 / 18.6807 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | R8009_M8018_MS809 | 16.7428 / 16.7428 | 16.7129 / 16.7129 | +0.0000 / +0.0000 | within 0.5 pp |
| nemotron | R8009_M8019_MS810 | 16.3699 / 16.3793 | 16.6533 / 16.6632 | -0.0094 / -0.0099 | within 0.5 pp |
| nemotron | R8009_M8020_MS810 | 14.3181 / 14.3313 | 14.3182 / 14.3319 | -0.0132 / -0.0136 | within 0.5 pp |
| pyannote | R8001_M8004_MS801 | 26.4572 / 22.0170 | 30.4064 / 24.5484 | +4.4402 / +5.8580 | information, not parity |
| pyannote | R8003_M8001_MS801 | 20.3952 / 17.3844 | 24.1139 / 19.1369 | +3.0107 / +4.9770 | information, not parity |
| pyannote | R8007_M8010_MS803 | 37.6432 / 34.2414 | 47.6091 / 48.9949 | +3.4018 / -1.3858 | information, not parity |
| pyannote | R8007_M8011_MS806 | 28.9305 / 20.1727 | 46.2390 / 28.1000 | +8.7578 / +18.1390 | information, not parity |
| pyannote | R8008_M8013_MS807 | 25.1735 / 28.4585 | 29.8080 / 32.9240 | -3.2850 / -3.1161 | information, not parity |
| pyannote | R8009_M8018_MS809 | 14.5736 / 11.9651 | 16.6198 / 13.0970 | +2.6085 / +3.5228 | information, not parity |
| pyannote | R8009_M8019_MS810 | 26.5512 / 8.7164 | 24.3318 / 9.2037 | +17.8348 / +15.1281 | information, not parity |
| pyannote | R8009_M8020_MS810 | 10.0264 / 7.8391 | 9.8560 / 8.1313 | +2.1874 / +1.7247 | information, not parity |

## Speed

Measured 2026-11-08 using the [structural harness](https://github.com/recursive-ai-oss/onespeech-csharp/blob/63002de486786163a0a7800a51811e76d511fae6/bench/OneSpeech.Bench/README.md), measurement version 5, one repetition per row. These are our own CPU measurements, not comparisons with another implementation. All models use fp32 at the publisher revisions pinned in the catalog.

- **x64:** AMD Ryzen 9 7950X3D, 16 physical / 32 logical CPUs, AVX2/AVX512F/FMA, Linux Mint 22.3.
- **arm64:** Apple M2, 8 logical CPUs, 8 GiB RAM, AdvSimd, macOS 27.0.1.
- Both use .NET SDK 10.0.403, runtime 10.0.14, OneModel 1.2.0 and OneAudio 1.7.2; no GPU. The process CPU-count limit equals the row's thread count. Silero retains its production one-thread setting.

Each filtered row ran in an exclusive `bench` queue job bounded to 20 minutes. Models were loaded and warmed on a **different** ten-second clip before timing. Load time is separate; download, conversion and verification are excluded. End-to-end is encoded audio to Transcript using retained model instances and the production recognition, deferred alignment, clustering and merge algorithms. This is **not** cold `Speech.Transcribe` latency: that helper releases sessions between stages, while this warm-throughput harness retains them and can use more memory.

Inputs are deterministic LibriSpeech test-clean concatenations of 60, 600 and 3600 seconds, without inserted silence; clustering uses the 1049.355-second AMI ES2004a Mix-Headset recording. These constructed inputs make no quality claim. `align` is omniASR-CTC forced alignment; `diarize` is Silero + CAM++ English clustering; `live` is Moonshine streaming-tiny with 100 ms chunks; `long` is the full file pipeline with Whisper tiny on 3600 seconds.

The structural bar is end-to-end / summed graph wall time **≤1.15**. RTF is elapsed seconds / audio seconds: **≤1 means at least real time**. RSS is decimal GB, sampled every five seconds; “steady” discards the first 30 seconds but includes phase changes, so its range is not evidence of a constant-memory plateau. Short rows without steady samples fall back to the sampled peak. Live working B/chunk excludes graph-runtime allocations and immutable emitted payloads (500 measured chunks after 100 warm chunks); it is not total process allocation.

| Machine | Row | Threads | Load s | E2E s | Graph s | Ratio | Peak / steady GB | RTF | B/chunk | Bench job |
|---|---|---:|---:|---:|---:|---:|---|---:|---:|---|
| x64 | turbo60 | 1 | 9.80 | 102.73 | 101.20 | 1.015 | 13.21 / 11.94–13.21 | 1.712 | — | 008051 |
| x64 | turbo60 | 4 | 8.89 | 37.17 | 36.13 | 1.029 | 13.23 / 12.63–13.23 | 0.619 | — | 008053 |
| x64 | turbo600 | 1 | 9.53 | 799.19 | 790.48 | 1.011 | 13.60 / 11.86–13.60 | 1.332 | — | 008054 |
| x64 | turbo600 | 4 | 9.13 | 313.43 | 305.18 | 1.027 | 13.54 / 11.90–13.54 | 0.522 | — | 008061 |
| x64 | small | 1 | 7.10 | 44.06 | 42.67 | 1.032 | 9.86 / 9.12–9.86 | 0.734 | — | 008029 |
| x64 | small | 4 | 6.35 | 21.63 | 20.77 | 1.041 | 9.78 / 9.78–9.78 | 0.361 | — | 008040 |
| x64 | align | 1 | 1.88 | 34.27 | 33.99 | 1.008 | 5.02 / 4.78–4.78 | 0.571 | — | 008042 |
| x64 | align | 4 | 1.81 | 9.99 | 9.72 | 1.027 | 5.04 / 5.04–5.04 | 0.166 | — | 008044 |
| x64 | diarize | 1 | 0.14 | 70.78 | 68.76 | 1.029 | 0.47 / 0.46–0.47 | 0.067 | — | 008045 |
| x64 | diarize | 4 | 0.14 | 46.77 | 44.76 | 1.045 | 0.47 / 0.47–0.47 | 0.045 | — | 008048 |
| x64 | live | 1 | 0.62 | 21.25 | 21.14 | 1.005 | 0.74 / 0.74–0.74 | 0.354 | 0 | 008050 |
| x64 | long | 8 | 5.43 | 627.14 | 579.96 | 1.081 | 9.95 / 7.94–9.95 | 0.174 | — | 008063 |
| arm64 | small | 1 | 33.94 | 84.13 | 82.43 | 1.021 | 2.77 / 1.32–2.77 | 1.402 | — | 002027 |
| arm64 | small | 4 | 31.39 | 38.48 | 37.38 | 1.029 | 2.43 / 1.86–1.86 | 0.641 | — | 002029 |
| arm64 | align | 1 | 1.00 | 54.76 | 54.58 | 1.003 | 3.70 / 2.78–3.29 | 0.913 | — | 002030 |
| arm64 | align | 4 | 0.94 | 17.85 | 17.58 | 1.015 | 3.27 / 3.27–3.27 | 0.298 | — | 002031 |
| arm64 | diarize | 1 | 0.10 | 91.42 | 88.66 | 1.031 | 0.46 / 0.44–0.46 | 0.087 | — | 002032 |
| arm64 | diarize | 4 | 0.09 | 54.63 | 51.75 | 1.056 | 0.46 / 0.45–0.46 | 0.052 | — | 002033 |
| arm64 | live | 1 | 0.39 | 29.29 | 29.17 | 1.004 | 0.76 / 0.75–0.76 | 0.488 | 0 | 002034 |
| arm64 | long | 4 | 29.68 | 1144.50 | 1087.01 | 1.053 | 4.79 / 0.48–4.79 | 0.318 | — | 002035 |

The load-average table below reports the one-minute component before / during / after each timed call. The during interval includes the workload itself. Queue startup CPU snapshots showed no other CPU-bound user workload; desktop and OS services remained active. Large initial load averages can reflect decaying earlier activity, not concurrent CPU utilization. Full 1/5/15-minute samples and queue logs were retained with the measurement records.

| Machine | Row / threads | Load before | During min–max | Load after |
|---|---|---:|---|---:|
| x64 | turbo60 / 1 | 1.62 | 1.23–1.68 | 1.25 |
| x64 | turbo60 / 4 | 1.70 | 1.70–3.10 | 3.10 |
| x64 | turbo600 / 1 | 1.95 | 1.12–1.95 | 1.27 |
| x64 | turbo600 / 4 | 1.59 | 1.54–3.92 | 3.37 |
| x64 | small / 1 | 17.04 | 9.25–17.04 | 9.25 |
| x64 | small / 4 | 6.97 | 5.65–6.97 | 5.65 |
| x64 | align / 1 | 4.27 | 3.26–4.27 | 3.26 |
| x64 | align / 4 | 2.86 | 2.79–2.95 | 2.79 |
| x64 | diarize / 1 | 2.57 | 1.69–2.57 | 1.80 |
| x64 | diarize / 4 | 1.75 | 1.75–2.87 | 2.59 |
| x64 | live / 1 | 2.38 | 1.91–2.38 | 1.91 |
| x64 | long / 8 | 2.73 | 2.73–7.20 | 4.94 |
| arm64 | small / 1 | 2.08 | 2.07–2.55 | 2.55 |
| arm64 | small / 4 | 2.51 | 2.28–3.30 | 3.30 |
| arm64 | align / 1 | 3.01 | 2.19–3.01 | 2.19 |
| arm64 | align / 4 | 2.42 | 2.42–3.08 | 3.08 |
| arm64 | diarize / 1 | 3.15 | 2.33–3.22 | 2.33 |
| arm64 | diarize / 4 | 2.33 | 2.33–3.27 | 3.27 |
| arm64 | live / 1 | 3.27 | 2.82–3.27 | 2.92 |
| arm64 | long / 4 | 2.53 | 2.49–6.38 | 4.67 |

**Results:** every measured structural ratio passes (maximum 1.081 on x64, 1.056 on arm64). The tier-1 live default meets real time on one core on both machines, with 0 B/chunk of our working allocation. This does not imply that all file models run in real time on one core: turbo's measured RTF is 1.712 on 60 seconds and 1.332 on 600 seconds.

**Working-set qualification:** the one-hour run is not literally flat on x64. After discarding the first minute of each phase, recognition's first/last fifth means are 8.001/8.013 GB and diarization's are 9.356/9.376 GB, but alignment rises from 8.837 to 9.772 GB (+10.6%, fitted slope +6.47 MB/s). The whole-run first/last fifth means are 8.003/9.365 GB. This row therefore does **not** establish a strict flat-RSS bar; it also cannot distinguish bounded buffer-pool residency from input-length growth. On arm64, the corresponding recognition means are 0.841/0.865 GB, alignment 4.352/2.026 GB, and diarization 0.610/0.525 GB. No tuning or forced collection was used to alter these rows. These observations should not be turned into a universal constant-memory guarantee.

**Missing row:** Qwen3-ASR-1.7B was not measured. The approximately 62 GiB shared model/data tree plus its upstream and converted files would exceed the run's approximately 70 GiB storage budget. No Qwen speed claim or cross-model default re-selection is inferred from this omission. The one-hour memory rows use tiny; turbo is covered only through 600 seconds. No int8 speed row was measured here.


## Model choices and remaining evidence

- **Files:** Whisper large-v3-turbo remains the default. Its test-other WER is 5.758% in the file-model comparison; Qwen3-ASR 1.7B is 4.131%, a turbo-minus-Qwen difference of +1.63 points (paired CI [1.02, 2.31]). Qwen 1.7B is available by name for accuracy. The fixed-slice CPU comparison favoured turbo's resource requirements; the completed structural measurements and their limits are reported in the Speed section.
- **Live:** Moonshine streaming-tiny is the default. On 20-file clean/other streaming slices, its WER floor is 1.70/8.33%, with mean finalisation latency 3057/1617 ms. Streaming-small is 1.95/3.74%, 3073/1622 ms. Latency is relative to Whisper-tiny DTW word ends, not independent boundary truth. There is no vendor streaming-policy reference for these floors.
- **Language identification:** Whisper and its reference each identified 15/15 FLEURS probes (five each English/French/German). This is information only, not multilingual coverage evidence.
- **Publisher availability:** VoxCeleb1 and VoxConverse audio are no longer available from the publisher. Their intended rows are unavailable; no unofficial re-upload is substituted.
- **Gated segmentation:** authorized segmentation parity and complete-set slot measurements are reported above. The segmentation/CAM++ slot is not a community-1 reimplementation; the community-1 comparison remains information, with its quality gaps disclosed.
- **Long-form:** full long-form rows are reserved for the 1.0 evaluation. Existing single-file observations do not establish equivalence. In particular, an Earnings-22 4474955 turbo temperature-zero comparison was 55.159% versus 58.386% reference WER: outside the 0.5-point bound, in our favour because the reference appended repeated tail text. Other decoding/pipeline settings produced different scores; they must not be combined into that row. Whole-recording Qwen reference runs exceeded the CPU run bounds.
- **Quantization and speed:** full Moonshine int8 quality is reported above; full Whisper fp32 and int8 comparisons are complete and within paired noise. The Speed section reports the measured structural-overhead and live-allocation rows, including the Nemotron allocation residual and unmeasured Qwen row; it does not establish a blanket throughput guarantee.

These results do not yet demonstrate every release target. See [evaluation conventions](https://recursiveai.org/onespeech/docs/eval/) for scoring and [word timing](https://recursiveai.org/onespeech/docs/word-timing/) for failure semantics.


<!-- https://recursiveai.org/onespeech/docs/eval/ -->

# Evaluation

Speech recognition for .NET. These tools score every system with the **same** reference text and normalizer. They are not part of the runtime package. The project is unaffiliated with other products or companies named OneSpeech.

## Normalizer version 1

`en` is our English scoring normalizer, written from the descriptions in Radford et al., *Robust Speech Recognition via Large-Scale Weak Supervision*, arXiv:2212.04356, Appendix C and §4.4, and *The Open ASR Leaderboard*, arXiv:2510.06961, §2.3. It is **not** a claim of byte-exact equivalence to either project's software. Report `en/1` in every score row; changing these rules requires a new version and rescoring both systems.

| Rule | Source and concrete policy |
|---|---|
| Unicode and casing | Appendix C: NFKC, invariant lowercase; English additionally decomposes diacritics and deletes combining marks. |
| Annotations | Appendix C rules 1–2: remove matched square-bracket/parenthetical phrases. One non-nested pass; unmatched delimiters become punctuation separators. |
| Fillers | Appendix C rule 3: remove whole words `hmm mm mhm mmm uh um`. No other fillers are inferred. The leaderboard describes filler removal but does not supply a list. |
| Contractions | Appendix C rules 4–5: unify curly apostrophes, remove whitespace before apostrophes; expand `n't`, `'re`, `'ve`, `'ll`, `'m`, `'d` (would). Explicit irregular forms: can't/cannot→can not, won't→will not, shan't→shall not, ain't→is not, let's→let us. A finite list of pronoun/demonstrative `'s` forms expands to “is”. Other possessive `'s` remains a separate `s`. Informal gonna/wanna/gotta/imma expand to going to/want to/got to/i am going to. Ambiguity is resolved lexically, not from context. |
| Punctuation | Appendix C rules 6–8,11: remove digit-internal commas; punctuation/symbols become spaces (hyphens split words), except a decimal point between digits, currency prefix `$ £ €` before digits and `%` after digits. Diacritics are removed, not separated. |
| Numbers | Appendix C rule 9; leaderboard §2.3: zero through nineteen, tens through ninety, hundred, thousand/million/billion use Arabic digits. Lower units add, scales multiply the current group; “and” is consumed before a following number word. `point` plus single-digit words forms a decimal. Digit literals can take scales (`$68 million`→`$68000000`). No year/phone grouping, fractions, negative-word interpretation, or semantic number disambiguation. Unsupported words remain words. |
| Ordinals | Our finite extension of number normalization: first through twelfth, twentieth, thirtieth, hundredth; compound values take st/nd/rd/th with 11–13 exception. Other ordinal words remain words. |
| Currency | Appendix C rule 9: a numeric value followed by dollar(s), pound(s), euro(s) becomes a prefixed numeric token. No cents composition, exchange rates or regional currency inference. |
| Spelling | Appendix C rule 10: a finite explicit table, not a dictionary dependency. colour(s), favour, favourite, honour, labour, neighbour, centre(s), theatre, metre, litre, organise(d), recognise, realise, travelling, cancelled, grey, defence, licence map to their American forms. Other spellings remain unchanged. |
| Whitespace | Appendix C rule 12: collapse whitespace and trim. |

Repetitions and repairs are **preserved**. The leaderboard §2.2 discusses dataset transcriptions with those disfluencies removed; it does not specify an executable semantic repair algorithm. Deleting repeated words could conceal actual decoder errors. All finite tables above are project choices where the papers leave details unspecified, not transcriptions of a reference implementation's tables.

`basic/1`: NFKC, Unicode 17.0.0 default full case folding (CaseFolding.txt C and F mappings, no Turkic tailoring), replace Unicode M/S/P categories with spaces, collapse whitespace; retains precomposed accents. The generated case-fold table comes from the Unicode Character Database under Unicode License V3 (notice in `eval/OneSpeech.Eval/Unicode/LICENSE.txt`). CER counts Unicode scalar values after normalization, omitting whitespace; no grapheme segmentation. Use CER, rather than whitespace WER, for text without word separators. The sources describe basic multilingual normalization but do not supply language-specific linguistic rules.

## Metrics

WER = (substitutions + deletions + insertions) / reference word count. Corpus WER is the ratio of summed counts, **not** the mean of utterance rates. CER uses the same formula over scalar values. Levenshtein cost is one per edit, zero for an exact match. Ties prefer diagonal, then deletion, then insertion. Counts use linear-row memory; optional alignment traces are bounded to 16 million cells. An empty reference has undefined rate (`null`), but its insertion count still contributes to a corpus with a nonempty reference. Extra and missing IDs are errors; a failed decode must be an explicit empty hypothesis.

Bootstrap: sort utterances by ordinal ID, draw N utterances with replacement for each replicate, summing counts before dividing. Two systems use the **same** draw. Difference is first WER minus second WER, in rate units (multiply by 100 for percentage points). Default 1000 resamples, seed 1729; fewer than 1000 are refused. SplitMix64 selects an index by unsigned remainder modulo N. Sort replicates and report percentile bounds at floor(0.025×(B−1)), ceil(0.975×(B−1)). These are descriptive percentile intervals, not hypothesis-test p-values. Fewer than two utterances or any zero-length normalized reference makes CIs unavailable (avoids undefined zero-denominator resamples); the corpus score is still reported.

Verdict: `within-noise` iff the paired difference CI includes zero. Only if a CI is unavailable, absolute difference ≤0.005 (0.5 percentage points) gives `within-fallback`. Otherwise `different`; undefined corpus rates or no second system give `not-comparable`. A small difference with a computable CI excluding zero does **not** use the fallback.

Timing: align **word objects** by their normalized text with the same Levenshtein tie policy. Compare only exactly matching diagonal pairs; do not interpolate unmatched tokens or split multi-token expansions into invented timed words. Report matched, timed, missing counts; mean and median absolute start/end error in milliseconds and separate start/end fractions within inclusive 50/100 ms. Missing intervals are counted, not assigned a zero error. Timing summaries appear per utterance in JSON; no identity or temporal nearest-neighbor matching is used.

## Files

UTF-8 JSON lines, one object per utterance:

```json
{"id":"1089-134686-0000","text":"Hello world","words":[{"text":"Hello","start":0.25,"end":0.65}]}
```

IDs must be nonblank and unique. Words are optional. Times are finite nonnegative seconds on the original file clock; both endpoints must be null or both numbers, end ≥ start. JSONL lines are limited to 2 Mi characters and JSON nesting to 32. An optional first record with `"type":"header"` holds reproducibility metadata and is ignored by the scorer. Records with missing text, duplicate IDs, malformed JSON or invalid word intervals fail scoring; no records are silently dropped. Input parsers are for local evaluation artifacts, not internet endpoints.

A single version-1 `Transcript` JSON document is also accepted through the core's validated reader, with the filename stem as utterance ID. See [Transcript JSON](https://recursiveai.org/onespeech/docs/transcript-json/). It cannot stand in for multiple utterances without an explicit segmentation manifest.

Audio manifest for oracle drivers (not scoring inputs): JSONL `{"id":"...","audio":"/caller/selected/file.flac"}` in fixed order. Ranges are zero-based, start-inclusive/end-exclusive. Relative audio paths resolve from the process's working directory. Save the manifest and its hash with the row.

## Running a row

```sh
bash tools/scripts/fetch-librispeech.sh "$DATA/librispeech"
bash tools/scripts/fetch-earnings22.sh "$DATA/earnings22"
bash tools/scripts/install-oracles.sh "$ORACLES"
dotnet run --project eval/OneSpeech.Eval -- references --dataset librispeech --input "$DATA/librispeech/LibriSpeech/test-clean" --out refs.jsonl
dotnet run --project eval/OneSpeech.Eval -- references --dataset earnings22 --input "$DATA/earnings22/transcripts/nlp_references" --out earnings-refs.jsonl
```

LibriSpeech reads the text after the ID in each `*.trans.txt`. Earnings-22 reads column 1 of each pipe-separated `.nlp` row (eight columns, header required); punctuation/case metadata is not needed for normalized scoring. It also preserves the optional timing columns. Use original `nlp_references`, not force-aligned copies, so files do not duplicate IDs.

The fetch scripts verify existing archives/media without downloading them again. Text files are verified against Git blob IDs at pinned revision `c05ab6fd8b4b627d123c922a22a39e993dd37635`; SHA-256 and URL/licence records are written beside the cache. Existing hash mismatches fail instead of being overwritten. LibriSpeech is CC BY 4.0; Earnings-22 CC BY-SA 4.0, evaluation only, never committed or redistributed by these scripts. Extraction is from hash-verified archives. Delete the extraction marker to re-extract.

Oracle environments are separate `uv` projects with lockfiles and CPU torch. `run.py` uses only public APIs. Every system receives the same soundfile float32 decode, arithmetic channel mean, and librosa `soxr_hq` resampling to 16 kHz, with no amplitude normalization or ffmpeg. CPU/fp32, language English, task transcription, greedy decoding, no previous-text conditioning, and no stochastic fallback are explicit. No GPU support is offered. The output header includes the public transcribe signature defaults and DecodingOptions for OpenAI, and the model and explicitly supplied effective generation configurations for Transformers. Signature metadata is inspected without reading source or bytecode. These configurations are **not** a promise that long-form pipelines are equivalent. Transformers uses 30-second pipeline chunks; OpenAI uses its own long-form transcribe procedure. Consequently this driver is a reference sanity check, not an assertion of token identity.

The OpenAI driver calls `whisper.load_model("tiny", download_root=...)` when `--model tiny` is specified without `--checkpoint`. The released checkpoint is downloaded and verified by that public API; the driver records its SHA-256. An explicit `--checkpoint` can instead select an already downloaded release artifact. Use the HF tiny revision below for Transformers; do not substitute tiny.en.

```sh
wq run --class heavy --max 30m --cores 8 --label "onespeech reference tiny shard" -- \
  "$ORACLES/openai-whisper/bin/python" eval/drivers/run.py --oracle openai-whisper \
  --manifest manifest.jsonl --audio-root "$DATA/librispeech" --output tiny-0000.jsonl --model tiny --revision released-20250625 \
  --cache "$MODELS" --start 0 --end 100 --threads 8
wq run --class heavy --max 30m --cores 8 --label "onespeech reference transformers shard" -- \
  "$ORACLES/transformers/bin/python" eval/drivers/run.py --oracle transformers \
  --manifest manifest.jsonl --audio-root "$DATA/librispeech" --output hf-0000.jsonl --model openai/whisper-tiny \
  --revision 169d4a4341b33bc18d8881c4b69c2e104e1cc0af --cache "$MODELS" --start 0 --end 100 --threads 8
dotnet run --project eval/OneSpeech.Eval -- score --ref refs.jsonl --hyp tiny.jsonl --hyp2 hf.jsonl --normalizer en --resamples 1000 --seed 1729 --out scores.json
```

Repeat disjoint ranges through 2620 for the complete test-clean set; concatenate records (headers are ignored by the scorer). Choose smaller shards if needed to remain under the job bound. Each output header records versions, settings, range and manifest hash; completed utterances are flushed and fsynced. Resume checks the exact header and IDs. A truncated final JSON line fails rather than silently skipping it: remove only that incomplete line before resuming. Keep all shard headers with the row. The faster-whisper variant uses `--oracle faster-whisper` and a pinned compatible publisher checkpoint, never assumes HF safetensors are CTranslate2 weights. Fetch scripts do not download models.

Output JSON includes per-utterance WER/CER counts, timing summaries, corpus counts, bootstrap bounds and verdict; rates are fractions, console output is percent. Keep model weight hashes, dataset hashes, oracle locks, machine CPU/ISA/core count, load, thread count, SDK/runtime and full command/job IDs with measured rows. These tools alone do not constitute a measured quality claim.

### Fixed targeted slice

`eval/manifests/librispeech-test-clean-first100.jsonl` pins the first 100 test-clean utterance IDs in ordinal order, their relative audio paths and each FLAC's SHA-256. It contains metadata only, not audio or reference text. Regenerate it from the verified cache with `eval/drivers/prepare_slice.py`; that script also writes a caller-selected reference JSONL outside the repository. Pass the dataset cache as `--audio-root` and use `--start 0 --end 100`. The driver checks manifest audio hashes before decoding. This small sanity check does not replace full-corpus evaluation.

The fixed slice was checked with openai-whisper 20250625 (`tiny`, original MIT release artifact) and Transformers 5.22.2 (`openai/whisper-tiny`, revision `169d4a4341b33bc18d8881c4b69c2e104e1cc0af`). Both used fp32 CPU, greedy decoding, temperature 0 and English. Original `tiny.pt` SHA-256: `65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9`; it is a different artifact from the HF safetensors. With `en/1`, 1000 paired bootstrap resamples and seed 1729:

| System | Errors / reference words | WER | 95% CI |
|---|---:|---:|---:|
| OpenAI released tiny | 143 / 2357 | 6.067% | [4.932%, 7.300%] |
| HF tiny | 141 / 2357 | 5.982% | [4.852%, 7.231%] |

Paired difference (OpenAI minus HF): **0.085 percentage points**, 95% CI **[−0.626, 1.077]**, verdict **within-noise**. This is a small correctness check, not a full-corpus quality claim or a performance comparison. The run used Linux x64 on an AMD Ryzen 9 7950X3D (16 cores / 32 logical CPUs), eight oracle threads, torch 2.15.0+cpu, numpy 2.5.4, soundfile 0.13.1 and librosa 0.11.0; no GPU. Full per-utterance hypotheses and resolved decoding headers are evaluation artifacts, not embedded library data.

## Diarization scoring

```
onespeech-eval der --ref reference.rttm --hyp hypothesis.rttm --out der.json
```

The `der` command implements the definitions in the *Third DIHARD Challenge Evaluation Plan*, version 1.2 (2020-10-30), §4 and Appendix C. This is a scorer specification reference only: no DIHARD data is fetched. Rates in JSON are fractions, not percentages. Output includes reference speaker-seconds, miss, false alarm, confusion, reference speaker count, DER and JER. No collar is applied and all overlap is scored. Scoring rejects endpoints beyond `TimeSpan.MaxValue.TotalSeconds`, even if a syntactically valid RTTM contains such numbers; this avoids overflowing assignment arithmetic on impossible audio clocks. These definitions are implemented independently, with hand-computed tests; an external released-scorer cross-check is still pending.

Each recording/channel is scored separately with independent, optimal one-to-one speaker mappings; speaker names have no significance across recordings. Corpus DER sums error counts before dividing by total reference speaker-seconds. Duplicate or overlapping segments of the same speaker are unioned. For each constant-activity interval of duration d, with R reference speakers, H hypothesis speakers and C correctly mapped concurrent speakers:

* miss = d × max(0, R−H);
* false alarm = d × max(0, H−R);
* confusion = d × (min(R,H)−C).

The DER mapping maximizes total pairwise intersection duration via Hungarian assignment. Reference speaker-time includes multiplicity in overlap. DER can exceed 100%. Empty reference speech has undefined DER (`null`), but false-alarm seconds are retained.

JER uses a separate Hungarian assignment maximizing the sum of per-reference-speaker intersection-over-union. An unmatched reference speaker contributes error 1. JER is the reference-speaker-macro mean, including across recordings; unmatched hypothesis speakers do not add terms of their own (but can affect the pairing/union). No reference speakers means undefined JER (`null`). Maximum 512 distinct speakers per recording/channel; more is refused rather than allocating an unbounded assignment matrix.

The scored domain is the union of all reference and hypothesis intervals, including hypothesis-only files/channels, which count as false alarms. There is currently **no UEM support**: do not use this command for a benchmark with excluded or restricted scoring regions without first clipping both sides to those regions. Missing hypothesis files are treated as empty, never dropped from corpus denominators. Pure silence outside all listed turns contributes zero.

RTTM input is strict UTF-8 and accepts blank lines, `#` comment lines and ten-field `SPEAKER` records. Fields are type, recording ID, positive integer channel, onset seconds, duration seconds, orthography, speaker type, speaker ID, confidence and lookahead. Onset must be finite and nonnegative; duration positive, finite and distinguishable from zero when added to onset. Unsupported record types, malformed fields, invalid UTF-8, lines exceeding 16,384 UTF-16 code units or more than one million segments are rejected. The confidence/lookahead/orthography/type placeholders are not used by scoring. Segments of the same speaker are treated as a set even when duplicated.

Either input may instead be a version-1 `Transcript` JSON file (extension `.json`). Assigned turns become RTTM-style channel-1 segments, ignoring turn text and words; `Unassigned` turns are omitted. Set `--recording meeting-id` to identify the transcript when comparing JSON to RTTM. Default ID is `recording`; JSON-vs-JSON inputs therefore share a clock by default. A transcript has no channel field, so a multi-channel RTTM must be converted explicitly rather than silently conflated.

`fuzz/run-rttm-smoke.sh <absolute-scratch-directory> [seconds]` exercises the bounded RTTM reader through SharpFuzz/libFuzzer, with the same pinned development tools as the other reader smoke scripts; maximum duration 300 seconds.

## CAM++ diarization development and floor runner

`onespeech-eval diarize` runs the fixed meeting slices with a SHA-verified converted CAM++ English graph. Paths below are caller-selected caches/result directories outside git. Fetch/prepare datasets first with the scripts in `tools/scripts/`; `fetch-ami-dev-channels.sh` adds only the four development headset files for channel-layout calibration.

```
onespeech-eval diarize dev GRAPH AMI_ROOT ES2011a RESULTS
onespeech-eval diarize dev GRAPH AMI_ROOT ES2011b RESULTS
onespeech-eval diarize select RESULTS
onespeech-eval diarize channels AMI_ROOT RESULTS
onespeech-eval diarize holdout GRAPH AMI_ROOT ES2004a RESULTS S128 A64-0.15
onespeech-eval diarize holdout GRAPH AMI_ROOT ES2004b RESULTS S128 A64-0.15
onespeech-eval diarize holdout GRAPH ALI_ROOT R8001_M8004_MS801 RESULTS S128 A64-0.15
onespeech-eval diarize holdout GRAPH ALI_ROOT R8003_M8001_MS801 RESULTS S128 A64-0.15
onespeech-eval diarize summary RESULTS
onespeech-eval diarize excerpt GRAPH AMI_ROOT ES2004a RESULTS/excerpt S128
```

Development commands refuse other IDs or non-development manifest entries. Selection requires the complete 57-profile grid on both development meetings, minimizing pooled DER, then speaker-macro JER, then ordinal profile ID. Freeze `selection.json` and `channel-selection.json` by SHA-256 before running holdouts; do not choose parameters from holdout scores. The example profile IDs are the measured selection in [diarization](https://recursiveai.org/onespeech/docs/diarization/), not an instruction to select them using a new holdout set. The holdout command accepts only the four named held-out IDs and explicitly supplied profile names. Completed meeting score files are not silently overwritten; a new experiment uses a new result root. Per-meeting completion makes runs resumable without combining partial meetings.

Only the first profile for a meeting runs embedding/VAD inference; subsequent clustering profiles replay in-RAM outputs after verifying identical PCM hashes and sequence lengths. Model weights are never fitted, caches are discarded at the meeting boundary, and no embeddings are written. All rows record input/reference/manifest SHA-256, checkpoint revision, converted SHA, precision, threads, VAD settings, clustering settings, reference speaker-seconds, miss/FA/confusion, DER/JER, speaker/window counts and hypothesis RTTM path. No collar; overlaps scored; no text normalizer. `summary` pools error counts rather than averaging meeting DER and reports reference-speaker-macro JER. Its descriptive 95% intervals resample the two meeting rows 1,000 times with seeded .NET Random(1729), with floor/ceiling percentile indices as in the scoring method; they are not intervals over a new independent speaker population. Individual-meeting CIs are unavailable.

Channel calibration uses ten correlated layout proxies from AMI dev headset 0/1: physical separated microphones and 1% statistical leakage are positives, 25%/50% leakage and dual mono negatives. For original per-frame energies X,Y and cross-energy C, symmetric leakage alpha gives energies `(X+2*alpha*C+alpha²*Y)/(1+alpha)²` and its channel-swapped counterpart, and cross-energy `(alpha*(X+Y)+(1+alpha²)*C)/(1+alpha)²`. No modified speech waveform is generated. Search 10/15/20/25/30 dB and normalized-dot limits 0.2/0.3/0.4/0.5/0.6; minimize balanced error, then distance from the initial 20 dB/0.3, then higher dB/lower dot limit. Three dominant frames and energy floor1e-8 remain fixed. This proxy fit is not a telephony accuracy estimate.

The excerpt command uses the real diarizer but a **generated word-timing fixture**, because ASR integration is separate. It saves a ≤300-second derived WAV in the dataset cache and emits a Transcript JSON plus an explicit fixture summary, including assigned/unassigned counts and JSON timing quantization. It is not an ASR hypothesis, WER or word-alignment evaluation. See [diarization](https://recursiveai.org/onespeech/docs/diarization/) for measured floors and limitations.

`tools/oracles/speaker-modelscope/diarize.py` probes the released ModelScope public pipeline using only the approved English checkpoint already present locally, on CPU, with network disabled. It never opens installed implementation files or prints dependency exception frames. A registry failure or need for unchecked auxiliary models is reported as unavailable, not as a scored reference. The existing locked oracle environment supplies dependencies; no repository recipe is executed.


<!-- https://recursiveai.org/onespeech/docs/model-data-readers/ -->

# Model-data and document readers

This is the inventory of file-reading code owned by OneSpeech. The mutation targets are development tools; [measured results](https://recursiveai.org/onespeech/docs/results/) report the completed campaigns separately.

| Reader | Model-supplied or caller-supplied data | Mutation target |
|---|---|---|
| Transcript JSON | Versioned document, including nested model metadata, words and turns | `transcript` |
| Catalog JSON | Model entries, files, capabilities, licence and tier metadata | `catalog` |
| Audio input adapter / G.711 WAV | Encoded stream headers and bounded decode reads | `wav`, `audio` |
| Tensor checkpoint | ZIP directory, inert pickle tensor index, storage views | `checkpoint`, `pickle` |
| Safetensors | Header JSON, names, shapes, dtypes, offsets and payload reads | `safetensors` |
| SentencePiece character vocabulary | Bounded protobuf model data | `sentencepiece` |
| Moonshine tokenizer / configuration | BPE JSON and offline/streaming configuration JSON | `moonshine` |
| Parakeet vocabulary / phrase tokenizer | Tokenizer JSON, merge data, precompiled normalization map | `parakeet` |
| Whisper tokenizer | `vocab.json`, `added_tokens.json`, UTF-8 `merges.txt` | `whisper-tokenizer` |
| Qwen tokenizer | `vocab.json`, `tokenizer_config.json`, UTF-8 `merges.txt` | `qwen-tokenizer` |
| Whisper configuration / generation controls | `config.json`; `generation_config.json` token IDs, language IDs, suppression and alignment heads | `whisper-config`, `whisper-generation` |
| Qwen ASR / forced-aligner configuration | Nested audio/text `config.json` | `qwen-config` |
| Nemotron ASR prompt configuration | `processor_config.json` language-to-prompt dictionary | `nemotron-processor` |
| Converter provenance | Bounded `upstream.txt`, known publisher revision and digest records | `provenance` |
| RTTM | Speaker intervals read by the evaluation tool, not core | `rttm` |

Whisper and Qwen byte vocabularies must contain the 256 byte symbols. IDs and file sizes are bounded: Whisper uses fewer than 60,000 IDs and at most 8 MiB per tokenizer file; Qwen uses fewer than 200,000 IDs and at most 16 MiB per file. A nonempty special-token spelling is required so token recognition always advances. Whisper's unused ordinary empty token is allowed and decodes to empty text. Merge references must be consistent with the declared vocabulary.

For the Whisper and Qwen tokenizer/configuration readers, syntactically malformed JSON throws `JsonException`. Incorrect JSON shapes, missing required properties, invalid value types and inconsistent token/configuration data are refused with `InvalidDataException`, rather than leaking JSON getter or collection exceptions. Provenance, integrity checks and the Nemotron processor reader use `ModelIntegrityException`; an unsupported Moonshine checkpoint retains its typed capability refusal. Filesystem failures remain I/O errors. Cancellation and public tokenizer text/ID argument errors retain their documented behavior.

Generation metadata token IDs must fit the configured vocabulary; alignment pairs must fit decoder layers/heads. Language entries are limited to 256, suppression entries to the vocabulary size, and selected heads to the decoder's layer/head product. The existing 1 MiB JSON bound also applies.

The two byte-BPE targets mutate each component file against deterministic synthetic companion files, then exercise encoding and decoding. Configuration targets parse without loading model graphs. The provenance target exercises Whisper, Moonshine and both Qwen roles; companion files intentionally stop a valid provenance record before neural graph loading. No checkpoint is downloaded or executed by these targets.

OneModel-format/ONNX graph parsing belongs to OneModel. Codec and container internals belong to OneAudio and OneMedia; the audio targets instrument OneSpeech's input adapter, not those dependencies. Caption imports consume OneMedia cue objects rather than implementing another subtitle-file parser. Binary speaker embeddings are not a library persistence format. The library does not read or persist a speaker-enrolment database.


<!-- https://recursiveai.org/onespeech/docs/safetensors-reader/ -->

# Internal safetensors reader

The internal `OneSpeech.Formats.SafeTensorFile` reads the published safetensors format (v0.6.2 format description). It does not load executable objects. A readable, seekable stream is required. Construction starts at file offset zero and owns the stream unless `leaveOpen` is true. Instances are not thread-safe; the underlying stream must not change during use.

Before allocating a header buffer, check the unsigned little-endian 64-bit header size against the remaining file and the 8 MiB header limit. The first header byte must be `{`. Parse strict UTF-8 JSON with depth at most 8; duplicate entries and duplicate tensor fields are rejected. Metadata must be a string-to-string map. Names and property strings have a 1024-byte encoded bound; metadata values have a 64 KiB bound. There are at most 50,000 root entries and at most 50,000 metadata keys, and tensor rank is at most 16.

Every tensor has exactly `dtype`, `shape` and `data_offsets`. Dimensions and offsets are nonnegative signed 64-bit integers. Scalar and empty tensors are allowed. Shape products use checked arithmetic; byte length must equal element count times dtype size. Sorted ranges must cover the payload exactly, with no overlaps, holes, out-of-file ranges or trailing unindexed bytes. Unsupported dtypes fail explicitly. Supported byte widths are BOOL/I8/U8, F16/BF16/I16/U16, F32/I32/U32 and F64/I64/U64; no sub-byte or float8 types are accepted.

`CopyTensorTo` copies original bytes with a 64 KiB buffer. `ReadFloat32` materializes F32/F16/BF16 only and refuses more than 134,217,728 elements (512 MiB output), checked before allocation. Conversion widens values; it does not calibrate, quantize or modify weights. Tensor NaNs and infinities remain data. Cancellation is checked between copy/read blocks. Header validation errors are `InvalidDataException`; unsupported dtypes are `NotSupportedException`. File-system and stream errors remain I/O errors.

`fuzz/OneSpeech.SafeTensors.Fuzz` generates its own corpus from synthetic headers and tests parsing and bounded float reads. `fuzz/run-safetensors-smoke.sh` runs one bounded libFuzzer process and puts binaries, tools, generated seeds and findings in the supplied scratch directory.


<!-- https://recursiveai.org/onespeech/docs/third-party-notices/ -->

# Third-party notices

## Model weights

Core embeds the Silero VAD v6.2.3 ONNX release model. No other model weights are included.

## Data

The LibriSpeech clip identified below is the only third-party audio fixture; ID-only LibriSpeech evaluation lists are also included, with attribution below. Other feature fixtures use deterministic synthetic signals.

## Libraries

Package dependencies retain their own licences and notices.

## Silero VAD v6.2.3

The embedded silero_vad.onnx release model is distributed under the following licence.

Upstream licence: https://github.com/snakers4/silero-vad/blob/5cd7945676eb32225748052e2e6a0580e4686a08/LICENSE

Model SHA-256: `1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3`.

MIT License

Copyright (c) 2020-present Silero Team

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

## Downloaded Whisper checkpoints

Whisper weights are not embedded. First use downloads the pinned publisher files and
converts them locally. The five catalog entries retain the original OpenAI MIT notice,
including "Copyright (c) 2022 OpenAI", in the embedded catalog attribution. The pinned
Hugging Face cards declare Apache-2.0 for tiny, base, small and large-v3, and MIT for
large-v3-turbo. The Apache-2.0 text is included in LICENSE-APACHE. Source repositories,
revisions and file digests are recorded in the catalog and model provenance.

## Optional speaker checkpoints

The core and tool packages do not include the following publisher checkpoints.
`RecursiveAI.OneSpeech.Models.CamPlusEnglish` includes the pinned English CAM++
publisher files below; without that package they are downloaded and converted on
first use of the default diarizer. The other entries are pinned by the development fetch tool. The model
cards declare Apache-2.0; the licence text is included in LICENSE-APACHE. These are published
pretrained weights, not models trained or fine-tuned by this project. The training-data statements
below are the publishers' descriptions, not an independent audit or a licence grant for that data.

- SpeechBrain ECAPA-TDNN, `speechbrain/spkrec-ecapa-voxceleb` revision
  `0f99f2d0ebe89ac095bcc5903c4dd8f72b367286`, `embedding_model.ckpt`,
  SHA-256 `0575cb64845e6b9a10db9bcb74d5ac32b326b8dc90352671d345e2ee3d0126a2`.
  Source: https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb .
  Publisher-stated training data: VoxCeleb1 and VoxCeleb2.
- CAM++ English — Hui Wang, Siqi Zheng, Yafeng Chen, Luyao Cheng and Qian Chen,
  published by 3D-Speaker/ModelScope iic under Apache-2.0.
  `iic/speech_campplus_sv_en_voxceleb_16k` revision
  `032b8131a7ad812f87061955ca974c99060c5a03`, `campplus_voxceleb.bin`,
  SHA-256 `5b1a88b6f8d85826fabef804779c3372b42f3af21457fa48bd5c097c0686b2de`.
  Source: https://modelscope.cn/models/iic/speech_campplus_sv_en_voxceleb_16k .
  Publisher-stated training data: VoxCeleb2, 5,994 speakers.
- CAM++ Chinese common, `iic/speech_campplus_sv_zh-cn_16k-common` revision
  `013c18884dffb82846cf472dd06dd4a08d97e2cd`, `campplus_cn_common.bin`,
  SHA-256 `3388cf5fd3493c9ac9c69851d8e7a8badcfb4f3dc631020c4961371646d5ada8`.
  Source: https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common .
  Publisher-stated training data: a large Chinese speaker dataset with over 100,000 speakers;
  the card does not identify the corpus by name.
- ERes2NetV2 Chinese common, `iic/speech_eres2netv2_sv_zh-cn_16k-common` revision
  `1cf80d41fb3435bd3d8df185b5c423333b2db42a`, `pretrained_eres2netv2.ckpt`,
  SHA-256 `0eb4057106b2573dd7b132cf0c36273ab29afd192c1610f80baa9c556dbb963c`.
  Source: https://modelscope.cn/models/iic/speech_eres2netv2_sv_zh-cn_16k-common .
  Publisher-stated training data: a large Chinese speaker dataset with approximately 200,000
  speakers; the card does not identify the corpus by name.

## LibriSpeech evaluation identifiers

`eval/manifests/librispeech-speaker-trials.tsv` and
`eval/manifests/librispeech-speaker-parity50.txt` contain utterance identifiers derived from
LibriSpeech test-clean/test-other (OpenSLR resource 12), by Vassil Panayotov, Guoguo Chen,
Daniel Povey and Sanjeev Khudanpur (2015).

Source: https://www.openslr.org/12/ . Dataset licence: Creative Commons Attribution 4.0
International, https://creativecommons.org/licenses/by/4.0/ .

OneSpeech selected and paired these identifiers for evaluation; no original audio or transcript
text is included in these files. The selection, pairing method and seed are documented in
`docs/speakers.md`. This attribution does not imply endorsement by the dataset authors.

## Reference observations

The tone feature fixtures were produced from deterministic synthetic audio by black-box calls to openai-whisper 20250625 (MIT). They contain only numerical outputs, not model weights, source code or third-party audio. The oracle harness also supports transformers 5.22.2 (Apache-2.0). Neither oracle is a shipped dependency.

## LibriSpeech fixture

`testdata/1089-134686-0001.flac` is from the LibriSpeech ASR corpus (2015), Vassil Panayotov, Guoguo Chen, Daniel Povey and Sanjeev Khudanpur. Test-clean utterance 1089-134686-0001 is distributed unmodified under Creative Commons Attribution 4.0: <https://creativecommons.org/licenses/by/4.0/>. Source: <https://www.openslr.org/12>. The archive path and SHA-256 are recorded in `testdata/manifest.json`.

### ECAPA reference-only auxiliary files

The pinned `speechbrain/spkrec-ecapa-voxceleb` reference also initializes `classifier.ckpt`
(5,534,328 bytes, SHA-256 `fd9e3634fe68bd0a427c95e354c0c677374f62b3f434e45b78599950d860d535`)
and `mean_var_norm_emb.ckpt` (1,921 bytes, SHA-256
`cd70225b05b37be64fc5a95e24395d804231d43f74b2e1e5a513db7b69b34c33`). They are under the same
publisher-declared Apache-2.0 licence at revision `0f99f2d0ebe89ac095bcc5903c4dd8f72b367286`.
These files are used only by the separate evaluation oracle; no classifier or global-embedding
normalization weights are included in OneSpeech's converted embedding graph or package.

## Qwen3-ASR and Qwen3-ForcedAligner

Qwen Team, Qwen3-ASR and Qwen3-ForcedAligner. The published model cards declare
Apache License 2.0. The full licence is provided in LICENSE-APACHE. Model data
and converted weights are downloaded separately, not embedded in this package.
Pinned checkpoint revisions and file digests are in the model catalog; model
provenance and attribution are retained in the returned ModelInfo.

## Omnilingual ASR checkpoint and character vocabulary

Copyright 2025 Meta Platforms, Inc. and affiliates.

The optional `facebook/omniASR-CTC-300M` checkpoint and vocabulary are published
by Meta AI / the Omnilingual ASR Team under Apache License 2.0, reproduced in
`LICENSE-APACHE`. Source:
https://huggingface.co/facebook/omniASR-CTC-300M/tree/8e35f0cc28fa6099e0c14d56db85ce0423baa691

OneSpeech converts the published tensors to its runtime format without training
or fine-tuning. Weights are downloaded separately; they are not embedded in the
core package. Retain the licence and these attribution notices when
redistributing the model or converted weights.

## Moonshine English speech recognition weights

The tiny, base, streaming-tiny, streaming-small and streaming-medium checkpoints
from moonshine-ai are MIT-licensed. Their pinned model cards and source links
are in the model catalog. The optional `.Models.MoonshineTiny`, `.MoonshineBase`
and `.MoonshineStreamingTiny` packages include their respective pinned publisher
files (weights, configuration and tokenizer data). These are published pretrained
weights, not models trained or fine-tuned by this project. Publisher-stated training data: tiny/base, 200,000 hours of internet audio and
open datasets; streaming-tiny, roughly 300,000 hours comprising those sources
plus 100,000 hours of internally prepared speech data. These descriptions are
not an independent audit or a licence grant for the training data. No Community License checkpoints are included.
Publisher licence: https://github.com/moonshine-ai/moonshine/blob/main/LICENSE

MIT License

Copyright (c) 2025 Useful Sensors, Inc. (dba Moonshine AI)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

## OpenMDW agreement text and Nemotron 3.5 ASR

The OpenMDW-1.1 agreement text is from https://openmdw.ai/license/1-1/;
copyright The Linux Foundation and its contributors. The site's contents are
licensed under CC-BY-4.0. The agreement is copied without substantive changes in
`docs/licenses/OpenMDW-1.1.txt` and in the model's provenance metadata. Model
rights are governed by OpenMDW-1.1, not the site's content licence.

NVIDIA Nemotron 3.5 ASR weights are loaded only by explicit name from the pinned
publisher revision and converted in the user's cache. They are not embedded,
mirrored, or redistributed by OneSpeech. The full agreement and NVIDIA origin
notice are retained in `ModelInfo`.

## NVIDIA Nemotron 3 Diarization (optional publisher download)

`nvidia/Nemotron-3-Diarization` revision
`f667ed73aee57d40cc39428eb768b4fd87a0a29e` is governed by the
[OpenMDW License Agreement, version 1.1](https://openmdw.ai/license/1-1/).
No weights or converted graphs are redistributed by this package. Explicit loading
fetches the pinned publisher file into the user's cache and constructs runtime
artifacts locally. `ModelInfo` retains NVIDIA's model origin and the full agreement.

## Synthetic contextual-tokenizer fixture

`parakeet-phrase-tokenizer-reference.json` contains outputs we generated from
synthetic text using the released tokenizers 0.23.3 public API (Apache-2.0) and
NVIDIA's pinned Parakeet tokenizer data. It contains no model weights or copied
tokenizer vocabulary. Source: NVIDIA, parakeet-tdt-0.6b-v3,
https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3 at revision
541d1f99c6b0c3cd0b11a95167540bb8edefd82b. Model licence: CC-BY-4.0,
https://creativecommons.org/licenses/by/4.0/. This notice identifies the fixture's
provenance; tier-2 models are still fetched only from the publisher and are never
bundled. The fixture generator and input texts are ours.


<!-- https://recursiveai.org/onespeech/docs/changelog/ -->

# Changelog

All notable changes are documented here, following Keep a Changelog.

## [1.0.0] - 2026-11-10

First release. Measured results and limits: [docs/results.md](https://recursiveai.org/onespeech/docs/results/).

### Highlights
- **Transcription** with four model families behind one seam: Whisper (tiny to large-v3 and large-v3-turbo, the default), Qwen3-ASR 0.6B and 1.7B, FastConformer (Parakeet-TDT-0.6B-v3 and Nemotron 3.5 ASR streaming, by name), and Moonshine (the live default). VAD-first long-file chunking in constant memory, language identification, optional mixed-int8 variants.
- **Word timing** from the best available method in a fixed order (native, Qwen3-ForcedAligner, omniASR-CTC, DTW), recorded per word; public `Speech.Align` for any text and audio.
- **Diarization**: a tier-1 default of Silero VAD, CAM++ embeddings and spectral/AHC clustering with resegmentation, merged into the transcript. The default does **not** detect overlapping speech. Tier-2 upgrades, loaded by name from the publisher: Nemotron 3 Diarization (overlap-aware, up to eight speakers) and the gated pyannote segmentation-3.0 slot.
- **Speaker recognition**: embeddings (CAM++, ECAPA-TDNN), enrolment, identification and verification with calibrated thresholds; in-memory only, never persisted.
- **Live sessions** with one event model (`Partial`, `Final`, `Endpoint`, `SpeakerTurn`) on one audio clock; true streaming partials on Moonshine and Nemotron 3.5 ASR, utterance mode for the other families.
- **Contextual biasing** through `WithPhrases` where the model supports it; other models refuse with a typed error.
- **Packages**: `RecursiveAI.OneSpeech`, `.Media` (containers, SRT/VTT/ASS/TTML, caption muxing), `.Cli` (the `onespeech` tool), and offline weight packages `.Models.CamPlusEnglish`, `.Models.MoonshineTiny`, `.Models.MoonshineBase`, `.Models.MoonshineStreamingTiny`. NativeAOT and trim clean; no native code.
- **Models**: tier-1 (MIT, Apache-2.0) models download on first use; tier-2 (CC-BY-4.0, OpenMDW-1.1, gated) models load only by name from the publisher's files and are never redistributed.

Changes since the last preview follow.

### Added
- The gated pyannote segmentation-3.0 model in the segmentation slot, with an overlap-aware diarizer over it (`PyannoteDiarizer`), loaded from publisher files the user is entitled to; never a default.
- Mixed-int8 variants of Whisper large-v3-turbo, Whisper small and Moonshine streaming-tiny, selected by model ID (`-int8`) and converted locally from the same pinned weights; defaults remain fp32.
- Contextual phrase biasing through `SpeechRecognizer.WithPhrases`: Whisper prompt conditioning, Qwen3-ASR context, and shallow fusion over a phrase trie for Parakeet and custom CTC/RNN-T/TDT models. Families that cannot bias refuse with `CapabilityNotSupportedException`.
- `Speech.Enroll`, `Speech.Identify`, `Speech.Verify` and `Speech.ClearSpeakers`, with enrolled names on transcript speakers.
- `Speech.PrefetchAsync(language, model)` fetches every model a call would need.
- Path-input `*Async` overloads for every verb, and stream input for `TranscribeStream` and `DetectLanguageAsync`.
- Uniform `Load(ResolvedModel, ...)` and `LoadAsync(store, model, ...)` on every downloadable family.
- `ModelUnavailableException`, and `ModelId` on model errors.
- `TranscriptJsonConverter`, so plain System.Text.Json writes the documented schema-1 shape; value equality for `Transcript`, `Segment`, `Turn` and `ModelInfo`.

### Changed
- Requires RecursiveAI.OneModel 1.2.0.
- Faster CAM++ feature extraction with identical output.

### Fixed
- `Transcript.ToJson` and `FromJson` handle recordings up to 24 hours (documented limits, checked before allocating).
- One language-tag rule for every entry point; unsupported languages are refused, and `Align` no longer returns untimed words when no aligner applies.
- The role base classes implement `IDisposable`. Wrappers and sessions borrow, and never dispose, the models they use.
- `CreateLiveSession` accepts any `VoiceActivityDetector`.
- Consistent optional `language` and `cancellationToken` parameters, with the token last everywhere; `MediaInput.TranscribeStreamAsync` is renamed `TranscribeStream`.
- `WhisperOptions.Progress` is `IProgress<TimeSpan>`; `MoonshineOptions.IntraOpThreads` is renamed `Threads`; thread counts default to 1.
- Invalid in-memory PCM throws `ArgumentException`.
- The Silero VAD provenance ID is the catalog ID `silero/vad-v6.2.3`.

## [0.4.0-preview.1]

### Added
- Qwen3-ASR 0.6B and 1.7B recognition, language identification and context biasing, and Qwen3-ForcedAligner word timing.
- CTC forced alignment with omniASR-CTC-300M and public `Speech.Align` for any text and audio.
- Moonshine recognition with true streaming partials, incremental decoding with cached attention, and live sessions that keep up in real time on one core with no steady-state allocation in OneSpeech's own code per audio chunk.
- Online speaker clustering for live sessions.
- Parakeet-TDT-0.6B-v3 and Nemotron 3 Diarization by name, loaded from the publisher's files and converted locally. They are never defaults and are never redistributed. Nemotron 3 Diarization detects overlapping speech.
- Gated publisher models: a user token is sent only to the publisher's own file URLs, with typed errors for missing tokens, rejected tokens and gates not yet accepted.
- Custom CTC and RNN-T/TDT models of a known shape, decoded by the public `Decoding` primitives with native word timing.
- NeMo mel features.
- `RecursiveAI.OneSpeech.Media`: audio from any container OneMedia opens, caption export (`ToSrt`, `ToVtt`, `ToAss`, `ToTtml`) and caption muxing.
- Whisper large-v3.

### Changed
- Requires RecursiveAI.OneModel 1.2.0-preview.2.
- Qwen3 BF16 graphs are converted only when requested.

## [0.2.0-preview.1]

### Added
- Live sessions with one event model (`PartialEvent`, `FinalEvent`, `EndpointEvent`, `SpeakerTurnEvent`): finals are immutable, partials carry a revision, and all events share one audio clock. `Speech.Listen` and `Speech.TranscribeStream`; utterance mode (voice-activity endpointed finals) for offline recognizers.
- Chinese speech uses the Chinese CAM++ speaker embedder for diarization; other languages use the English one.
- A diarization speech-activity profile fitted on development meetings.

### Fixed
- Transcript JSON writes times exactly at 100-ns resolution instead of rounding through binary floating point.

## [0.1.0-preview.1]

### Added
- Shared speech contracts, the `Transcript` document model and its versioned, bounded JSON form.
- `Speech.Transcribe`, `TranscribeAsync` and `DetectLanguage`: transcription of WAV, MP3, FLAC, Ogg, M4A/3GP/ADTS and AMR files, returning words with timings, confidence and timing method, segments, language and speaker turns.
- Whisper recognition (tiny, base, small, large-v3-turbo) with greedy and beam decoding, temperature fallback, VAD-first long-form chunking, cross-attention word timing and language identification.
- Embedded Silero VAD v6.2.3.
- Speaker embeddings (CAM++ English and Chinese, ECAPA-TDNN), an in-memory speaker registry, and clustering diarization (spectral and agglomerative) merged into transcripts, including channel diarization for separated-party stereo. The clustering diarizer does not detect overlapping speech.
- A pinned model catalog with verified download, local conversion on first use, a per-user cache, the `ModelSource` hook and `onespeech models pull`.

### Known limitations
- Whisper large-v3 was refused with a typed error in this release (fixed in 0.4.0-preview.1).