Build a custom speech recognition pipeline in C#

Choose the recognition, speaker-labeling and word-timing stages yourself.

dotnet add package RecursiveAI.OneSpeech

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

sample.flac is included in the example zip.

using OneSpeech.Moonshine;
using OneSpeech.Vad;
using OneSpeech.Pipeline;
 
using var recognizer =
    await MoonshineRecognizer.LoadAsync(
        model:
        "moonshine-ai/moonshine-tiny");
using var vad = new SileroVad();
var pipeline =
    new VadTranscriptionPipeline(
        recognizer, vad);
Console.WriteLine(pipeline.Transcribe(
    "sample.flac", "en").Text);

First use downloads about 0.11 GB of Moonshine tiny publisher files and caches them and locally converted graphs in ~/.cache/onespeech/models on Linux (change the cache location).

Duff it into you, his belly counseled him.

Unzip the example, then dotnet run.

Full program

The zip includes sample.flac, the unmodified mono 16 kHz LibriSpeech test-clean utterance 1089-134686-0001: “Stuff it into you, his belly counseled him.” Attribution: Vassil Panayotov, Guoguo Chen, Daniel Povey and Sanjeev Khudanpur (2015), CC BY 4.0. Recognition output can differ from that reference.

The full program adds explicit speaker labeling and timing candidates. It eagerly loads both timing models; it does not detect overlapping speakers.

Speaker embeddings are biometric data. Obtain informed consent, limit retention and access, and check applicable law, including Illinois BIPA and GDPR Article 9, before enrollment, identification or verification. The library does not persist embeddings; applications remain responsible for audio, transcripts, logs and any copies they retain. This is not legal advice.

First use of the full composition downloads about 3.28 GB of publisher files across its recognition, speaker and timing models and caches them and locally converted graphs in ~/.cache/onespeech/models on Linux (change the cache location).

using OneSpeech;
using OneSpeech.Audio;
using OneSpeech.Catalog;
using OneSpeech.Moonshine;
using OneSpeech.Vad;
using OneSpeech.Pipeline;
using OneSpeech.Embedders;
using OneSpeech.Diarization;
using OneSpeech.Qwen3;
using OneSpeech.Alignment;
 
using var recognizer = await MoonshineRecognizer.LoadAsync(
    model: "moonshine-ai/moonshine-tiny");
using var recognitionVad = new SileroVad();
// CAM++ turns voices into speaker embeddings.
using var embedder = await CampPlusEmbedder.LoadAsync();
using var speakerVad = new SileroVad(SileroVadOptions.Diarization);
// ClusteringDiarizer groups speech by speaker.
using var diarizer = new ClusteringDiarizer(embedder, speakerVad);
// Qwen3 aligns the recognized words to the audio.
using var qwen = await Qwen3ForcedAligner.LoadAsync();
// OmniASR supplies a second word-timing candidate.
using var ctc = await OmniAsrAligner.LoadAsync();
var pipeline = new VadTranscriptionPipeline(recognizer, recognitionVad)
{
    Diarizer = diarizer,
    TimingAligners = [qwen, ctc]
};
var result = pipeline.Transcribe("sample.flac", "en");
Console.WriteLine(result.Text);
var segment = result.Segments.First();
Console.WriteLine($"Speaker: {segment.Words.FirstOrDefault()?.Speaker?.Id ?? "unknown"}");
foreach (var word in segment.Words.Take(3))
    Console.WriteLine(FormattableString.Invariant(
        $"{word.Text}: {word.Start?.TotalSeconds:F3}–{word.End?.TotalSeconds:F3} s"));

Output:

Duff it into you, his belly counseled him.
Speaker: speaker-1
Duff: 0.546–0.786 s
it: 0.786–0.866 s
into: 0.866–1.186 s

Invalid or corrupt audio is rejected; I/O and model acquisition failures propagate as exceptions.

Variations

Use only the CTC timing candidate

In the full program’s pipeline initializer, replace the timing assignment with:

TimingAligners = [ctc]

First use of the full composition downloads about 3.28 GB of publisher files across its recognition, speaker and timing models and caches them and locally converted graphs in ~/.cache/onespeech/models on Linux (change the cache location).

The recognition text and speaker label stay the same; the first three word times change to:

Duff it into you, his belly counseled him.
Speaker: speaker-1
Duff: 0.606–0.726 s
it: 0.786–0.866 s
into: 0.926–1.126 s

Related recipes

• Transcribe an audio file
• Run offline speech recognition
• Get word timestamps

The short answer

Build a VadTranscriptionPipeline from a recognizer and a voice activity detector (VAD). Add a Diarizer for speaker labels and TimingAligners for word boundaries. These are separate roles: recognition supplies text, VAD selects speech intervals, diarization groups voices, and alignment places written words on the audio clock. The full program composes all four operations rather than asking Speech.Transcribe to choose them.

How do I build a custom speech recognition pipeline in C#?

Start with the two required collaborators shown in the snippet at the top. The recognizer converts speech into words. Silero finds the intervals that contain speech, so the pipeline can send bounded utterances to the recognizer instead of treating a long file as a single model input. Both objects must remain alive until processing finishes.

The full program adds CAM++ speaker embeddings, a second Silero instance configured for diarization, and a clustering diarizer. It deliberately keeps recognition VAD separate from speaker VAD. Those detectors use different profiles for different jobs; sharing the recognition detector is not the documented reconstruction of the file pipeline. Clustering assigns recording-local speaker labels, not a known person’s identity. This setup does not detect overlapping speakers.

Add word timing separately

Moonshine does not supply native word timestamps. The full program explicitly loads Qwen3-ForcedAligner and a CTC aligner as timing candidates. Both are acquired before transcription starts. This is different from automatic file transcription, which can defer acquiring timing models until a recording needs them. An explicit pipeline is useful when the application must control exactly which role instances exist, but that control also means taking responsibility for their lifetime and model selection.

TimingAligners = [ctc] is the variation above when you want only the CTC candidate. Omitting timing candidates is also different from producing precise word boundaries: segment boundaries found by VAD do not become word timestamps. Missing boundaries must remain unknown rather than being divided evenly across the text.

Keep ownership with the application

Pipelines and clustering diarizers borrow their collaborators. Disposing a wrapper does not dispose the recognizer, detector, aligner or embedder it was given. The using declarations in the full program keep each graph owner alive and dispose the objects in reverse declaration order after processing. Stop processing before releasing any model. For live use, dispose sessions before the recognizer and detector they borrow.

This explicit pipeline uses an English model and passes the English language hint. It does not reproduce the automatic per-recording language selection of Speech.Transcribe. Choose models and aligners that support the language of your recordings instead of assuming the wrapper can add an unsupported language to a recognizer.

The sample output intentionally includes the recognizer’s “Duff” transcription of the recording’s “Stuff.” Composing timing and speaker roles does not correct recognition text. Keep the original audio when reviewing an uncertain transcript, and treat a successful run as evidence that the pipeline completed, not that every word was recognized correctly. See the loading and ownership contract for the role interfaces and word timing for timing-stage behavior.