Recursive AI

Speaker diarization in C#

OneSpeech is a free, open-source speech recognition library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

NuGet GitHub Docs Download example (.zip)

Speaker diarization labels who spoke when, not people’s identities. The default clustering diarizer does not detect overlapping speech.

CAM++ is the speaker-embedding model: it turns speech into voice features for clustering. Silero VAD (voice activity detection) finds the speech parts.

Terminal
dotnet add package RecursiveAI.OneSpeech
dotnet add package RecursiveAI.OneSpeech.Models.CamPlusEnglish
dotnet add package RecursiveAI.OneAudio
dotnet add package RecursiveAI.OneSpeech.Models.MoonshineStreamingTiny

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

two-speakers.flac is included in the zip: a short NASA Apollo 11 radio exchange with two speakers taking turns.

C#
using OneSpeech;
using OneSpeech.Diarization;
using OneSpeech.Embedders;
using OneSpeech.Vad;
 
var clip = OneAudio.Audio.Load("two-speakers.flac");
var audio = new AudioBuffer(clip.Samples, 16000);
using var embedder = await CampPlusEmbedder.LoadAsync();
using var vad = new SileroVad(SileroVadOptions.Diarization);
using var diarizer = new ClusteringDiarizer(embedder, vad);
foreach (var turn in diarizer.Diarize(audio))
    Console.WriteLine($"{turn.Speaker.Id}: " +
        $"{turn.Start.TotalSeconds:F2}–" +
        $"{turn.End.TotalSeconds:F2}");

Prints:

Text
speaker-1: 0.23–3.11
speaker-2: 3.11–11.01

Unzip the example, then dotnet run.

Full program

The zip includes two-speakers.flac, an excerpt of NASA’s “The Eagle Has Landed” extended recording, listed on NASA Historical Sounds. Neil Armstrong and Charles Duke take turns speaking. The excerpt runs from 2.89 to 13.90 seconds of the original and is converted to mono, 16 kHz FLAC. NASA mission audio is US government material; see NASA’s media usage guidelines. SAMPLES.md in the zip records the source and conversion. The speaker IDs below are anonymous cluster labels, not inferred names.

C#
using OneSpeech;
using OneSpeech.Diarization;
using OneSpeech.Embedders;
using OneSpeech.Vad;
 
var clip = OneAudio.Audio.Load(
    args.Contains("--wav") ? "input.wav" : "two-speakers.flac",
    new OneAudio.LoadOptions { SampleRate = 16000, Channels = 1 });
var audio = new AudioBuffer(clip.Samples, 16000);
using var embedder = await CampPlusEmbedder.LoadAsync();
using var vad = new SileroVad(SileroVadOptions.Diarization);
using var diarizer = new ClusteringDiarizer(embedder, vad);
foreach (var turn in diarizer.Diarize(audio))
    Console.WriteLine($"{turn.Speaker.Id}: " +
        $"{turn.Start.TotalSeconds:F2}–" +
        $"{turn.End.TotalSeconds:F2}");
 
if (args.Contains("--transcript"))
{
    using var recognizer = await OneSpeech.Moonshine.MoonshineRecognizer.LoadAsync();
    foreach (var turn in diarizer.Diarize(audio))
    {
        int start = (int)Math.Round(turn.Start.TotalSeconds * 16000);
        int end = (int)Math.Round(turn.End.TotalSeconds * 16000);
        var part = new AudioBuffer(clip.Samples.Slice(start, end - start), 16000);
        foreach (var segment in recognizer.Transcribe(part, "en").Segments)
            Console.WriteLine($"{turn.Speaker.Id}: {segment.Text}");
    }
}

Output:

Text
speaker-1: 0.23–3.11
speaker-2: 3.11–11.01

Missing or corrupt input and missing model files fail instead of producing a successful empty result.

Variations

Decode and resample your own WAV or MP3

OneAudio can decode and convert the rate and channel count while loading. Keep the embedder, VAD and diarizer declarations from the full program; replace its audio-loading and turn-printing code with:

C#
var clip = OneAudio.Audio.Load("input.wav",
    new OneAudio.LoadOptions { SampleRate = 16000, Channels = 1 });
var audio = new AudioBuffer(clip.Samples, 16000);
foreach (var turn in diarizer.Diarize(audio))
    Console.WriteLine($"{turn.Speaker.Id}: " +
        $"{turn.Start.TotalSeconds:F2}–" +
        $"{turn.End.TotalSeconds:F2}");

input.wav in the zip is the same NASA excerpt at 44.1 kHz stereo, so this example actually resamples and downmixes before diarizing. Run dotnet run -- --wav to try it. For your own recording, replace the filename with your WAV or MP3 path; the requested output stays mono 16 kHz. Loading holds the decoded clip in memory, so use this variation for short recordings.

Output with the included WAV:

Text
speaker-1: 0.23–3.11
speaker-2: 3.11–11.01

Use a known speaker count

This recording contains two speakers. To constrain clustering to that known count, replace the diarizer declaration with:

C#
using var diarizer = new ClusteringDiarizer(embedder, vad,
    new ClusteringOptions { NumSpeakers = 2 });

Output:

Text
speaker-1: 0.23–3.11
speaker-2: 3.11–10.81
speaker-1: 10.81–11.01

The count fixes the number of clusters, not the number of turns. It also disables the default unconstrained resegmentation, so boundary assignments can differ: here the final short interval changes labels. Turn boundaries are estimates, not a verified account of the conversation.

The English recognition model package is installed above.

Moonshine is the English recognition model. After the audio and diarizer declarations above, transcribe each detected turn and print its speaker label beside each returned transcript segment:

C#
using var recognizer = await OneSpeech.Moonshine.MoonshineRecognizer.LoadAsync();
foreach (var turn in diarizer.Diarize(audio))
{
    int start = (int)Math.Round(turn.Start.TotalSeconds * 16000);
    int end = (int)Math.Round(turn.End.TotalSeconds * 16000);
    var part = new AudioBuffer(clip.Samples.Slice(start, end - start), 16000);
    foreach (var segment in recognizer.Transcribe(part, "en").Segments)
        Console.WriteLine($"{turn.Speaker.Id}: {segment.Text}");
}

The zip includes this path: run dotnet run -- --transcript to print the turn times followed by these lines. This noisy historical radio recording produces recognition mistakes; the output below is uncorrected:

Text
speaker-1: And you want to be fair? Is he going to have land?
speaker-2: Roger Twain. Crane Quarletie, we copy on the ground. You got a bunch of guys about to turn blue. We're breathing again. Thanks a lot.

The turn supplies the label; recognition supplies the text. This does not identify a person or create word-level timings. Each turn here fits Moonshine’s short-utterance input limit.

The short answer

Use a ClusteringDiarizer with a speaker embedder and a voice activity detector (VAD), which finds speech intervals. The example loads CAM++ English weights from a NuGet model package, finds speech in a two-speaker NASA recording and prints a document-local ID and a start/end time for each turn. It performs real voice clustering, not a rotation through a fixed list of speaker names.

What is speaker diarization?

Diarization answers “who spoke when?” within one recording. Recognition answers “what was said?” They are different roles. The default run labels turns without transcribing words; the “Print who said what” variation adds recognition. The two-speaker sample produces turns for two anonymous clusters; a meeting can produce multiple turns for the same speaker.

A label such as speaker-1 is meaningful only inside that recording. It is not a stable identity across meetings. Reordering input or changing clustering settings can change the labels. Keep the recording ID together with its turn list in your application. Do not treat an anonymous label as a contact record or as proof of identity.

The diarization guide explains how turn intervals and word assignment work. When merging turns with a transcript, uncovered words remain unassigned and unknown word boundaries stay null. A turn boundary alone is not evidence of an individual word’s timing.

How do I implement speaker diarization in C#?

The program above decodes a short, mono, 16 kHz FLAC into memory, wraps its samples in an AudioBuffer and calls Diarize. The sample’s existing rate and channel count match that constructor. For another file, decode and convert to the required format first rather than labeling arbitrary samples as 16 kHz.

For long files, the library also exposes BlockDiarizer and a file transcription pipeline. The clustering implementation processes consecutive blocks on a shared recording clock, rather than independently restarting the speaker numbering for each block. File transcription with a block diarizer makes a diarization pass and a recognition pass. Temporary PCM is bounded, but turn metadata and embedding storage still grow with the work. See the file pipeline.

Keep the embedder and detector alive until diarization finishes. The diarizer borrows them; disposing it does not dispose the models. The using declarations in the example release these resources in the right order.

Which speaker diarization model should I use?

The example explicitly selects CAM++ English for English audio. The high-level transcription helper instead chooses CAM++ Chinese for requested or detected Chinese and CAM++ English otherwise. The docs call this a language-based starting choice, not a quality-validated default for every language.

For separated-party stereo calls, channel diarization can use the existing channel separation. It cannot distinguish two people sharing one channel or prove their identities. For mixed recordings, voice clustering is the relevant path.

The default clustering model assigns one speaker at a time and does not detect overlapping speech. An explicit overlap-aware diarizer is documented separately in the overlap guide. Detecting simultaneous activity does not separate voices or recover words a recognizer missed. Choose based on your recording layout and overlap needs, not just the number of voices you expect.

All onespeech recipes