Recursive AI

Detect overlapping speech in C#

OneSpeech is a free, open-source speech recognition library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

NuGet GitHub Docs Download example (.zip)

A diarizer labels who speaks when. Nemotron is the speech model used here to allow simultaneous speaker labels; the default diarizer does not detect overlap.

Terminal
dotnet add package RecursiveAI.OneSpeech

OneAudio is a dependency of RecursiveAI.OneSpeech; no separate install is needed.

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

overlap.wav is a four-second mixed recording in the zip. This explicitly selected tier-2 model uses OpenMDW-1.1, not the library’s license; first use downloads 396,954,592 bytes of publisher weights and builds runtime graphs locally.

C#
using OneSpeech;
using OneSpeech.Diarization;
var clip = OneAudio.Audio.Load("overlap.wav",
    new OneAudio.LoadOptions { SampleRate = 16000, Channels = 1 });
using var diarizer = await NemotronDiarizer.LoadAsync();
foreach (var turn in diarizer.Diarize(new AudioBuffer(clip.Samples, 16000)))
    Console.WriteLine($"{turn.Speaker.Id}: {turn.Start.TotalSeconds:F2}–{turn.End.TotalSeconds:F2}");

Unzip the example, then dotnet run.

Prints:

Text
speaker_0: 0.00–4.00
speaker_1: 0.30–3.11
speaker_1: 3.59–4.00

speaker_0 remains active while speaker_1 speaks at 0.30–3.11 and 3.59–4.00 seconds: those intersecting turns are the detected overlap.

overlap.wav mixes two four-second parts of NASA Apollo 11 radio audio, starting at 0 and 6 seconds within the included-source excerpt. The NASA audio is adapted from Apollo 11 historical radio audio, a US government work; see NASA’s media policy. No endorsement is implied. This is a constructed overlap example, not a diarization accuracy test.

The diarizer rejects PCM at rates other than 16 kHz and throws on concurrent use of the same instance.

Variations

Print only intersections between different speaker channels. Replace the foreach loop with:

C#
var turns = diarizer.Diarize(new AudioBuffer(clip.Samples, 16000));
for (int i = 0; i < turns.Length; i++)
for (int j = i + 1; j < turns.Length; j++)
{
    if (turns[i].Speaker.Id == turns[j].Speaker.Id) continue;
    var start = turns[i].Start > turns[j].Start ? turns[i].Start : turns[j].Start;
    var end = turns[i].End < turns[j].End ? turns[i].End : turns[j].End;
    if (start < end)
        Console.WriteLine($"Overlap: {start.TotalSeconds:F2}–{end.TotalSeconds:F2}");
}
Text
Overlap: 0.30–3.11
Overlap: 3.59–4.00

The short answer

To detect overlapping speech in C#, explicitly load NemotronDiarizer and inspect its returned speaker turns. Different anonymous speakers can have intersecting time ranges. The default clustering diarizer cannot detect overlap, so switching only the input file or a transcript format is not enough.

The returned turns do not recognize words, name participants or separate the mixture into clean voice tracks.

What is overlapping speech?

Overlapping speech occurs when more than one voice is active at the same time. A diarizer that assigns only one speaker to every instant cannot preserve that distinction. An overlap-aware diarizer can emit separate active ranges for multiple channels.

Nemotron’s activity classifier uses independent speaker outputs rather than forcing one winning speaker per frame. The published OneSpeech implementation supports up to eight anonymous speakers. Its turn boundaries use a 10 ms grid and are clipped to the recording’s duration. Inputs shorter than one complete 10 ms hop produce no turns.

The speaker IDs in the output are anonymous channels. A label such as speaker_0 does not identify a real person, and an overlapping interval does not tell you which words each voice said.

How does diarization handle overlapping speech?

Choose the overlap-aware model explicitly. NemotronDiarizer.LoadAsync resolves the pinned publisher checkpoint, verifies its files and builds runtime graphs locally when needed. This is a tier-2 model and is never selected as the default. Its publisher agreement is OpenMDW-1.1; the library package’s permissive license does not replace that model agreement.

The low-level example decodes and resamples a short file to mono 16 kHz PCM before processing it. For larger recordings, the diarizer also accepts consecutive 16 kHz blocks and averages channels. Internal waveform and model state are bounded, while returned turn storage grows with speech changes. No VAD removes gaps before diarization, so output times stay on the original audio clock. See Nemotron 3 Diarization.

To obtain overlap intervals, compare turns from different speaker IDs. For each pair, take the later start and earlier end; a strictly positive intersection is simultaneous activity. The variation does exactly this. With more than two channels, pairwise intersections can overlap each other; combine ranges if your application needs a single union of all overlap time rather than individual speaker-pair events.

Do not share a diarizer instance between concurrent operations. Dispose it after processing. If a downstream transcript must preserve overlap, retain the simultaneous turns instead of flattening them into one speaker label per moment.

All onespeech recipes