Recursive AI

Label stereo call channels in C#

OneSpeech is a free, open-source speech recognition library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

NuGet GitHub Docs Download example (.zip)

Channel diarization labels each side of a separated stereo recording. It requires two channels, not a mixed mono call.

Terminal
dotnet add package RecursiveAI.OneSpeech
dotnet add package RecursiveAI.OneAudio

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

call.wav is the constructed NASA speech fixture in the zip (16 kHz, stereo).

C#
using OneSpeech;
using OneSpeech.Diarization;
using OneSpeech.Vad;
 
var clip = OneAudio.Audio.Load("call.wav");
var audio = new AudioBuffer(clip.Samples, clip.Format.SampleRate, clip.Format.Channels);
using var vad = new SileroVad(
    SileroVadOptions.Diarization);
using var diarizer = new ChannelDiarizer(vad);
foreach (var turn in diarizer.Diarize(audio))
    Console.WriteLine($"{turn.Speaker.Id}: " +
        $"{turn.Start.TotalSeconds:F2}–" +
        $"{turn.End.TotalSeconds:F2}");
Text
channel-1: 0.23–3.29
channel-2: 2.70–10.58

Unzip the example, then dotnet run.

The fixture routes the first voice left and the second right, deliberately shifting the second turn earlier to create overlap. It is not an original dual-channel call recording.

ChannelDiarizer throws CapabilityNotSupportedException when separation is inconclusive; it never silently downmixes the input.

Variations

Bypass auto-detection for a known separated call

Replace the diarizer declaration in the program with:

C#
using var diarizer = new ChannelDiarizer(vad, forceSeparated: true);
Text
channel-1: 0.23–3.29
channel-2: 2.70–10.58

This bypasses separation detection, not the two-channel requirement. Use it only when you already know the channels contain separate parties.

Reject duplicated mono audio

This complete variation copies the left channel to the right to make dual-mono input:

C#
using OneSpeech;
using OneSpeech.Diarization;
using OneSpeech.Vad;
 
var clip = OneAudio.Audio.Load("call.wav");
if (clip.Format.Channels != 2)
    throw new InvalidOperationException("This variation requires stereo input.");
var samples = clip.Samples.ToArray();
for (int i = 0; i < samples.Length; i += 2)
    samples[i + 1] = samples[i];
var dualMono = new AudioBuffer(samples, clip.Format.SampleRate, clip.Format.Channels);
Console.WriteLine(ChannelDiarizer.IsSeparated(dualMono));
using var vad = new SileroVad(SileroVadOptions.Diarization);
using var diarizer = new ChannelDiarizer(vad);
try
{
    diarizer.Diarize(dualMono);
}
catch (CapabilityNotSupportedException)
{
    Console.WriteLine("Dual-mono rejected");
}
Text
False
Dual-mono rejected

The short answer

Keep the original stereo channels and pass them to ChannelDiarizer with a voice activity detector (VAD), which finds speech intervals. It emits channel-1 and channel-2 turns on the recording’s original clock. It does not infer names or distinguish different people who share a channel. No speaker-embedding model or gated segmentation model is needed for this route.

This is appropriate when the recorder stored separate parties on separate channels. A file having two channels is not sufficient evidence: dual-mono, silence and one-sided conversations are inconclusive under the default detection rule.

What does speaker diarization mean for a stereo call?

In this case, “who spoke when” means which recorded side was active. Channel provenance supplies the label, while the VAD supplies the speech intervals. The sample does not transcribe speech. Its two printed turns overlap because speech activity is retained independently for each side, rather than forcing every instant to have one winner.

The fixture is a deliberately constructed example, not evidence about how any call recorder separates parties. Its NASA excerpt was split at the original turn boundary, routed into separate channels, and partly overlapped. Use your recorder’s actual separated output for real calls; do not route arbitrary mixed voices left and right and expect channel labels to identify them.

When voices share one mixed channel, use a voice-clustering diarizer instead. That is a different task: acoustic features group speech by voice rather than by recording channel. If you make that change, the library’s privacy note applies: “Speaker embeddings are biometric data. Obtain informed consent, limit retention and access, and check applicable law, including Illinois BIPA and GDPR Article 9, before enrollment, identification or verification.” The library does not persist embeddings; your application remains responsible for retained audio, transcripts, logs and copies. This is not legal advice.

How do I keep timestamps when diarizing stereo channels in C#?

Do not remove each channel’s silence or concatenate its speech clips before detecting turns. That would replace the shared recording clock with a different timeline for each party. The program passes interleaved stereo samples together with the actual sample rate and channel count to AudioBuffer. The fixture is already 16 kHz stereo; for another input, decode or resample while preserving both channels and use the real resulting metadata.

The channel diarization documentation describes independent VAD blocks of at most 30 seconds. Their results retain original-clock times, but VAD state resets at block boundaries. This is not continuous-state VAD equivalence. The caller’s in-memory input remains borrowed, and output metadata grows with the number of turns.

The default separation test looks for dominant activity on both sides and low whole-recording waveform correlation. Those are initial heuristics, not thresholds fitted or validated on a speech development set. Alternating noises or uncorrelated stereo music can pass, so a true result cannot prove separated parties.

Use forceSeparated: true only with independent knowledge of the recording layout. It is useful when a genuine separated call has activity on only one side, but it cannot turn mixed audio into separated parties. The dual-mono variation demonstrates the opposite decision: duplicated channels are refused rather than accepted as two speakers. Keep that failure visible to the caller so it can request the correct recording or select an appropriate mixed-voice workflow.

The zip’s SAMPLES.md records the source: NASA’s “The Eagle Has Landed” recording, linked from NASA Historical Sounds. The speech is US government mission audio; see NASA’s media usage guidelines. The fixture edits are described in that file. No speaker embeddings are computed here.

All onespeech recipes