Recursive AI

Make an exclusive speaker caption timeline in C#

OneSpeech is a free, open-source speech recognition library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

Select one speaker at a time from existing turns, leaving gaps uncovered. This does not recognize words.

Terminal
dotnet add package RecursiveAI.OneSpeech

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using OneSpeech;
using OneSpeech.Diarization;
 
var turns = new[] { Make("b"), Make("a") };
var timeline =
    TranscriptDiarization.Exclusive(turns);
foreach (var turn in timeline)
    Console.WriteLine(turn.Speaker.Id);
 
static ActiveTurn Make(string id) => new() {
    Turn = new Turn { Speaker = new() { Id = id },
        Start = TimeSpan.Zero,
        End = TimeSpan.FromSeconds(2), Text = "" }
};

Unzip the example, then dotnet run.

Prints:

Text
a

The full program

Both turns in the snippet at the top cover 0–2 seconds with equal activity. Ordinal ID order selects a, regardless of input order.

The full program uses these synthetic turns; no audio, models or speaker embeddings are involved:

SpeakerStart (seconds)End (seconds)Activity
b030.5
a120.8
a240.5
b560.5
C#
using OneSpeech;
using OneSpeech.Diarization;
 
ActiveTurn[] turns =
[
    Make("b", 0, 3, 0.5),
    Make("a", 1, 2, 0.8),
    Make("a", 2, 4, 0.5),
    Make("b", 5, 6, 0.5)
];
foreach (var turn in TranscriptDiarization.Exclusive(turns))
    Console.WriteLine($"{turn.Start.TotalSeconds:0}-{turn.End.TotalSeconds:0}: {turn.Speaker.Id}; text=\"{turn.Text}\"; words={turn.Words.Length}");
 
static ActiveTurn Make(string id, double start, double end, double activity) => new()
{
    Turn = new Turn { Speaker = new SpeakerLabel { Id = id },
        Start = TimeSpan.FromSeconds(start), End = TimeSpan.FromSeconds(end), Text = "" },
    Activity = activity
};

dotnet run prints:

Text
0-1: b; text=""; words=0
1-4: a; text=""; words=0
5-6: b; text=""; words=0

Invalid turns, such as a zero-length interval or an unassigned speaker, throw ArgumentException.

Variations

Report uncovered gaps

Append this to the full program, above the Make function:

C#
var timeline = TranscriptDiarization.Exclusive(turns);
for (int i = 1; i < timeline.Length; i++)
    if (timeline[i - 1].End < timeline[i].Start)
        Console.WriteLine($"Gap: {timeline[i - 1].End.TotalSeconds:0}-{timeline[i].Start.TotalSeconds:0}");

It adds:

Text
Gap: 4-5

The short answer

Call TranscriptDiarization.Exclusive with ActiveTurn values. It partitions the timeline at turn endpoints, selects the highest-activity speaker in each covered interval, breaks equal-activity ties by ordinal speaker ID and coalesces adjacent intervals for the same speaker. Gaps remain gaps.

The output is a speaker view, not a transcript. Its turn text and words are empty. Place caption text separately using real timing information; the exclusive view does not infer which words were spoken during its selected intervals.

How do I make an exclusive speaker caption timeline in C#?

Start with original-clock speaker turns, each with a named, assigned document-local speaker ID, a nonnegative start and a positive duration. Wrap each in ActiveTurn. Activity is optional and defaults to one. When supplied, it is a relative value from zero to one, not a calibrated probability that the speaker is correct.

In the complete example, b initially covers 0–3 seconds. Speaker a has greater activity at 1–2, then equal activity at 2–3. The equal-activity comparison selects a by ID. Speaker a continues alone until 4. Those adjacent winning intervals become a single 1–4 turn, even though the input contained separate a turns.

The final b turn begins at 5, leaving 4–5 uncovered. Exclusive does not extend the previous turn, assign an “unknown” speaker or generate a silence caption for that gap. The variation shows how an application can report it explicitly. Use the original turns when you also need to retain simultaneous speech; an exclusive timeline deliberately chooses only one speaker at a time.

What is the difference between a caption and a transcript?

For this API, the important distinction is between a timeline of speaker activity and timed words. Exclusive supplies only the first. It returns empty Text and Words on purpose. Copying all the original transcript text into those turns would invent a text-to-time alignment.

TranscriptDiarization.Merge is the separate operation for assigning already-timed words to speakers. It totals covered duration per speaker for each word after resolving simultaneous turns. Null timings are never filled in. A word with no coverage, invalid endpoints or zero duration remains unassigned. Use real recognized or aligned word timings before making captions that depend on word positions.

The diarization documentation also describes the overlap flag. A positive-duration intersection with different simultaneous speakers can mark a word as overlapping regardless of which speaker wins. That information is different from an exclusive caption view and should not be inferred from the absence of overlapping output turns.

How should I identify speakers in captions?

The IDs here are document-local labels, not names or verified identities. The API’s tie-break order is ordinal ID order, not a judgment about who is more important, louder or more likely to have said the words. Keep IDs stable within a document if you want reproducible tie results.

A caption renderer can map those labels to display names when the application has reliable information, but that mapping is separate from selecting timeline intervals. This example does not listen to voices, enroll people or create embeddings. It uses invented labels and supplied times only.

Caption text also needs an editorial decision when people speak simultaneously: choosing one timeline interval does not preserve every contribution. Keep the overlapping source turns and timed transcript if your output needs to represent both people. An exclusive timeline is useful for a single-speaker display track, not a replacement for the full record of an overlapping conversation.

All onespeech recipes