Recursive AI

Run Whisper int8 inference in C#

OneSpeech is a free, open-source speech recognition library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

NuGet GitHub Docs Download example (.zip)

Select a mixed-int8 Whisper model explicitly; the default remains fp32.

Terminal
dotnet add package RecursiveAI.OneSpeech

OneAudio is a dependency of RecursiveAI.OneSpeech; no separate install is needed.

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

sample.flac is in the zip.

C#
using OneSpeech;
var transcript = Speech.Transcribe(
    "sample.flac", language: "en",
    model: "openai/whisper-large-v3-turbo-int8");
Console.WriteLine(transcript.Text);

Unzip the example, then dotnet run.

Prints:

Text
stuffered into you,' his belly counselled him.

“stuffered” is a recognition error: the recording says “stuff it into you”.

On an empty cache, this English run acquires about 3.49 GB of publisher files: Whisper turbo int8’s source checkpoint, Qwen3-ForcedAligner and CAM++ English, including configuration/tokenizer files in the pinned catalog. Converted runtime files require additional disk space; timing fallbacks can acquire more models.

sample.flac is the LibriSpeech test-clean clip 1089-134686-0001: “stuff it into you his belly counselled him”. It is included under CC BY 4.0, credited to Vassil Panayotov, Guoguo Chen, Daniel Povey and Sanjeev Khudanpur (source). First use downloads publisher weights and converts the selected model locally. The high-level call also loads word-timing and speaker-grouping models.

Model download or integrity errors fail the call rather than silently substituting an fp32 model.

Variations

Read the same recording from a stream. Replace the transcription call with:

C#
using var input = File.OpenRead("sample.flac");
var transcript = await Speech.TranscribeAsync(
    input, language: "en",
    model: "openai/whisper-large-v3-turbo-int8");
Text
stuffered into you,' his belly counselled him.

The short answer

To run Whisper int8 inference in C#, pass openai/whisper-large-v3-turbo-int8 as the model ID. OneSpeech explicitly selects its mixed-int8 graph rather than changing the default model. The transcription API and transcript JSON schema are unchanged.

Quantization does not promise identical tokens to fp32. Compare recognition on your application’s recordings before changing models.

What does INT8 quantization mean?

In the supported Whisper variants, constant-weight linear projections use signed int8 weights with a separate scale for each output channel. Dynamic activation quantization feeds integer matrix multiplication; accumulation uses int32. The result is scaled back to floating point before adding any floating-point bias.

Other operations remain fp32, including convolutions, embedding lookup, normalization, attention score/value products, residuals and nonlinearities. These are mixed-int8 graphs, not all-integer networks. Quantization changes logits, so the decoder can choose different tokens even though the tokenizer and publisher checkpoint are the same.

The conversion is deterministic and takes place in the local model cache. The catalog pins converted file sizes and SHA-256 digests. Source and revision metadata still identify the original publisher checkpoint, while the model ID identifies the selected int8 variant. See Integer inference for the numerical scheme.

How do I run a C# Whisper int8 inference example?

Replace sample.flac with another supported recording to keep the same int8 recognizer.

The catalog also offers openai/whisper-small-int8. A third variant, moonshine-ai/moonshine-streaming-tiny-int8, quantizes a different subset: its decoder MLPs and vocabulary projection, not its acoustic encoder, attention or cross-attention projections. Do not infer that every model accepts an -int8 suffix. Choose an ID actually listed in the catalog.

The high-level transcription pipeline may need more than the named recognizer. Word timing and speaker grouping can load companion models. Selecting an int8 recognizer does not quantize those companions, make them unnecessary, or prepare every file needed for disconnected deployment.

Can you run Whisper locally?

Yes. Inference runs locally after the required models are available. First use can download publisher files and create verified runtime files in the local cache. Plan that preparation step separately when deploying to a disconnected host, and keep sufficient disk space for both publisher and converted files. See Model files and pre-fetch.

Integer inference is not a guarantee of lower latency on every CPU or thread count. Choose it based on measurements on your deployment hardware and recognition tests on your own audio, not on the name of its numeric format.

All onespeech recipes