Encode and decode Opus in C#

To encode and decode Opus in C#, add the RecursiveAI.OneAudio package. Audio.Transcode("in.wav", "out.opus") writes an Ogg Opus file from WAV, MP3, FLAC or Ogg Vorbis, and Audio.Load("out.opus") decodes one. For voice chat and other real-time uses, OpusEncoder and OpusDecoder work on raw packets, with packet loss concealment. Everything is managed C# with no libopus to ship, the decoder passes all 12 RFC 8251 test vectors, and the encoder’s packets are byte-identical with the RFC 6716 reference encoder in all 269 configurations the project tested. Every sample below was compiled and run against OneAudio 1.2.1 on .NET 10.

Before you start

dotnet new console -o OpusDemo
cd OpusDemo
dotnet add package RecursiveAI.OneAudio --version 1.2.1

OneAudio targets .NET 10 and is licensed MIT OR Apache-2.0. The samples use music.wav, 30 seconds of stereo music at 44.1 kHz, and voice16k.wav, 30 seconds of 16 kHz mono speech.

Encode a WAV file to Ogg Opus and decode it

using OneAudio;
using OneAudio.Opus;
 
// WAV (or MP3, FLAC, Vorbis) to Ogg Opus at 96 kbit/s. 44.1 kHz input is resampled to 48 kHz on the way.
Audio.Transcode("music.wav", "music.opus", new TranscodeOptions
{
    Opus = new OpusFileWriterSettings { Encoder = new OpusEncoderSettings { BitrateBps = 96_000 } },
});
Console.WriteLine($"music.wav {new FileInfo("music.wav").Length:N0} bytes -> music.opus {new FileInfo("music.opus").Length:N0} bytes");
 
// Decode it again: Opus always decodes at 48 kHz.
AudioClip clip = Audio.Load("music.opus");
Console.WriteLine($"{clip.Format.SampleRate} Hz, {clip.Format.Channels} channels, {clip.Frames} frames = {clip.Frames / 48000.0:F3} s");

music.wav 5,292,078 bytes -> music.opus 358,929 bytes
48000 Hz, 2 channels, 1440001 frames = 30.000 s

• The file is 6.8% of the WAV’s size at 96 kbit/s. OpusEncoderSettings also sets Application (Audio, Voip, RestrictedLowDelay), BitrateMode (VBR by default, constrained VBR or CBR), Complexity 0 to 10 and the bandwidth. Leave BitrateBps unset and the encoder picks a rate itself.
• Opus works at 48 kHz, so 44.1 kHz input goes through the polyphase resampler on the way in. The file’s header records the original rate, and decoders return 48 kHz audio.
• The decoded length is the 48 kHz length of the input: the writer stores the encoder’s pre-skip and trims the padding of the last packet. The streaming resampler can add a frame at the end, which is the 1,440,001 here.
• For samples your own code produces, such as a recording, write them with OpusFileWriter: Write as often as you like, then Finish. Write allocates nothing. More than two channels are written with channel mapping family 1, up to 8 channels.
• Audio.Load and Audio.Open read any Ogg Opus file, including files written by ffmpeg with libopus, and chained files play as one stream with exact seeking.

Encode and decode raw Opus packets

Voice chat, game networking and RTP send raw Opus packets, not files. OpusEncoder turns a frame of 48 kHz samples into one packet, and OpusDecoder turns a packet back into samples. This loop codes 30 seconds of speech in 20 ms packets at 24 kbit/s and drops every 50th packet to show concealment:

using OneAudio;
using OneAudio.Opus;
 
// Speech for a voice chat: 48 kHz mono, cut into 20 ms frames (960 samples).
float[] speech = Audio.Load("voice16k.wav", new LoadOptions { SampleRate = 48000 }).Samples.ToArray();
const int FrameSize = 960;
 
var encoder = new OpusEncoder(new OpusEncoderSettings
{
    Channels = 1,
    Application = OpusApplication.Voip,
    BitrateBps = 24_000,
});
var decoder = new OpusDecoder(48000, 1);
 
byte[] packet = new byte[1275];                      // the largest Opus packet
float[] decoded = new float[5760];                   // 120 ms, the longest packet duration
int packets = 0, bytes = 0, samplesOut = 0;
for (int pos = 0; pos + FrameSize <= speech.Length; pos += FrameSize)
{
    int length = encoder.Encode(speech.AsSpan(pos, FrameSize), FrameSize, packet);
    packets++;
    bytes += length;
 
    // Send packet[..length] over the network. Here, every 50th packet is "lost":
    // an empty packet tells the decoder to conceal the gap.
    ReadOnlySpan<byte> received = packets % 50 == 0 ? ReadOnlySpan<byte>.Empty : packet.AsSpan(0, length);
    samplesOut += decoder.Decode(received, decoded);
}
Console.WriteLine($"{packets} packets, {bytes / (double)packets:F1} bytes each = {bytes * 8 / (packets * 0.020) / 1000:F1} kbit/s");
Console.WriteLine($"decoded {samplesOut} samples ({samplesOut / 48000.0:F2} s), lookahead {encoder.LookaheadSamples} samples");

1500 packets, 59.8 bytes each = 23.9 kbit/s
decoded 1440000 samples (30.00 s), lookahead 312 samples

• A frame is 120, 240, 480, 960, 1920 or 2880 samples per channel at 48 kHz (2.5 to 60 ms). 20 ms is the usual choice for speech.
• An empty packet tells the decoder the packet was lost, and it conceals the gap, so the output keeps its length: 1,500 packets give 1,440,000 samples whether or not some are lost. For lossy networks, set InBandFec = true with a PacketLossPercent above 0 to add redundancy to speech packets, and Dtx = true to send one-byte packets during silence.
• Both classes allocate everything in their constructors. Encode and Decode allocate nothing, so they can run on an audio thread.
• LookaheadSamples (312 here) is the encoder’s delay, the value an Ogg file stores as pre-skip.
• OpusDecoder outputs 48 or 24 kHz, mono or stereo; a stereo stream decoded to mono is downmixed as the Opus specification says.

Concentus and OpusSharp

The two packages that come up most for Opus in C# are Concentus and OpusSharp.

• Concentus 2.2.2 (May 26, 2024) is a C# port of libopus 1.1.2, according to its README, under the Opus project’s BSD-style license. Its package description says it does “NOT contain code to parse .ogg or .opus container files”, which is the job of a separate Concentus.OggFile library. Its XML documentation says OpusCodecFactory uses a native libopus through P/Invoke when one is present, controlled by AttemptToUseNativeLibrary, which is true by default.
• OpusSharp 1.6.8 (MIT) wraps the native libopus: its README says the core library “uses the native compiled DLL’s/binaries”, shipped in OpusSharp.Natives for Windows, Linux, macOS, Android and iOS.

The same voice loop with Concentus, written from the signatures in its XML documentation, gives nearly the same packet sizes. Reading the WAV is left to another library, OneAudio here:

using Concentus;
using Concentus.Enums;
using OneAudio;
 
// The same voice loop with Concentus. It works on raw packets only: decoding the WAV is left to another library.
float[] speech = Audio.Load("voice16k.wav", new LoadOptions { SampleRate = 48000 }).Samples.ToArray();
const int FrameSize = 960;
 
IOpusEncoder encoder = OpusCodecFactory.CreateEncoder(48000, 1, OpusApplication.OPUS_APPLICATION_VOIP);
encoder.Bitrate = 24_000;
IOpusDecoder decoder = OpusCodecFactory.CreateDecoder(48000, 1);
 
byte[] packet = new byte[1275];
float[] decoded = new float[5760];
int packets = 0, bytes = 0, samplesOut = 0;
for (int pos = 0; pos + FrameSize <= speech.Length; pos += FrameSize)
{
    int length = encoder.Encode(speech.AsSpan(pos, FrameSize), FrameSize, packet, packet.Length);
    packets++;
    bytes += length;
    samplesOut += decoder.Decode(packet.AsSpan(0, length), decoded, 5760, false);
}
Console.WriteLine($"{packets} packets, {bytes / (double)packets:F1} bytes each = {bytes * 8 / (packets * 0.020) / 1000:F1} kbit/s");
Console.WriteLine($"decoded {samplesOut} samples ({samplesOut / 48000.0:F2} s)");

1500 packets, 59.9 bytes each = 23.9 kbit/s
decoded 1440000 samples (30.00 s)

If you only need raw packets and already have Concentus working, it does that job. OneAudio adds the parts around it in the same package: reading and writing Ogg Opus files, resampling to 48 kHz, decoding MP3, FLAC, Vorbis and WAV input, and, with the Devices package, the microphone and speaker at either end of a voice call. It never looks for a native library, so what runs is the managed code you tested.

Conformance and speed

From the project’s published results:

• Decoder: all 12 RFC 8251 test vectors pass, on both the plain and the no-phase-inversion sets, compared with opus_compare at 48 kHz.
• Encoder: packets compared byte for byte with the RFC 6716 reference encoder are identical in 269 of 269 configurations (CELT, low-rate SILK and hybrid, mode transitions and random configurations).
• Decode speed: on the RFC test vectors, OneAudio decoded 1.07 to 1.20 times as fast as Concentus 2.2.2 and allocated nothing per packet (at most 1.5 KB per file, against 24 to 52 MB for Concentus). Measured with BenchmarkDotNet 0.15.8 on an AMD Ryzen 9 7950X3D under Linux with .NET 10.0.13.
• Encode speed: on 10 seconds of stereo music at 48 kHz, OneAudio encoded in 36.6 to 84.5 ms against 49.4 to 103.9 ms for Concentus at 16 kbit/s, and within 8% of it at 64 and 128 kbit/s, allocating nothing against 40 to 56 MB, on the same machine.

Where next

• The cookbook: encode Opus and decode Opus.
• The Opus section of the API overview: every encoder setting, modes and multistream.
• Record audio in C# and play audio in C#, for both ends of a voice call.
• Audio in .NET without FFmpeg, if you are replacing ffmpeg -c:a libopus.