Make a video play in browsers in C# (MakePlayable)

AudioTranscoder.MakePlayable(input, output, profile) turns a file a browser profile can’t play into one it can. It does the least that works: moves an MP4’s index to the front, remuxes into a container the browser takes, or re-encodes an audio track to Opus when that’s the only other thing in the way. Video is never re-encoded; a video codec the browser lacks is reported, not fixed. The inputs are the Sintel trailer (CC BY 3.0, © copyright Blender Foundation | durian.blender.org), whose index is at the end, a copy of it made with FFmpeg as MKV with Vorbis audio (the video copied, the audio encoded), and MDN’s CC0 flower clip as WebM (VP8/Vorbis).

dotnet new console -o MakePlayable && cd MakePlayable
dotnet add package RecursiveAI.OneMedia.Audio --version 0.3.0-preview.1
curl -L -o sintel.mp4 https://download.blender.org/durian/trailer/sintel_trailer-480p.mp4
curl -L -o flower.webm https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.webm
ffmpeg -v error -i sintel.mp4 -c:v copy -c:a libvorbis -q:a 4 sintel-vorbis.mkv

RecursiveAI.OneMedia.Audio brings in the core package and OneAudio, which does the audio decoding and the Opus encoding. Put this in Program.cs and run dotnet run:

using RecursiveAI.OneMedia.Audio;
using RecursiveAI.OneMedia.Playback;
 
Make("sintel-vorbis.mkv", "sintel-safari.mp4", BrowserProfiles.Safari);
Make("sintel.mp4", "sintel-chrome.mp4", BrowserProfiles.Chrome);
Make("sintel-vorbis.mkv", "sintel-generic.mp4", BrowserProfiles.GenericMp4);
Make("flower.webm", "flower-ios.mp4", BrowserProfiles.IosSafari);
 
static void Make(string input, string output, BrowserProfile profile)
{
    MakePlayableReport report = AudioTranscoder.MakePlayable(input, output, profile);
    Console.WriteLine($"{input} for {profile.Id}: {report.Status}"
                      + (report.Target is { } target ? $", written as {target}" : ""));
    foreach (MakePlayableAction action in report.Actions)
        Console.WriteLine($"  {action.Kind}: {action.Detail}");
    foreach (PlaybackBlocker blocker in report.Remaining)
        Console.WriteLine($"  still blocked: {blocker.Code}: {blocker.Message}");
}

sintel-vorbis.mkv for safari: Made, written as Mp4
  Remux: Copy the tracks into mp4, movie box first.
  TranscodeAudio: Audio track 1: vorbis to opus, 96000 bit/s, 48000 Hz, 2 ch, encoder delay 312/48000 s.
sintel.mp4 for chrome: Made
  Faststart: Moved the 39456-byte movie box in front of the media data.
sintel-vorbis.mkv for generic-mp4: Refused
  still blocked: transcode.no-target: Generic MP4 plays none of the codecs an audio transcode can produce (opus) in mp4.
flower.webm for ios-safari: Refused
  still blocked: codec.unsupported: Safari (iOS) does not play vp8.

• MKV with Vorbis, for Safari. Safari doesn’t take Matroska, and in the containers it does take it doesn’t play Vorbis, so the H.264 is remuxed into MP4 and the Vorbis is re-encoded to Opus at 96 kbit/s; the video packets are copied.
• MP4 with its index at the end, for Chrome. Chrome plays it already, but only after fetching the end of the file, so the movie box is moved to the front (MP4 faststart); everything else is copied by range.
• The same MKV, for the generic MP4 profile. That profile plays AAC and MP3 but not Opus, and Opus is the only audio MakePlayable can produce, so it refuses with transcode.no-target and writes nothing.
• VP8 for iOS Safari. A video blocker: nothing is written, and the report says why.

report.Status is AlreadyPlayable (copied as it is), Made, NeedsAudioTranscode (only from the core’s Media.MakePlayable, which has no transcoder), Refused or Failed. The report also holds the verdicts before and after (Before, After), the remux report (Remux) and each audio transcode (AudioTranscodes). The output is judged by opening it and asking CanPlay again; one that still doesn’t play is deleted and reported as Failed.

The audio target is Opus only

From the project’s MakePlayable page: “Opus is the one target; profiles that play neither Opus nor what the file has (generic-mp4) refuse.” The encode is “Opus, 48 kHz, mono or stereo, 20 ms packets, OneAudio’s Opus encoder”, at 64 kbit/s mono or 96 kbit/s stereo by default (MakePlayableOptions.AudioBitRate), and the input is decoded by OneAudio first: AAC-LC, MP3, Vorbis, FLAC, Opus and AMR, or PCM. AC-3, E-AC-3, ALAC and WMA have no decoder and are refused with transcode.no-decoder; more than two channels with transcode.channels.

Check it with ffprobe

ffprobe -v error -show_entries stream=index,codec_name,sample_rate,channels:format=format_name,duration -of compact sintel-safari.mp4

stream|index=0|codec_name=h264
stream|index=1|codec_name=opus|sample_rate=48000|channels=2
format|format_name=mov,mp4,m4a,3gp,3g2,mj2|duration=52.124000

Checked in headless Chrome

The two written files were served over HTTP with range requests, loaded into a <video> element in headless Chrome 149 on Linux and played for 2.5 s, with the audio level measured through the Web Audio API:

file               canPlayType  readyState  duration  played to  frames  audio peak
sintel-safari.mp4  probably     4           52.124 s  2.52 s     65      0.092
sintel-chrome.mp4  probably     4           52.208 s  2.51 s     64      0.088

Both play, with audio. sintel-safari.mp4 was made for the safari profile; Chrome plays H.264 with Opus in MP4 too, which is what this check shows. It was not run in Safari for this page: the safari profile rests on the project’s own check in Playwright’s macOS WebKit build, and MakePlayable’s written outputs played in Chromium 140, Firefox 141 and WebKit 26 in the project’s runs.

Limits

• No video re-encoding. A video codec, profile, level, bit depth or chroma format the browser lacks is a VideoBlocker; the file is refused.
• Opus only, as above. An Opus track can make the audio up to one packet (20 ms) plus the encoder’s look-ahead longer than the input; the delay is written in the file so players line it up.
• Downloaded files. The goal is the progressive verdict (a URL in a <video> element). Fragmented output for Media Source Extensions is packaging, which is not in this release.
• MKV input is remuxed with the same rules as Convert MKV to MP4, including its 0.3.0-preview.1 limit: a Matroska video with B-frames comes out one or two frames short at the end in players that follow the MP4 edit list (Chrome reports 52.124 s for sintel-safari.mp4 against 52.208 s for the source).
• AVI with H.264. An AVI that stores H.264 as avc1, as ffmpeg -c copy from an MP4 writes it, comes out without its video track in 0.3.0-preview.1, and the status is still Made. Check report.Actions for a DropTrack action before using the output.
• The output must not exist; MakePlayableOptions.Overwrite replaces it. The CLI form is onemedia playable in out --profile safari.

To see the verdict without writing anything, see Check if a browser can play a video.