Read an HLS (m3u8) stream in C#

Media.Open("master.m3u8") opens an HLS playlist as one media file: the segments of one variant are read in order, with continuous times, and give the same tracks and packets as any other file. A DASH manifest.mpd opens the same way. For a playlist on a web server, Media.OpenPresentationAsync(url, resolver) reads it through a resolver you write, so you choose the HTTP client. Reading HLS and DASH is new in 0.7.0-preview.1 and is a preview.

Source limits. From the README: “Sources: live DASH is not reloaded; HLS subtitle renditions are not read; a segment is held whole in memory, up to MediaLimits.MaxSegmentBytes (256 MiB); segments under EXT-X-KEY or ContentProtection are reported Protected and not decrypted; a progressive MP4 with mdat first cannot be read until it is finished.”

The presentation here is our own: the Sintel trailer (© copyright Blender Foundation, durian.blender.org, CC BY 3.0) packaged as HLS by onemedia package, as in Convert MP4 to HLS. It has one video variant, an audio group and six CMAF segments per track.

dotnet new console -o ReadHls && cd ReadHls
dotnet add package RecursiveAI.OneMedia --version 0.7.0-preview.1
dotnet tool install --global RecursiveAI.OneMedia.Tool --version 0.7.0-preview.1
curl -L -o sintel.mp4 https://download.blender.org/durian/trailer/sintel_trailer-480p.mp4
onemedia package sintel.mp4 hls --hls > /dev/null

Put this in Program.cs and run dotnet run:

using RecursiveAI.OneMedia;
 
// A local HLS playlist (or DASH manifest) opens like any media file.
using MediaFile media = Media.Open("hls/master.m3u8");
Console.WriteLine($"{media.Format}, duration {media.Duration:mm\\:ss\\.fff}, protected: {media.IsProtected}");
 
foreach (MediaTrack track in media.Tracks)
{
    int packets = 0, keyframes = 0;
    long bytes = 0;
    MediaTime first = MediaTime.Unknown, end = MediaTime.Zero;
    foreach (OwnedPacket packet in media.Packets(track))
    {
        using (packet)
        {
            packets++;
            bytes += packet.Info.Size;
            if (packet.Info.IsKeyframe) keyframes++;
            if (!first.IsKnown || packet.Info.Pts.CompareTo(first) < 0) first = packet.Info.Pts;
            MediaTime packetEnd = packet.Info.Pts + packet.Info.Duration;
            if (packetEnd.CompareTo(end) > 0) end = packetEnd;
        }
    }
    string what = track.Video is { } v ? $"{v.Width}x{v.Height}"
                : track.Audio is { } a ? $"{a.SampleRate} Hz, {a.Channels} ch" : "";
    Console.WriteLine($"track {track.Index}: {track.Kind} {track.Codec} {what} [{track.Rfc6381Codec()}]");
    Console.WriteLine($"  {packets} packets ({keyframes} keyframes), {bytes:N0} bytes, " +
                      $"{(end - first).ToTimeSpan().TotalSeconds:0.###} s");
}

hls, duration 00:51.000, protected: False
track 0: Video h264 854x480 [avc1.64001E]
  1253 packets (11 keyframes), 3,510,198 bytes, 52.208 s
track 1: Audio aac 48000 Hz, 2 ch [mp4a.40.2]
  2435 packets (2435 keyframes), 822,671 bytes, 51.947 s

Every packet of the MP4 comes back: 1,253 video and 2,435 audio packets, the counts ffprobe gives for the trailer, with the times the segments state. The packets span 52.208 s of video. Duration says 51 s: in 0.7.0-preview.1 it adds up the playlist’s EXTINF lengths rounded to whole seconds each (9.33 + 7.17 + 7.29 + 8.38 + 10.42 + 9.63 s become 51), so take the length from the packets as here. A DASH manifest’s Duration is its stated mediaPresentationDuration, 52.208 s for the same segments.

Over HTTP

The library has no network code: PresentationSource.OpenAsync asks a resolver for the playlist and each segment, and the resolver fetches them. (Media.OpenPresentationAsync(url, resolver) does the same and returns a MediaFile; the demuxer used here also lists the variants.) This one uses HttpClient and counts what it fetched. To try it, serve the hls folder with a local web server (Python’s, here), standing in for a CDN:

python3 -m http.server 8000 --directory hls > /dev/null 2>&1 &
SERVER=$!
sleep 1

using RecursiveAI.OneMedia;
using RecursiveAI.OneMedia.Streaming.Sources;
 
var url = new Uri(args.Length > 0 ? args[0] : "http://localhost:8000/master.m3u8");
using var http = new HttpClient();
int requests = 0;
long fetched = 0;
 
// Each playlist and segment is fetched once, whole; the reader keeps at most two open.
var resolver = new DelegatePresentationResolver(async (location, ct) =>
{
    byte[] bytes = await http.GetByteArrayAsync(location, ct);
    requests++;
    fetched += bytes.Length;
    return new MemoryMediaSource(bytes);
});
 
await using PresentationDemuxer hls = await PresentationSource.OpenAsync(url, resolver);
foreach (PresentationVariant v in hls.Variants)
    Console.WriteLine($"variant {v.Index}: {v.Bandwidth:N0} bit/s {v.Width}x{v.Height} {v.Codecs}");
Console.WriteLine($"opened with {requests} requests, {fetched:N0} bytes; protected: {hls.IsProtected}, live: {hls.IsLive}");
foreach (MediaTrack track in hls.Tracks)
    Console.WriteLine($"track {track.Index}: {track.Kind} {track.Codec} [{track.Rfc6381Codec()}]");
 
var packets = new int[hls.Tracks.Count];
while (await hls.ReadPacketAsync())
    packets[hls.Current.Info.TrackIndex]++;
Console.WriteLine($"read {string.Join(" + ", packets)} packets with {requests} requests, {fetched:N0} bytes");

dotnet run
kill $SERVER

variant 0: 670,058 bit/s 854x480 avc1.64001E,mp4a.40.2
opened with 7 requests, 777,930 bytes; protected: False, live: False
track 0: Video h264 [avc1.64001E]
track 1: Audio aac [mp4a.40.2]
read 1253 + 2435 packets with 17 requests, 4,370,916 bytes

The variant’s bit rate is the playlist’s AVERAGE-BANDWIDTH when it states one, as here. Opening reads the multivariant playlist, the two media playlists and the first segment of each track with its initialization segment, 7 requests, not the whole presentation. Reading then fetches each segment once, the video and audio interleaved by decode time. PresentationOptions.VariantIndex picks another variant of a ladder (the default is the first), and MediaLimits.MaxSegmentBytes bounds the memory a segment can take.

Limits

• A preview. From the README: “Sources: live DASH is not reloaded; HLS subtitle renditions are not read; a segment is held whole in memory, up to MediaLimits.MaxSegmentBytes (256 MiB); segments under EXT-X-KEY or ContentProtection are reported Protected and not decrypted; a progressive MP4 with mdat first cannot be read until it is finished.” A protected presentation lists its tracks and delivers no packets.
• Duration of an HLS playlist. In 0.7.0-preview.1 Duration rounds each segment to a whole second (above); the packet times are exact.
• Times. Packet times are the ones the segments state, not re-timed; after an EXT-X-DISCONTINUITY they continue where the previous segment ended and the first packet is flagged. A seek lands at the start of the segment that holds the target (HLS and DASH as sources).
• Your resolver, your network. Local files go through FilePresentationResolver, which refuses any location outside the playlist’s folder; HTTP, retries, headers and caching are up to the resolver you pass.
• It decodes nothing. The packets are compressed video and audio, as in the segments. To put them back into one file, remux the playlist: Media.Remux("hls/master.m3u8", "sintel-again.mp4") wrote this presentation’s 1,253 and 2,435 packets into an MP4 whose packets ffprobe lists with the same times, flags, sizes and MD5s as the original’s (Convert MKV to MP4).