Remux

Normative description of how OneMedia copies the tracks of one container into another (RecursiveAI.OneMedia.Remuxing, Media.Remux, onemedia remux) and how it moves the movie box of an MP4 to the front (Remuxer.Faststart, Media.Faststart). Muxer rules: docs/muxing.md. Editors that change a file in place: docs/writer.md.

What a remux does

A remux reads packets from any reader and gives them to a muxer. Compressed payload bytes are copied. Three things change, and only these:

1. Framing the target container does not store the way the source did (below).
2. Codec configuration records are built the way the target stores them: avcC/hvcC from the parameter sets in the stream, esds from an AudioSpecificConfig, dOps from OpusHead, dfLa from FLAC STREAMINFO. The muxer does these mappings (CodecConfigMapper, docs/muxing.md sample-entry table); the remux layer only supplies a configuration the source did not store.
3. Times: rescaled exactly by the muxer, with the first decode time of a track moved to zero and the difference put in the edit list; Ogg-style times that already have the priming taken off are moved forward by the priming (below).

Everything the output does not hold is listed in RemuxReport.Losses.

API

RemuxOptions: Faststart (default true), Select and Edits (drop a track; change language, name, disposition, display rotation), AllowDrop (default true), Limits, ConfigurationProbeBytes, CreationTime, Overwrite, MovieTimescale (ISO targets; 0 lets the muxer choose), Observer (every packet as written).

For callers that drive a RemuxSession themselves (trim, split, concatenation): a track's Edit (priming, start time, edit list segments) reaches the muxer unchanged, apart from the Ogg-style priming shift described below; RemuxSession.IsoOptions is the hook for per-track ISO settings (extra sample entry boxes, PCM layout, track id), and Remuxer.IsoTrackOptionsFor builds the sample entry boxes of an ISO input track.

Targets

RemuxFormat: Mp4, M4a, M4v, Mov, ThreeGp, ThreeG2, FragmentedMp4, Cmaf (one track), Matroska, WebM. The ISO family is written by IsoMuxer, Matroska and WebM by MatroskaMuxer (docs/muxing-matroska.md; WebM takes VP8, VP9, AV1, Vorbis and Opus only). Faststart has no meaning for Matroska, whose index is written by the muxer.

What a Matroska or WebM output does not carry is listed: display rotation, AAC priming and edit lists (Timing), ISO sample entry boxes for colour, field order, clean aperture and HDR metadata (CodecField, NotRepresentable), and for WebM video the Colour element and SegmentUID (the validator the muxer is checked with rejects them there). Changes notes that BlockAdditions, interlacing and stereo mode are not written.

Choosing and dropping tracks

Every input track is wanted unless Select returns false or its TrackEdit.Drop is set (reported as Deselected). Tracks that are not wanted are switched off in the demuxer, so their payloads are not read. A wanted track the target cannot hold is refused by the muxer when it is declared; with AllowDrop (default) the track is left out and reported with the muxer's reason mapped to a LossReason:

With AllowDrop false the first such track throws the muxer's MuxRefusedException and nothing useful is written. If every track is left out, Finish throws MuxRefusedException (UnsupportedCodec) naming each track.

Repackaging

Packets are held back only until every track that needs a configuration derived from the stream has it, up to ConfigurationProbeBytes (32 MiB); then they are replayed in order. A track still without one is left out as MissingConfiguration. Memory after that does not depend on the length of the input.

Priming and times

• An edit list, start offset or priming the reader reports goes to the muxer in MediaTrack.Edit; the ISO muxer copies the edit list or builds one from priming and start time (docs/muxing.md).
• Ogg Opus packet times already have the pre-skip taken off (first packet at minus the pre-skip, TrackEditInfo.PacketTimesIncludePriming). The remux moves every time of that track forward by the priming, so the first packet is at zero, clears the flag and keeps the priming in the edit information. The report says so in Changes.
• Transport and program streams start at their first time stamp; the muxer moves the first decode time to zero and writes the difference as an initial empty edit, which keeps presentation time.
• Decode times from presentation order. Matroska stores one time per block, the presentation time, so its video packets arrive without a decode time. The session gives the k-th packet in decode order the k-th smallest presentation time, moved back by one constant lag so that no packet is due before it can be decoded. Both are judged over a window of 24 packets (or 32 MiB), which is held back; the lag is fixed from the first window. Video without reordering gets decode time equal to presentation time. Changes says so once per track.
• Audio in laced Matroska blocks. A laced block carries one time for all its frames, and without a DefaultDuration no frame has a duration. The codec gives each frame's length (the Opus TOC byte, RFC 6716 §3.1; 1536 samples for AC-3, 1024 for AAC, 1152 or 576 for MPEG audio), so a frame that would start before the previous one ended is placed where it ended. Other lacing is left as the reader delivers it.
• PCM layout. Integer or float, width and byte order are not in the neutral track; the WAV, AIFF and CAF readers and FLV's codec name state them and the remux hands them to the ISO muxer (fpcm for floats, ipcm for integers; QuickTime codes in MOV).
• ALAC cookies. CAF and QuickTime store the magic cookie as atoms (frma, alac, terminator) and a bare 24-byte configuration also occurs; the ISO alac box holds the version/flags word and the 24 bytes, so the cookie is unwrapped to that.
• Streams the input did not declare. A packet whose track number is past the reader's track list (a logical stream that begins after the headers, in Ogg) is skipped and reported once as a Track loss with reason NotInspected.
• A raw elementary stream has no time stamps; the reader numbers frames at the declared or default rate and the output keeps that numbering.

What is reported

RemuxReport: the muxer's MuxReport (bytes, packets and payload bytes per output track, its notes), one RemuxTrackReport per input track (output index or -1, packets, repackaging), Losses, Changes and the input's defect count.

Changes lists what the remux did to carried data: configuration derived from the stream, framing changed, times moved. Mux.Notes lists what the muxer rounded or filled in.

Unknown top-level and movie-level boxes of an ISO source (udta payloads outside the tag readers, vendor uuid, meta with other handlers) are not carried by a remux; use Faststart, which copies them, when only the layout should change.

Faststart

Remuxer.Faststart is an edit, not a remux. A progressive MP4/MOV/3GP whose moov follows mdat is rewritten as ftyp, moov, then every other box in its original order. All media data and all other boxes are copied by range, so the output is exactly the input plus one moved box. Only the chunk offset tables change: ISO/IEC 14496-12 §8.7.5 stores absolute file offsets in stco/co64, and every entry that pointed past the new position of the movie box and before the old one moves forward by the movie box's size. A table whose offsets no longer fit 32 bits is promoted to co64, which grows the movie box, which moves the offsets again; the rewrite repeats until the size is stable (rounds are bounded by the number of tables).

• The movie box is held in memory (bounded by MediaLimits.MaxAllocationBytes); media data is streamed through a 1 MiB buffer. Peak memory does not depend on the file size.
• A file whose moov is already before mdat is copied unchanged (Relocated false). Fragmented files (moof, mvex) are refused: there is no single movie box.
• Not adjusted (as in the editors): iloc item locations of HEIF-style files and vendor boxes that hold absolute offsets.
• Remuxer.Remux(…, Faststart = true) (the default) gets the same layout by writing moov first through the muxer's spool; that path rebuilds the sample tables (see docs/muxing.md). Use Faststart when the file should stay exactly as it is.

Differential method

tools/OneMedia.RemuxCheck remux <in> <out> remuxes a file with Remuxer and writes <out>.map (output ← input track, repackaging) and <out>.pkts (every packet as handed to the muxer, with its SHA-256). It also checks that re-framed video keeps its NAL units: the NAL units of the input (parameter sets excluded) hash to the same value as those parsed back out of the length-prefixed samples written (nals=same). tools/remux-compare.py runs ffprobe -show_packets -show_data_hash on both files and compares per stream, in order: pts, dts, duration, size, flags and data hash. It is tools/mux-compare.py (classes in docs/muxing.md) with these differences: a track marked annexb-to-length-prefixed or adts-to-raw is compared by size and hash with what was written (.pkts), because the bytes differ by design, and the class untimed-input covers raw elementary streams, which have no time stamps for ffprobe to print. tools/run-remux-diff.sh adds the structure check (MP4Box -info, mkvalidator for Matroska) and ffmpeg -f null on input and output and counts messages that appear only for the output. tools/run-mux-diff.sh-style summaries are written to summary.tsv.

Differential results

Every corpus file up to 1 GiB remuxed, then packets compared with ffprobe 7.0.2 (tools/remux-compare.py: payload hash after the documented framing change, times after the documented shift, flags), the output checked with MP4Box -info or mkvalidator, and decoded to null with ffmpeg. Files a target cannot hold are refused with a typed reason and counted as not written.

• The 419: avchd_lite_14s_playback_speed_problems.MTS declares a field rate in its SPS (59.96 per second) while the pictures are 3003 ticks apart, so the TS reader's per-packet duration is 1501 ticks where ffprobe reports 3003; the Matroska output keeps the reader's duration. Timestamps and payloads match.
• MP4Box messages new in the output: 3 files (zero stts deltas from repeated input decode times). ffmpeg messages new in the output: 55 files, of which 10 are the cover pictures of tagged MP3 fixtures, now carried into covr (the fixture images are not valid PNG); the rest are ffmpeg's non-monotonic-DTS warnings for index-less inputs and decoder messages on damaged samples.
• mkvalidator messages new in the output: 2 files (h264-aac-errorflag.ts); the input is a TS file mkvalidator cannot read, so its message list is empty and any line counts as new (the line is an empty file "" banner).
• Tag carry spot check (26 files, ExifTool 13.60, MediaInfo 26.10): tags the input had reappear in the output of ID3, CAF and TS service-name inputs; unreadable-to-ExifTool inputs (CAF, TS) show the carried tags.