Adaptive Bitrate Packaging

Within the Media Delivery & Publishing Automation layer, adaptive bitrate packaging is the stage that turns a single normalized mezzanine asset into a set of time-aligned renditions and the manifests that let a player switch between them mid-stream. It protects a concrete delivery SLA: every published asset must expose a valid HLS master playlist and, where required, an MPEG-DASH MPD, whose declared codecs, resolutions, and segment durations exactly match the bytes on disk — so a player negotiating bandwidth never stalls on a missing segment, a mislabeled CODECS string, or a variant whose timeline drifts out of alignment with its siblings. Packaging is deterministic transformation, not creative encoding: known ladder in, byte-identical segment set and conformant manifests out, ready for the CDN.

This component sits immediately after loudness and codec conditioning and before distribution. It consumes a single conditioned source — the output of audio codec normalization workflows muxed with a conformed video stream — and emits a directory tree of CMAF segments, HLS playlists, an optional DASH manifest, and a machine-readable packaging record that the CDN distribution and cache invalidation stage keys its cache lifecycle against.

Prerequisites & Environment

Pin every tool. Packaging output is only reproducible if the encoder and the fragmenter are byte-stable across the fleet; an unpinned FFmpeg or Bento4 build will emit subtly different segment boundaries and break cross-rendition alignment.

• Python: 3.10+ (the orchestration examples use tomllib and structural pattern matching).
• FFmpeg: 6.x (6.0 or 6.1), built with libx264 and the native aac encoder. Confirm with ffmpeg -version; verify the fMP4 HLS muxer is present with ffmpeg -h muxer=hls | grep hls_segment_type.
• Bento4: the mp4fragment, mp4hls, mp4dash, and mp4info tools from a pinned Bento4 build (1.6.0-641 or newer). These do the fragmenting, HLS/CMAF assembly, and DASH packaging that FFmpeg alone does less cleanly.
• Apple HLS tools (optional but recommended): mediastreamvalidator for conformance checks against the HLS authoring specification.
• Python libraries (pinned):

# ffmpeg-python==0.2.0   # thin subprocess wrapper around the FFmpeg CLI
# pydantic==2.7.1        # validates the ladder and packaging-record contracts
# tomli-w==1.0.0         # writes back the resolved ladder for the audit record

• Hardware: encoding the ladder is CPU-bound and embarrassingly parallel across rungs — budget one physical core per concurrent rung and 400–700 MB RAM per libx264 job at 1080p. Fragmenting and playlist writing are I/O-bound and cheap. No GPU is required, though an NVENC-capable ladder can be substituted where determinism across driver versions is acceptable.
• Environment variables the packaging worker reads at startup:

ABR_SEGMENT_DURATION=6            # target segment length in seconds (GOP-aligned)
ABR_LADDER_CONFIG=/etc/mediapipe/ladder.toml
ABR_FFMPEG_BIN=/usr/bin/ffmpeg
ABR_BENTO4_DIR=/opt/bento4/bin    # dir holding mp4fragment, mp4hls, mp4dash
ABR_HLS_VERSION=7                 # 7 = fMP4 (CMAF) segments required
ABR_OUTPUT_PREFIX=s3://media-delivery/hls
ABR_TMP_DIR=/var/lib/mediapipe/pkg
ABR_WORKERS=6                     # rung encoders per host

Segment duration is a delivery decision, not a code constant: 6 seconds is the common VOD default, 4 seconds trades cache efficiency for finer seeking, and low-latency configurations push toward 2 seconds with partial segments. The full tradeoff is worked through in tuning HLS segment duration for seek latency; bake the chosen value into ABR_SEGMENT_DURATION and drive both the encoder GOP and the segmenter from it so they can never diverge.

Architecture & Queue Topology

The packaging worker is a fan-out batch job. A single conditioned mezzanine is pulled from the packaging queue, encoded into N renditions in parallel, and the renditions converge into a segmenter that produces one CMAF track per rung. A playlist writer then emits the variant playlists and the master playlist (and the MPD for DASH), a manifest validator asserts conformance, and only a passing package is promoted to object storage. A failed validation diverts the whole asset to quarantine rather than publishing a half-written manifest — a partial master playlist referencing a segment that does not exist is the single most common cause of mid-stream player stalls.

The worker is stateless per asset: the ladder definition and segment duration are the only configuration, and every rung is a pure function of the mezzanine plus its rung parameters. That makes the whole job idempotent and safely retryable, and it lets the encoder fan-out ride the same Celery task routing for video jobs machinery the rest of the pipeline uses.

Step-by-Step Implementation

Each step ends with a verification command you can run before promoting the package.

1. Define the bitrate ladder as a versioned contract

The ladder is the spec the whole stage is measured against, so it lives in configuration, not code. Represent each rung as an explicit tuple of resolution, target bitrate, maxrate, bufsize, and codec profile so the encoder is fully determined by the contract.

# pydantic==2.7.1
from pydantic import BaseModel, field_validator

class Rung(BaseModel):
    name: str
    width: int
    height: int
    v_bitrate_k: int     # target video bitrate, kbps
    maxrate_k: int       # VBV ceiling, ~1.07x target
    bufsize_k: int       # VBV buffer, ~1.5x target
    a_bitrate_k: int = 128
    profile: str = "high"
    level: str = "4.0"

    @field_validator("width", "height")
    @classmethod
    def even(cls, v: int) -> int:
        if v % 2:
            raise ValueError("H.264 requires even dimensions")
        return v

LADDER = [
    Rung(name="1080p", width=1920, height=1080, v_bitrate_k=7800, maxrate_k=8346, bufsize_k=11700, level="4.0"),
    Rung(name="720p",  width=1280, height=720,  v_bitrate_k=4500, maxrate_k=4815, bufsize_k=6750,  level="3.1"),
    Rung(name="540p",  width=960,  height=540,  v_bitrate_k=2000, maxrate_k=2140, bufsize_k=3000,  level="3.1"),
    Rung(name="360p",  width=640,  height=360,  v_bitrate_k=730,  maxrate_k=781,  bufsize_k=1095,  level="3.0", a_bitrate_k=96),
]

Verify the ladder is monotonic and every rung has even dimensions before spending any compute:

python -c "import packaging; ok=all(a.v_bitrate_k>b.v_bitrate_k for a,b in zip(packaging.LADDER, packaging.LADDER[1:])); print('ladder monotonic:', ok)"

2. Encode each rung with keyframe-aligned FFmpeg jobs

Every rung must place an IDR keyframe on the same wall-clock boundary so that segments line up across renditions — a player that switches from 720p to 1080p must find both streams cut at the identical instant. Force this by disabling scene-cut keyframes (-sc_threshold 0), fixing the GOP to fps × segment_duration, and pinning key frames with -force_key_frames. This alignment is the single most important correctness property of the stage; segment-duration math and its consequences are covered in depth in tuning HLS segment duration for seek latency.

# python 3.10+
import subprocess

def encode_rung(src: str, rung, seg: int = 6, fps: int = 30) -> str:
    gop = fps * seg                       # e.g. 30 * 6 = 180 frames
    dst = f"{rung.name}.mp4"
    cmd = [
        "ffmpeg", "-y", "-hide_banner", "-i", src,
        "-c:v", "libx264", "-preset", "slow",
        "-profile:v", rung.profile, "-level", rung.level,
        "-b:v", f"{rung.v_bitrate_k}k",
        "-maxrate", f"{rung.maxrate_k}k",
        "-bufsize", f"{rung.bufsize_k}k",
        "-vf", f"scale={rung.width}:{rung.height}",
        "-r", str(fps),
        "-g", str(gop), "-keyint_min", str(gop), "-sc_threshold", "0",
        "-force_key_frames", f"expr:gte(t,n_forced*{seg})",
        "-c:a", "aac", "-b:a", f"{rung.a_bitrate_k}k", "-ac", "2", "-ar", "48000",
        "-movflags", "+faststart",
        dst,
    ]
    subprocess.run(cmd, check=True, timeout=3600)
    return dst

Verify the keyframe cadence landed exactly on the segment grid — every keyframe timestamp should be an integer multiple of the segment duration:

ffprobe -v error -select_streams v:0 -skip_frame nokey \
  -show_entries frame=pts_time -of csv=p=0 720p.mp4 | head
# expect 0.000000, 6.000000, 12.000000, ...

3. Fragment and segment the renditions into CMAF fMP4

FFmpeg produces progressive MP4s; Bento4’s mp4fragment rewrites each into a fragmented MP4 whose fragment boundaries fall on the keyframes from step 2. Fragmenting first is what makes the later HLS and DASH packaging share one identical CMAF track set instead of duplicating segments per protocol.

# python 3.10+
import os, subprocess

def fragment(src: str, bento4_dir: str, seg_ms: int = 6000) -> str:
    dst = src.replace(".mp4", ".frag.mp4")
    subprocess.run(
        [os.path.join(bento4_dir, "mp4fragment"),
         "--fragment-duration", str(seg_ms), src, dst],
        check=True, timeout=600,
    )
    return dst

Verify the fragment index is present and the fragment duration matches the target:

/opt/bento4/bin/mp4info --format json 720p.frag.mp4 | \
  jq '.movie.fragments_info // "not fragmented"'

4. Write master and variant playlists and the MPD

With every rung fragmented on the same grid, mp4hls assembles a CMAF HLS package — one media playlist per rung plus a master playlist whose #EXT-X-STREAM-INF lines carry the correct BANDWIDTH, AVERAGE-BANDWIDTH, RESOLUTION, and CODECS. Passing --hls-version=7 selects fMP4 segments; mp4dash produces the parallel MPD from the same fragmented inputs.

# python 3.10+
import os, subprocess

def package_hls(fragments: list[str], out_dir: str, bento4_dir: str) -> str:
    subprocess.run(
        [os.path.join(bento4_dir, "mp4hls"),
         "--hls-version=7",                 # fMP4 / CMAF segments
         "--output-single-file",            # byte-range segments per rung
         f"--output-dir={out_dir}",
         "--master-playlist-name=master.m3u8",
         *fragments],
        check=True, timeout=600,
    )
    return os.path.join(out_dir, "master.m3u8")

def package_dash(fragments: list[str], out_dir: str, bento4_dir: str) -> str:
    subprocess.run(
        [os.path.join(bento4_dir, "mp4dash"),
         "--use-segment-timeline",
         f"--output-dir={out_dir}", *fragments],
        check=True, timeout=600,
    )
    return os.path.join(out_dir, "stream.mpd")

Verify the master playlist enumerates every rung and declares fMP4 segments:

grep -c "EXT-X-STREAM-INF" out/master.m3u8      # expect one per rung
grep -q "EXT-X-MAP:URI" out/*/media.m3u8 && echo "fMP4 init segment present"

5. Validate manifests and promote to object storage

Never publish an unvalidated package. Run Apple’s mediastreamvalidator against the master playlist to catch codec-string mismatches, missing segments, and target-duration violations, then persist a packaging record and atomically promote the tree to object storage. The record is the idempotency key and the input to invalidating CDN cache on episode republish.

# python 3.10+
import hashlib, json, pathlib, subprocess

def validate_and_record(master: str, out_dir: str) -> dict:
    proc = subprocess.run(
        ["mediastreamvalidator", master, "--timeout", "60"],
        capture_output=True, text=True, timeout=180,
    )
    if proc.returncode != 0:
        raise ValueError(f"HLS validation failed: {proc.stderr[-400:]}")
    tree = sorted(pathlib.Path(out_dir).rglob("*"))
    digest = hashlib.sha256(
        b"".join(p.read_bytes() for p in tree if p.is_file())
    ).hexdigest()
    record = {"sha256": digest, "master": "master.m3u8",
              "segment_type": "fmp4", "schema_version": 1}
    pathlib.Path(out_dir, "package.json").write_text(json.dumps(record, sort_keys=True))
    return record

Verify the record and confirm the validator reported no errors:

jq -e '.sha256 and .segment_type == "fmp4"' out/package.json
mediastreamvalidator out/master.m3u8 2>&1 | grep -i "error" || echo "no errors"

Data Contracts

Two contracts govern this stage: the ladder that determines the encoders, and the master-playlist variant record that the player and the CDN read. Any deviation halts the job rather than shipping a manifest that lies about its bytes.

The output contract is the package tree plus its package.json: the sha256 over all segments and playlists (idempotency key), the segment type, and the master-playlist filename. Downstream distribution trusts this record instead of re-reading the manifests.

Resilience Patterns

Packaging jobs are pure functions of the mezzanine plus the ladder, which makes failure handling tractable.

• Idempotency. The job key is sha256(mezzanine) + ladder_version + segment_duration. A retried job that finds a matching package.json short-circuits and returns the existing tree — no re-encode of an already-published asset.
• Rung isolation. Each rung encodes independently, so a single failed rung (an out-of-memory libx264 kill, say) is retried alone rather than restarting the whole ladder. The master playlist is written only after every rung reports success.
• Retry policy. Distinguish deterministic failures from transient ones: a malformed FFmpeg filter, an impossible level, or a validation error is non-retryable and goes straight to quarantine; an OOM kill, a timeout, or an object-store 5xx is retryable with exponential backoff (roughly 5s, 20s, 60s), capped at three attempts, reusing the shared retry logic and dead-letter queues.
• Atomic promotion. Write the whole tree to a temp prefix under ABR_TMP_DIR and promote by renaming (or copying then swapping a pointer) only after validation passes, so a crashed playlist writer never leaves a master playlist that references a segment still being uploaded.
• Manifest-first invariant. The manifest is published last. Every segment it names must already exist in the object store, which is the property the manifest validator exists to guarantee.

Observability & Debugging

Instrument the worker so a broken package is caught before a player ever requests it.

• Key metrics (Prometheus):
  • abr_package_duration_seconds — histogram, labeled by rung_count and segment_duration.
  • abr_rung_encode_seconds — histogram, labeled by rung; the tallest rung dominates wall-clock time.
  • abr_validation_failed_total — counter, labeled by reason (codec_mismatch, missing_segment, target_duration).
  • abr_segment_count — gauge per package; a sudden change at fixed duration signals a keyframe-alignment regression.
• Structured log fields: asset_id, sha256, ladder_version, segment_duration, rung, segment_type, ffmpeg_exit, validator_exit, duration_ms.
• Common errors and root causes:
  • Player stalls when switching renditions — keyframes are not aligned across rungs; -sc_threshold 0 was omitted or the GOP does not equal fps × segment_duration.
  • CODECS attribute mismatch from the validator — the avc1.* string in the master playlist does not match the actual profile/level; let mp4hls derive it rather than hand-writing it.
  • Oversized #EXT-X-TARGETDURATION — a single long segment (usually a trailing partial) exceeds the declared target; re-check the -force_key_frames expression.

For the authoritative behavior of the fMP4 HLS muxer and the -force_key_frames expression, consult the FFmpeg formats documentation; for the packaging tools, the Bento4 documentation covers mp4hls and mp4dash options in full.

Performance Tuning

• Rung concurrency. Cap concurrent rung encoders at the physical core count; libx264 at -preset slow is heavily threaded already, so oversubscribing cores across rungs only thrashes cache. A ProcessPoolExecutor sized to min(len(LADDER), os.cpu_count()) is the ceiling.
• Preset vs. bitrate. -preset slow is the sweet spot for VOD; veryslow buys ~3–5% bitrate savings for roughly double the CPU and rarely pays off at scale. Reserve it for the top rung only if storage cost dominates.
• Fragment once, package twice. Because HLS and DASH both consume the same mp4fragment output, never re-encode for DASH — feed the identical fragmented tracks to mp4dash. This halves compute versus packaging each protocol from source.
• Segment size vs. object count. Shorter segments multiply object count and per-object overhead in the store and on the CDN; --output-single-file byte-range segmentation collapses each rung to one object, which is often the better tradeoff for VOD. Weigh it against seek behavior in tuning HLS segment duration for seek latency.
• Co-scheduling. The encode phase is CPU-bound and GPU-idle, so it co-schedules cleanly on the same hosts as GPU-bound transcription, with per-job CPU pinning to keep the two workloads from contending.

Frequently Asked Questions

Related

• Media Delivery & Publishing Automation — the parent layer this packaging stage belongs to.
• Packaging HLS with FFmpeg and Bento4 — the concrete end-to-end recipe for a CMAF HLS package.
• Tuning HLS segment duration for seek latency — how to choose the segment length this stage encodes against.
• CDN distribution and cache invalidation — publishes the validated package and manages its cache lifecycle.
• Audio codec normalization workflows — conditions the audio stream this stage muxes into every rung.
• Retry logic and dead-letter queues — the shared failure-routing mechanics for quarantined packages.