Video Container Parsing with Python

Within the Media Ingestion & Format Architecture pipeline, video container parsing is the deterministic gate that reads stream topology, codec tags, and timing metadata out of a file before any worker spends compute on it. It protects a hard pre-flight SLA: every asset that passes this stage carries a validated, machine-readable manifest — stream count, codec names, durations, frame rates, timebases, and seekability — so downstream demuxing, normalization, and transcoding never have to guess at container internals or recover from a half-decoded surprise. Treat the parser as a stateless validation gate: arbitrary uploads in, a contract-bound manifest out, with malformed inputs diverted before they reach an expensive node.

This component sits at the very front of ingestion, immediately after a file lands in object storage and before it is routed to any transformation queue. It decodes no media payload; it only inspects container structure and publishes a manifest that the rest of the pipeline trusts.

Prerequisites & Environment

Pin every dependency. Container parsing rides on FFmpeg’s demuxers through a C-extension binding, so an unpinned FFmpeg build silently changes which malformed files are accepted.

• Python: 3.10+ (the examples use list[...] generics, structural pattern matching, and the match statement).
• FFmpeg: 6.0 or newer. PyAV links against the FFmpeg shared libraries, so the binding version and the system libraries must agree. Verify with python -c "import av; print(av.library_versions)".
• Python libraries (pinned):

# av==12.0.0            # PyAV — Pythonic bindings to libav* for in-process parsing
# pydantic==2.7.1       # validates and freezes the container manifest contract
# ffmpeg-python==0.2.0  # used only for the ffprobe fallback path

• Hardware: parsing is I/O-bound and cheap on CPU — it reads container headers and a small index, not frames. Budget ~50 MB RAM per worker and one logical core per concurrent parse; the bottleneck is object-storage latency, not compute. No GPU is involved at this stage.
• Environment variables the worker reads at startup:

PARSE_PROBE_TIMEOUT=15        # hard ceiling per parse, in seconds
PARSE_MAX_PROBE_BYTES=5242880 # cap libav probesize at 5 MiB so a huge moov can't OOM a worker
PARSE_ALLOWED_FORMATS=mov,mp4,m4a,matroska,webm  # accepted container family allowlist
PARSE_TMP_DIR=/var/lib/mediapipe/tmp
PARSE_ROUTING_QUEUE=ingest.routing            # where valid manifests are published
PARSE_QUARANTINE_QUEUE=ingest.quarantine      # where structural failures are diverted

The choice of binding is workflow-specific and analyzed in depth in FFmpeg vs PyAV for video ingestion: in-process PyAV avoids subprocess fork overhead at high volume, while a subprocess call to ffprobe gives full process isolation for hostile inputs. Bake the decision into configuration so a single worker image can run either path.

Architecture & Queue Topology

The parse worker pulls a storage key from the ingest queue, opens the container in read-only mode with a bounded probe budget, extracts the manifest, and publishes it to the routing queue. Structurally invalid files never reach routing — they are diverted to quarantine and classified by media validation and error routing, which owns the dead-letter taxonomy. Valid manifests fan out: audio-only assets go to audio codec normalization, multi-stream video goes to the demux-and-transcode lanes dispatched by Celery task routing for video jobs, and long-form podcasts are batched by FFmpeg batch processing for podcasts.

The stage is deliberately stateless per asset: the only output is the manifest JSON, the worker holds no cross-asset memory, and the storage key plus a content hash is the idempotency key. That makes every parse safely retryable.

Step-by-Step Implementation

Each step ends with a verification command you can run before promoting the asset to routing.

1. Define the manifest contract

The manifest is the only thing downstream stages see, so freeze it as a strict schema first. Anything libav cannot supply cleanly is rejected here rather than guessed at later.

# pydantic==2.7.1
from typing import Optional
from pydantic import BaseModel, Field

class StreamMetadata(BaseModel):
    index: int
    codec_name: str
    codec_type: str            # "video", "audio", "subtitle", "data"
    time_base: float
    frame_rate: Optional[float] = None
    bit_rate: Optional[int] = None
    disposition: dict = Field(default_factory=dict)

class ContainerManifest(BaseModel):
    file_path: str
    format_name: str
    duration_seconds: float
    stream_count: int
    streams: list[StreamMetadata]
    has_moov: bool = True
    is_seekable: bool = True

Verify the model imports and rejects a missing required field:

python -c "from manifest import StreamMetadata; StreamMetadata(index=0)"  # should raise ValidationError

2. Open the container with a bounded probe budget

Open in read-only mode and cap probesize so a pathological moov atom cannot force libav to buffer the whole file into RAM. The context manager guarantees the file descriptor is released even on a partial read.

# av==12.0.0
import os
import av

PROBE_BYTES = int(os.environ.get("PARSE_MAX_PROBE_BYTES", 5 * 1024 * 1024))

def open_container(filepath: str) -> av.container.InputContainer:
    # options are passed straight to the demuxer; probesize caps header buffering
    return av.open(
        filepath,
        mode="r",
        metadata_encoding="utf-8",
        options={"probesize": str(PROBE_BYTES), "analyzeduration": "5000000"},
    )

Verify the probe budget is honored on a real file:

python -c "import parse; c=parse.open_container('input.mp4'); print(c.format.name); c.close()"

3. Enumerate streams and normalize every field

Walk the streams once, coercing each libav value into the canonical Python type the contract expects. time_base and average_rate are Fraction objects in PyAV; cast them to float so downstream arithmetic never trips over a rational. disposition is an integer bitmask — surface it explicitly as a serializable dict.

# av==12.0.0
def extract_streams(container) -> list["StreamMetadata"]:
    streams = []
    for idx, stream in enumerate(container.streams):
        frame_rate = float(stream.average_rate) if stream.average_rate else None
        streams.append(StreamMetadata(
            index=idx,
            codec_name=stream.codec_context.name,
            codec_type=stream.type,
            time_base=float(stream.time_base),
            frame_rate=frame_rate,
            bit_rate=stream.bit_rate or None,
            disposition={"flags": int(stream.disposition or 0)},
        ))
    return streams

Verify stream enumeration against ground truth from ffprobe:

ffprobe -v error -show_entries stream=index,codec_name,codec_type -of csv input.mp4

4. Resolve duration and prove seekability

container.duration is reported in AV_TIME_BASE units (microseconds); multiply by av.time_base to get seconds. A container that cannot seek to offset zero is unusable for the batch lanes downstream, so detecting non-seekability here — and failing fast — prevents a worker from stalling on a stream it can never rewind.

# av==12.0.0
from av.error import InvalidDataError, FFmpegError

def resolve_timing(container) -> tuple[float, bool]:
    duration_sec = float(container.duration * av.time_base) if container.duration else 0.0
    try:
        container.seek(0)
        seekable = True
    except (FFmpegError, OSError):
        seekable = False
    if not seekable:
        raise InvalidDataError("Non-seekable container; divert to quarantine")
    return duration_sec, seekable

Verify seekability independently — a non-seekable MP4 usually means the moov atom trails the mdat:

ffprobe -v error -show_entries format=duration -of default=nw=1 input.mp4

5. Assemble, validate, and publish the manifest

Compose the pieces into a single ContainerManifest. Pydantic validates the whole structure in one shot; a non-retryable structural failure is logged and re-raised so the queue layer can route it to quarantine.

# python 3.10+
import logging
logger = logging.getLogger(__name__)

MOOV_FORMATS = ("mov", "mp4", "m4a")

def parse_container(filepath: str) -> ContainerManifest:
    try:
        with open_container(filepath) as container:
            streams = extract_streams(container)
            duration_sec, seekable = resolve_timing(container)
            fmt = container.format.name
            return ContainerManifest(
                file_path=filepath,
                format_name=fmt,
                duration_seconds=duration_sec,
                stream_count=len(streams),
                streams=streams,
                has_moov=any(f in fmt for f in MOOV_FORMATS),
                is_seekable=seekable,
            )
    except (InvalidDataError, FFmpegError) as e:
        logger.error("container_parse_failure", extra={"file": filepath, "err": str(e)})
        raise RuntimeError(f"structural validation failed: {e}") from e

Verify the end-to-end manifest is valid JSON and carries the expected stream count:

python -c "import parse,json; print(json.dumps(parse.parse_container('input.mp4').model_dump(), indent=2))"

Data Contracts

The handoff between the parser and the routing layer is governed by a strict schema. Any deviation triggers an immediate halt and a quarantine route rather than a best-effort guess.

The output contract is the ContainerManifest JSON published to PARSE_ROUTING_QUEUE. Downstream stages trust the declared codec_name and codec_type instead of re-probing the file — that single source of truth is what lets normalization and transcoding run without their own header inspection.

Resilience Patterns

Parsing is a pure function of its input bytes, which makes resilience easy to reason about.

• Idempotency. The content hash of the source plus the schema version is the job key. A retried parse that finds an existing manifest with a matching hash short-circuits and republishes it — no double work.
• Retry policy. Classify failures, do not blanket-retry. An InvalidDataError or a codec/format reject is deterministic — it will fail identically on retry — so it goes straight to quarantine. A PARSE_PROBE_TIMEOUT breach or a transient object-storage read error is retryable.
• Backoff. Use exponential backoff with jitter, capped at three attempts (roughly 2s, 8s, 30s). A storage 503 during a fan-in spike is the common transient cause and clears within that window.
• DLQ routing. Structurally invalid assets are diverted to PARSE_QUARANTINE_QUEUE and classified by media validation and error routing, so one corrupt upload never blocks the worker pool. The shared dead-letter mechanics — visibility timeouts, replay, poison-message caps — are covered under retry logic and dead-letter queues.
• Partial-state safety. The parser writes nothing but the manifest message; it never mutates the source asset. A crashed worker leaves no half-written artifact, so recovery is simply re-enqueueing the original storage key.

Observability & Debugging

Instrument the worker so a malformed-input spike is visible before it stalls the fan-out.

• Key metrics (Prometheus):
  • parse_duration_ms — histogram, labeled by format_name; the p95 should stay under a few hundred milliseconds since no decoding occurs.
  • parse_failures_total — counter, labeled by reason (invalid_data, timeout, format_rejected, non_seekable).
  • parse_stream_count — histogram, to catch a sudden shift in topology from an upstream encoder change.
  • parse_quarantine_total — counter; a sustained rise points at a bad ingestion source.
• Structured log fields: asset_id, content_hash, format_name, stream_topology, duration_ms, validation_status, ffmpeg_err.
• Common errors and root causes:
  • Invalid data found when processing input — truncated upload or a non-media file; the first 1024 bytes will not match a known container signature. Route to quarantine.
  • moov atom not found — a faststart MP4 was uploaded mid-write, or the moov trails the mdat and the probe budget was exhausted before reaching it. Re-ingest the complete file or raise PARSE_MAX_PROBE_BYTES for that source.
  • Parse hangs near the timeout — a network filesystem is serving the bytes; stream to PARSE_TMP_DIR on local NVMe first so probesize reads do not stall on the wire.

For atom-level debugging, dump the leading bytes and check them against the container’s box signatures: xxd -l 64 input.mp4 should show an ftyp box at offset 4 for an ISO-BMFF file. The authoritative demuxer behavior is documented in the official FFmpeg formats documentation.

Performance Tuning

• Concurrency. Parsing is I/O-bound, so the worker pool can comfortably exceed the core count — size it to roughly 2–4× logical cores and let workers overlap on storage latency. The ceiling is open file descriptors, not CPU.
• File descriptors. Each concurrent parse holds at least one descriptor; set ulimit -n well above the pool size (8192 is a safe floor) so a fan-in burst does not exhaust the table mid-parse.
• Probe budget. probesize and analyzeduration are the two dials that trade accuracy for speed. The 5 MiB / 5 s defaults resolve standard MP4/MKV reliably; only raise them for sources with very large headers, and never remove the ceiling — an unbounded probe is how a single file OOMs a worker.
• Memory. Budget ~50 MB per worker and set a cgroup memory limit so an over-budget probe fails loudly instead of swapping. Because parsing never decodes frames, steady-state memory is flat.
• Process isolation vs. throughput. For untrusted public uploads, the ffprobe subprocess path contains a malicious file inside a short-lived process at the cost of fork overhead; for trusted internal volume, in-process PyAV is faster. The full tradeoff is quantified in FFmpeg vs PyAV for video ingestion.

Frequently Asked Questions

Related

• Media Ingestion & Format Architecture — the parent pipeline this gate belongs to.
• FFmpeg vs PyAV for video ingestion — the binding-selection benchmark for this stage.
• Audio codec normalization workflows — consumes the audio streams this parser isolates.
• Media validation and error routing — classifies the structural failures this gate diverts.
• FFmpeg batch processing for podcasts — the concurrent executor fed by valid manifests.
• Celery task routing for video jobs — dispatches multi-stream assets to demux and transcode lanes.