Invalidating CDN Cache on Episode Republish

This page sits beneath CDN Distribution and Cache Invalidation within the broader Media Delivery & Publishing Automation layer, and it solves one sharp, common failure: you re-encoded or re-tagged an already-published episode, the pipeline reports success, and yet a large fraction of listeners keep downloading the old file for hours. The audio has a typo in the intro you just fixed, or the ID3 title is wrong, or the loudness was off — and the edge is happily serving the stale bytes it cached before you ever touched the object.

Problem Framing

The symptom is a support queue that contradicts your deploy log. Your republish job finished cleanly, the origin object is correct, but the complaints keep coming and your own spot-check is inconsistent depending on which POP you hit. The tell is in the edge response headers:

$ curl -sI https://cdn.example.com/audio/ep-0912.mp3 | grep -i 'x-cache\|age\|etag'
x-cache: HIT, HIT
age: 74213                 # ~20 hours old — cached before the re-encode
etag: "3f1c8a02e9d1"       # the OLD object; origin now has "7be4…"

An Age of 74,213 seconds on an object you republished an hour ago is the whole story: the edge is serving a copy it fetched long before your fix, and its TTL has not expired. Nothing about a successful origin write invalidates an already-cached edge copy — a CDN only re-checks origin when the cached object goes stale or is explicitly purged. Because the URL audio/ep-0912.mp3 never changed, the edge has no way to know the bytes behind it did.

There are exactly two ways this goes wrong on republish, and they need opposite treatment:

  • You kept the URL. A same-URL overwrite (the enclosure ep-0912.mp3, the manifest, the feed) leaves a cached copy at the edge that is now wrong. This must be purged.
  • You changed the URL. If the republished object is content-hashed, its bytes changed so its URL changed, and the stale copy is now orphaned — nothing requests it. This must not be purged; there is nothing to fix.

The mistake that produces the stuck-listener incident is treating a same-URL overwrite as if hashing would save you, or blindly purging everything (including immutable media) and collapsing your cache hit ratio for no benefit.

Solution Architecture

Before issuing a single purge, classify what actually changed. Content-hashed media that got new URLs needs no action. The stable-URL objects — the feed, the playlists, and any enclosure you deliberately overwrote in place — are the purge targets, and they should be invalidated together by a shared surrogate key so the fix is atomic from a listener’s point of view.

Decision flow for invalidating CDN cache after an episode republish An episode republish event feeds a classify step that splits changed assets into three cases. Re-encoded segments and hashed enclosures changed their content-hashed URL, so the old URL ages out and no purge is needed. Manifests and playlists keep a stable URL but now point to new segments, so they are soft-purged by the episode surrogate key. A same-URL enclosure overwrite keeps its URL while its bytes changed, so it is soft-purged and then verified at the edge. The classify step routes each changed asset to exactly one of these three outcomes. Episode republish re-encode or re-tag Classify changed assets did the URL change? Re-encoded media content-hashed URL changed Manifest / playlist stable URL, now points to new segments Same-URL enclosure byte overwrite, URL kept No purge needed old URL orphaned, ages out at zero cost Soft purge ep-<id> manifest + sprites, one key, atomic Soft purge + verify confirm fresh ETag at the edge

The lasting fix for the same-URL case is to stop overwriting in place at all: give re-encoded audio a content-hashed URL and let the feed point at the new URL, so the republish is just a new immutable object plus a feed purge. But when an enclosure URL is already public and you cannot change it — podcast directories have cached it, listeners have subscribed to it — a same-URL overwrite followed by a purge is the only path, and it must be done deliberately.

Implementation

The script below classifies the republished assets, purges only the stable-URL group by surrogate key using soft purge, and returns the purge id for logging. It uses the official Fastly client and is safe to re-run — a purge is idempotent.

# fastly==13.1.0   tenacity==8.2.3   python 3.10+
import os
import fastly
from fastly.api import purge_api
from tenacity import retry, stop_after_attempt, wait_exponential_jitter, retry_if_exception_type

SERVICE_ID = os.environ["FASTLY_SERVICE_ID"]

def _purge_api() -> purge_api.PurgeApi:
    cfg = fastly.Configuration()
    cfg.api_token = os.environ["FASTLY_API_TOKEN"]   # token scoped to purge only
    return purge_api.PurgeApi(fastly.ApiClient(cfg))

@retry(
    retry=retry_if_exception_type(fastly.ApiException),
    wait=wait_exponential_jitter(initial=1, max=30),   # backoff on HTTP 429 / 5xx
    stop=stop_after_attempt(5),
    reraise=True,
)
def soft_purge_keys(keys: list[str]) -> dict:
    """Soft-purge up to 256 surrogate keys in one atomic batch request."""
    api = _purge_api()
    resp = api.bulk_purge_tag(
        service_id=SERVICE_ID,
        surrogate_key=" ".join(keys),   # space-separated, header-style
        fastly_soft_purge=1,            # mark stale, don't hard-evict — no stampede
    )
    return resp.to_dict()

def republish_purge(episode_id: str, show_id: str, url_changed: bool) -> dict | None:
    """
    Purge only what a republish actually invalidated.

    url_changed=True  -> media got new content-hashed URLs; purge nothing but
                         still refresh the feed/manifest that now points at them.
    url_changed=False -> a same-URL enclosure was overwritten in place; that
                         object AND the feed must be invalidated.
    """
    keys = [f"show-{show_id}", f"ep-{episode_id}"]   # feed + episode group
    result = soft_purge_keys(keys)
    return {"purged_keys": keys, "url_changed": url_changed, "fastly": result}

The load-bearing parameters:

  • fastly_soft_purge=1 marks the object stale rather than evicting it. The first request after the purge is served instantly from the stale copy while the edge revalidates against origin — no latency spike, no origin stampede on a popular back-catalogue episode.
  • surrogate_key is space-separated so a single bulk_purge_tag call invalidates the show feed and the episode group together, atomically, in one of your limited purge slots.
  • Purging both show-<id> (the feed) and ep-<id> (the manifest and any in-place enclosure) is correct even when the URL did change: the media is untouched, but the feed still needs to advertise the corrected metadata. Purging the immutable segments is neither needed nor done — they are not tagged for this group’s revalidation because their URLs already moved.

Wrap the call in tenacity so a transient HTTP 429 — surrogate-key purges are limited to an average of 100,000 per hour per customer — is retried with backoff rather than dropped. Full production ordering (durable origin write, then purge, then warm) and the per-asset cache policy this relies on live in the parent CDN distribution and cache invalidation workflow. See the official Fastly purging reference for the current API surface and limits.

Verification

Prove the edge is serving the corrected object across POPs before you tell anyone the incident is closed.

# 1. Capture the NEW origin ETag you expect the edge to converge on.
curl -sI "https://media-origin-prod.s3.example.com/audio/ep-0912.mp3" \
  | grep -i etag
# etag: "7be4d9c1a0f2"   <- the corrected object

# 2. Run the purge, then re-probe the edge. Age must reset toward 0 and the
#    ETag must match origin.
python -c "import republish; print(republish.republish_purge('0912','hn', url_changed=False))"
curl -sI "https://cdn.example.com/audio/ep-0912.mp3" | grep -i 'x-cache\|age\|etag'
# x-cache: MISS, MISS   age: 0   etag: "7be4d9c1a0f2"   <- fresh, matches origin

# 3. Check a second POP (Fastly-Debug surfaces the serving node) to confirm
#    propagation isn't a single-POP fluke.
curl -sI -H "Fastly-Debug: 1" "https://cdn.example.com/audio/ep-0912.mp3" \
  | grep -i 'x-served-by\|etag'
# x-served-by: cache-lhr-... , cache-jfk-...   etag: "7be4d9c1a0f2"

A healthy result is an Age at or near zero, an ETag that matches the new origin object, and the same fresh ETag from more than one serving node. If Age is still large after the purge, the purge either targeted the wrong surrogate key or ran before the origin write was durable — re-check the key mapping and the write ordering.

Failure Modes & Edge Cases

Edge case Symptom Remediation
Purged before origin write durable Age resets, then old ETag returns minutes later Confirm the origin PUT succeeded (expected ETag) before purging; re-purge
Purged immutable segments too Cache hit ratio dips, origin egress spikes, no listener benefit Purge only show-<id> and ep-<id>; never tag content-hashed media for purge
Same-URL overwrite, no purge Listeners keep the old audio for the full TTL Soft-purge the enclosure key; better, switch to content-hashed enclosure URLs
Wrong surrogate key on origin object Purge returns ok but edge never refreshes Verify the origin response carries the expected Surrogate-Key header
Hard purge on a popular episode Latency spike and origin stampede at purge time Use fastly_soft_purge=1 so the stale copy serves during revalidation
Podcast app cached the URL upstream App still shows old file after edge is fresh Client-side caches are outside your control; content-hashed URLs are the only guarantee
Purge rate limit hit on bulk re-tag HTTP 429 from the purge API Batch keys with bulk_purge_tag, back off with jitter, spread across the hour

FAQ

If I content-hash every asset, do I ever need to purge on republish?

You still purge the two objects that keep a fixed URL by design: the RSS feed and the HLS/DASH manifest, because clients discover them at a stable address. Everything else — segments, hashed enclosures, artwork — gets a new URL when its bytes change, so the old copy is simply orphaned and never requested again. Full hashing shrinks the purge to those one or two stable documents per show, which is the cheapest and safest possible invalidation.

Why is my re-encoded episode still stale after a successful purge?

Almost always an ordering bug: the purge ran before the corrected object was durable at origin, so a request between the purge and the write refilled the edge from the old bytes. A CDN purge only tells the edge to re-fetch on the next request — if origin still has the old object at that instant, the edge re-caches it. Confirm the origin PUT returned the new ETag, then purge, then verify. Re-issuing the purge is safe because it is idempotent.

Should I purge a specific URL or a surrogate key?

Prefer the surrogate key. A single ep-<id> key can group the manifest, an in-place enclosure, and any thumbnail sprite, so one call invalidates the whole episode atomically instead of racing several single-URL purges. Key-based purges are also batchable up to 256 keys per request, which matters when you re-tag a back catalogue and would otherwise blow through the hourly purge budget one URL at a time.