Mapping Transcript Timestamps to Chapter Markers

This page sits beneath Automated Chapter Generation within the broader Transcription & Speaker Diarization pipeline, and it solves one narrow, mechanical problem: the topic segmenter has told you between which two windows each chapter begins, and you now have to turn that into an exact integer-millisecond start time that lands on a real break in the audio — then write it into a chapters JSON that both the container embed step and the podcast feed will consume verbatim.

Problem Framing

Topic segmentation produces boundaries in transcript space, not audio space. A candidate boundary is an index: “a new chapter starts before window 47.” Window 47’s first word might begin at 184.312 seconds, but that instant is often the middle of a spoken clause — the model detected the topic drift a beat late, or the word that opens the new topic is preceded by a filler word that acoustically belongs to the previous sentence. If you take the raw word start and write it straight into the marker, players jump the listener into the middle of a sentence, and the defect is glaringly obvious the moment anyone taps the chapter.

The symptoms show up in review logs as boundaries that don’t line up with anything audible:

WARN  chapter_snap  ep=142  boundary=47  raw_start_ms=184312  nearest_silence_ms=183090  delta_ms=1222
WARN  chapter_gap   ep=142  chapter=3     start_ms=246500  prev_start_ms=228000  gap_ms=18500  min_gap_ms=60000
INFO  chapter_emit  ep=142  chapters=9    duration_ms=3411000

Two distinct failures hide in those lines. The first is a snapping problem: the raw start is 1,222 ms away from the nearest natural break, so the marker needs to move. The second is a spacing problem: chapter 3 starts only 18.5 seconds after chapter 2, below the 60-second minimum gap, so it should never have become a separate marker at all. Both must be resolved before the JSON is written, because every downstream consumer treats the file as authoritative and will faithfully embed a bad start time.

Solution Architecture

The mapping is a deterministic three-pass transform over the boundary list. First, resolve each boundary index to the specific word that opens the new topic. Second, snap that word’s start time to the nearest natural break — a silence gap or a sentence-final punctuation mark — searching a bounded window around it. Third, walk the snapped starts in order, drop any that fall within the minimum gap of their predecessor, derive each chapter’s end_ms from the following start, and emit the array. Nothing here re-segments or re-titles; it only converts positions to times and enforces spacing.

Working in integer milliseconds end to end is not incidental. Word timestamps often arrive as floating-point seconds, and rounding them inconsistently — floor in one place, round-half-up in another — produces markers that are a frame or two off and, worse, chapters whose end_ms doesn’t exactly equal the next chapter’s start_ms. Convert to int milliseconds once, at the boundary of this stage, and never touch seconds again.

Implementation

The following pass takes the transcript words (each with start_ms, end_ms, optional speaker), the list of boundary word indices from the segmenter, and the configuration, and returns a validated chapter list ready to serialize.

# python 3.10+   pydantic==2.7.1
from dataclasses import dataclass
from pydantic import BaseModel

@dataclass
class Word:
    text: str
    start_ms: int
    end_ms: int
    speaker: str | None = None

class Chapter(BaseModel):
    start_ms: int
    end_ms: int
    title: str
    confidence: float

def to_ms(seconds: float) -> int:
    """Single, consistent seconds-to-ms conversion for the whole stage."""
    return int(round(seconds * 1000))

def snap_start(words: list[Word], idx: int, window_ms: int) -> int:
    """Move words[idx].start_ms to the nearest silence or sentence boundary."""
    target = words[idx].start_ms
    best_ms, best_rank, best_delta = target, 9, window_ms + 1
    lo = max(1, idx - 40)
    hi = min(len(words), idx + 40)
    for i in range(lo, hi):
        prev, cur = words[i - 1], words[i]
        delta = abs(cur.start_ms - target)
        if delta > window_ms:
            continue
        gap = cur.start_ms - prev.end_ms
        rank = (0 if gap >= 400                       # a real pause in speech
                else 1 if prev.text.rstrip().endswith((".", "?", "!"))
                else 9)                               # not a clean break
        # prefer a better break; break ties by proximity to the target
        if rank < best_rank or (rank == best_rank and delta < best_delta):
            best_rank, best_ms, best_delta = rank, cur.start_ms, delta
    return best_ms

def map_boundaries_to_chapters(
    words: list[Word],
    boundaries: list[int],          # word indices where topics change
    titles: list[str],              # one title per boundary segment
    confidences: list[float],
    duration_ms: int,
    snap_window_ms: int = 2500,
    min_gap_ms: int = 60_000,
) -> list[Chapter]:
    # 1. First chapter always starts at 0; snap the rest.
    raw_starts = [0] + [snap_start(words, b, snap_window_ms) for b in boundaries]

    # 2. Enforce the minimum gap, keeping the title/confidence of survivors.
    kept: list[tuple[int, int]] = [(0, 0)]  # (start_ms, source_index)
    for src, s in enumerate(raw_starts[1:], start=1):
        if s - kept[-1][0] >= min_gap_ms:
            kept.append((s, src))

    # 3. Derive end_ms from the next start (or the episode duration) and build.
    chapters: list[Chapter] = []
    for pos, (start_ms, src) in enumerate(kept):
        end_ms = kept[pos + 1][0] if pos + 1 < len(kept) else duration_ms
        chapters.append(Chapter(
            start_ms=start_ms,
            end_ms=end_ms,
            title=titles[src] if src < len(titles) else "Introduction",
            confidence=confidences[src] if src < len(confidences) else 1.0,
        ))
    return chapters

Three parameters carry the behaviour. snap_window_ms (default 2500) bounds how far a boundary may move — wide enough to reach a natural break, narrow enough that a chapter never jumps into a different topic. min_gap_ms (default 60000) is the spacing floor that removes markers too close to matter. The seconds-to-ms conversion lives in exactly one function, to_ms, so rounding is identical everywhere and adjacent chapters tile perfectly.

Serialize the result and hand it to the delivery layer. The container step reads this same array to write chapter atoms into the MP4 — see embedding podcast chapters with mp4chaps — and the feed generator reads it to write chapter tags into the RSS. Both consumers treat the JSON as immutable.

import json, pathlib

def write_chapters(chapters: list[Chapter], dst: str) -> str:
    payload = [c.model_dump() for c in chapters]
    tmp = dst + ".tmp"
    pathlib.Path(tmp).write_text(json.dumps(payload, indent=2))
    pathlib.Path(tmp).replace(dst)   # atomic
    return dst

Verification

Confirm each invariant independently before trusting the file downstream.

# 1. Every start lands on a word boundary and starts are strictly increasing.
jq -e '. == sort_by(.start_ms) and (map(.start_ms) | . == unique)' chapters.json

# 2. Chapters tile the timeline: each end_ms equals the next start_ms.
python -c "
import json
c = json.load(open('chapters.json'))
assert all(c[i]['end_ms'] == c[i+1]['start_ms'] for i in range(len(c)-1)), 'gap/overlap'
assert c[0]['start_ms'] == 0, 'first chapter must start at 0'
print('tiling ok:', len(c), 'chapters')
"

# 3. No two markers are closer than the minimum gap.
jq -e 'reduce range(1; length) as $i (true; . and (.[$i].start_ms - .[$i-1].start_ms) >= 60000)' chapters.json

A healthy run shows strictly increasing integer starts, end_ms values that exactly meet the next start_ms, and no gap below the minimum. If the tiling assertion fails, a floating-point conversion crept in somewhere other than to_ms; if the min-gap assertion fails, the filter ran before snapping instead of after.

Failure Modes & Edge Cases

FAQ

Related

• Up to the parent stage: Automated Chapter Generation
• Pipeline overview: Transcription & Speaker Diarization
• Downstream consumer: Embedding podcast chapters with mp4chaps
• Sibling technique: LLM-assisted chapter titling from transcripts