M1-09 follow-up: make the transcript parts actually assemble #24

Merged

wk merged 7 commits from m1-09-vtt-assembly into main

2026-08-07 14:18:37 -04:00

agent commented

2026-08-07 14:18:13 -04:00

Member

Closes #9, reopened after #23 merged.

What was wrong

#23 claimed assembly was concatenation. It was not. A rendered part ended on its last spoken word
rather than on a blank line, and a WebVTT cue block runs until a blank line — so joining two parts
put the next part's WEBVTT signature inside the previous cue. FFmpeg read it as words that had
been spoken. The signature is also only a signature at the very start of a file.

The close-out comment on #9 said "concatenating the parts yields the full transcript", and the
test behind it checked that every line of text was present. It was present. That is why neither
caught this.

Three more in the same code, from the same Codex review:

Cue text was written raw. A &, a < or a --> in what somebody said could change the line or
truncate it, and a blank line inside a cue ended the cue and took the rest of its own words.
Timestamps rounded the seconds field after splitting off hours and minutes, so 59.9996
rendered 00:00:60.000. WebVTT holds that field to 0...59.
A part that failed to write was dropped rather than retried. TranscriptParts has already
released those cues by then, so a moment of full storage cost a minute of transcript
permanently, with nothing marking the hole.

What changed

Assembly is VTT.assemble, not cat. First part keeps its signature, the rest give theirs up
at assembly. Every part on disk still carries one, because being openable on its own mid-walk is
the only reason parts exist.

Cue text is escaped, so nothing a person says can be read as syntax. Line endings normalise
and a run of them collapses to one — the single thing here that does not round-trip, and it costs
a line break rather than a word.

Timestamps round to milliseconds before they are split, and an end is always at least a
millisecond after its own start.

A failed part is retried at every later boundary and at stop, the written list stays in part
order so a retry cannot land out of sequence, and a stuck part is reported while the recording
runs — storage is the failure that happens here and the one somebody can act on, but only while
there is still a recording for the freed space to take.

The first commit is a separate concern: Wes's rewrite of the development process in AGENTS.md
plus the .githooks that back it, which were sitting uncommitted in the tree. Split out so it is
reviewable apart from the fix.

Validated

Device — wes-iphone, iPhone 17 Pro, iOS 26, one three-minute recording, three parts, 21
cues. Each part opens with its own signature and ends on a blank line. Assembled: one signature,
cue numbers 1–21 unbroken, starts monotonic, every end after its own start, no illegal seconds
field, FFmpeg parses it clean. Both seams exact — part 1 ends 00:01:04.260, part 2 starts
00:01:04.260 — with nothing duplicated or dropped across either. This is also the only check
that the writer is still wired to the transcriber; no unit test can reach that.

Tests — 50, up from 21. Built on a WebVTT conformance reader written from the grammar with no
knowledge of VTT.render, plus a test proving that reader rejects all four pre-fix behaviours, so
the assertions on top of it can actually fail. make lint clean.

Not validated

Escaping, on device. The recogniser transcribed "ampersand, less than, greater than" as
those words rather than the symbols, so no reserved character reached the cue text and the
escaping path never fired. Covered byte for byte by unit tests; not exercised by dictation.
Worth knowing on its own that speech-to-text spells these out.
Storage exhaustion. Filling the phone means hours of recording, so the retry and its
on-screen report are covered against a synthetic block only.
Whether cue times match the words by ear. Open since #8, unchanged.
A recording ended by being killed, and a microphone swap mid-recording.

Codex

Four passes — plan, code, tests, security. Every finding's disposition is in
.agent/m1-09-vtt-assembly/SUMMARY.md. The ones that changed the outcome:

Retry could land parts out of order and reverse the transcript — a real hole in the plan.
A prefix check took WEBVTTX for a signature and deleted the line, losing a cue.
Both retry tests cleared the blockage right before finish(), so a writer that only retried at
stop would have passed them.
onlyCompletedPartsAreVisible could not fail; renamed to what it actually asserts.
Review artifacts carried absolute paths, publishing a local username into a public repository.
Nothing was pushed yet, so the branch was rewritten rather than leaving them in reachable
history, and codex-run.sh now strips them at the source.

Two were declined and both are recorded with reasons: the stash-docs decision file, deferred to
Stash/stash-docs#3 because there is no decisions convention to write into yet, and pre-push
validating the checked-out branch rather than the refs on stdin — real, but Wes's in-flight work
on the process gate rather than anything this issue touches.

Closes #9, reopened after #23 merged. ## What was wrong #23 claimed assembly was concatenation. It was not. A rendered part ended on its last spoken word rather than on a blank line, and a WebVTT cue block runs until a blank line — so joining two parts put the next part's `WEBVTT` signature *inside* the previous cue. FFmpeg read it as words that had been spoken. The signature is also only a signature at the very start of a file. The close-out comment on #9 said "concatenating the parts yields the full transcript", and the test behind it checked that every line of text was *present*. It was present. That is why neither caught this. Three more in the same code, from the same Codex review: - Cue text was written raw. A `&`, a `<` or a `-->` in what somebody said could change the line or truncate it, and a blank line inside a cue ended the cue and took the rest of its own words. - Timestamps rounded the seconds field after splitting off hours and minutes, so `59.9996` rendered `00:00:60.000`. WebVTT holds that field to `0...59`. - A part that failed to write was dropped rather than retried. `TranscriptParts` has already released those cues by then, so a moment of full storage cost a minute of transcript permanently, with nothing marking the hole. ## What changed **Assembly is `VTT.assemble`, not `cat`.** First part keeps its signature, the rest give theirs up at assembly. Every part on disk still carries one, because being openable on its own mid-walk is the only reason parts exist. **Cue text is escaped**, so nothing a person says can be read as syntax. Line endings normalise and a run of them collapses to one — the single thing here that does not round-trip, and it costs a line break rather than a word. **Timestamps round to milliseconds before they are split**, and an end is always at least a millisecond after its own start. **A failed part is retried** at every later boundary and at stop, the written list stays in part order so a retry cannot land out of sequence, and a stuck part is reported *while the recording runs* — storage is the failure that happens here and the one somebody can act on, but only while there is still a recording for the freed space to take. The first commit is a separate concern: Wes's rewrite of the development process in `AGENTS.md` plus the `.githooks` that back it, which were sitting uncommitted in the tree. Split out so it is reviewable apart from the fix. ## Validated **Device** — `wes-iphone`, iPhone 17 Pro, iOS 26, one three-minute recording, three parts, 21 cues. Each part opens with its own signature and ends on a blank line. Assembled: one signature, cue numbers 1–21 unbroken, starts monotonic, every end after its own start, no illegal seconds field, FFmpeg parses it clean. Both seams exact — part 1 ends `00:01:04.260`, part 2 starts `00:01:04.260` — with nothing duplicated or dropped across either. This is also the only check that the writer is still wired to the transcriber; no unit test can reach that. **Tests** — 50, up from 21. Built on a WebVTT conformance reader written from the grammar with no knowledge of `VTT.render`, plus a test proving that reader rejects all four pre-fix behaviours, so the assertions on top of it can actually fail. `make lint` clean. ## Not validated - **Escaping, on device.** The recogniser transcribed "ampersand, less than, greater than" as those words rather than the symbols, so no reserved character reached the cue text and the escaping path never fired. Covered byte for byte by unit tests; not exercised by dictation. Worth knowing on its own that speech-to-text spells these out. - **Storage exhaustion.** Filling the phone means hours of recording, so the retry and its on-screen report are covered against a synthetic block only. - **Whether cue times match the words by ear.** Open since #8, unchanged. - A recording ended by being killed, and a microphone swap mid-recording. ## Codex Four passes — plan, code, tests, security. Every finding's disposition is in `.agent/m1-09-vtt-assembly/SUMMARY.md`. The ones that changed the outcome: - Retry could land parts out of order and reverse the transcript — a real hole in the plan. - A prefix check took `WEBVTTX` for a signature and deleted the line, losing a cue. - Both retry tests cleared the blockage right before `finish()`, so a writer that only retried at stop would have passed them. - `onlyCompletedPartsAreVisible` could not fail; renamed to what it actually asserts. - Review artifacts carried absolute paths, publishing a local username into a public repository. Nothing was pushed yet, so the branch was rewritten rather than leaving them in reachable history, and `codex-run.sh` now strips them at the source. Two were declined and both are recorded with reasons: the `stash-docs` decision file, deferred to Stash/stash-docs#3 because there is no decisions convention to write into yet, and `pre-push` validating the checked-out branch rather than the refs on stdin — real, but Wes's in-flight work on the process gate rather than anything this issue touches.

agent added 7 commits

2026-08-07 14:18:13 -04:00

Process: fifteen-step issue workflow with Codex in the loop 7bc3668825

AGENTS.md replaces the six-step "Working an issue" section with the process as
it is actually run: plan, codex-pair, one checkpoint with Wes, build, three
Codex passes, hand off, device test, PR. Branch naming is load-bearing because
.agent/<branch>/ and the pre-push hook both derive from it.

.githooks/ splits the enforcement. pre-commit lints, because it runs many times
a day and has to be fast. pre-push requires the review artifacts and runs the
tests, because push is where work leaves the machine — gating commits on a
review that does not exist yet would be routed around with --no-verify inside
an hour, and a hook bypassed by habit is worse than no hook.

.agent/m1-09-vtt/ carries the Codex artifacts from the run that exercised this
process on M1-09.

M1-09: make the parts actually assemble [pre-human-testing] 5ae534a18e

The part writer shipped in #23 claimed assembly was concatenation. It was not.
A rendering ended on its last word rather than on a blank line, so joining two
parts put the next part's WEBVTT signature inside the previous cue — verified
against FFmpeg, which read it as spoken words. The signature is also only a
signature at the start of a file.

So assembly becomes VTT.assemble: the first part keeps its signature and the
rest give theirs up. The files on disk all keep one, because being openable on
its own mid-recording is the only reason a part exists.

Three more from the same review:

Cue text is escaped. A "&", a "<" or a "-->" in what somebody said could
change the line or truncate it, and an embedded blank line ended the cue and
took the rest of its own words with it. Line endings normalise and a run of
them collapses to one, which is the single thing here that does not
round-trip — a line break rather than a word.

Timestamps round to milliseconds before hours, minutes and seconds are split
off. Rounding the seconds field last is how 59.9996 rendered 00:00:60.000,
which WebVTT does not allow and a conforming reader rejects.

A part that fails to write is retried at every later boundary and at stop,
instead of being dropped. TranscriptParts has already released those cues by
the time a write can fail, so the old behaviour left a minute-shaped hole that
nothing marked and nothing could fill. The written list is kept in part order
rather than in the order writes succeeded, or a retried part would be
assembled after the parts that followed it.

The reassembly test asserted only that every line was present, which is why it
passed against output FFmpeg mangled. It now asserts the bytes.

M1-09: report a stalled part while there is still time to act on it df35dbc23c

From the second Codex review of this branch.

A part that will not write now says so during the recording rather than only
at stop. Storage is what fails here and it is the one failure a person can do
something about — but only while there is still a recording for the freed
space to take. It reports through partsError rather than status, because
transcription itself is working and calling it unavailable names the wrong
thing as broken.

A cue whose ends round onto the same millisecond now ends a millisecond after
it starts. 1.0001 to 1.0004 rendered 00:00:01.000 --> 00:00:01.000, and WebVTT
lets a reader discard a cue that does not move forwards. A negative duration is
rejected outright.

TranscriptParts' doc comment still described concatenation as sound and still
said a part closes after "more than" a minute when the code closes it at
exactly one. Both corrected.

M1-09: test the spec, not the bytes this code happens to emit f5b23b4781

A conformance reader written from the WebVTT grammar, with no knowledge of
VTT.render, and a test proving it rejects all four pre-fix behaviours — so
every assertion built on it can actually fail. The byte-comparisons were
written from the same mental model as the assembler and could only ever pin
today's output.

Fills the gaps that left: escaping across nine things a person could say
including "ampersand" spoken aloud, which is what catches escape ordering;
\r\n and bare \r; six times sitting under minute and hour boundaries rather
than one; assembling nothing, one part, and something that did not come from
render; parts assembled backwards; the boundary at exactly sixty seconds; two
parts blocked at once and retried together; pending going 2 to 0; and past 999
parts, where the padding gives way rather than wrapping.

A write failure now names every part still waiting rather than the most recent
one — two minutes stuck behind a full disk read as one, and that count is what
somebody deciding whether to go and free space is deciding on. It names them
padded, 001, because that is what the file on disk is called.

The boundary comment said "strictly greater" over a >= comparison. The code was
right and AGENTS.md agreed with it; the comment described a different rule.

M1-09: close the holes the test review found 7aa2472e46

A prefix check took WEBVTTX for a signature and deleted the line. That is an
ordinary cue identifier, and a signature is exactly WEBVTT or WEBVTT followed
by a space or tab — so the assembler was able to lose a cue rather than a
header.

Both retry tests cleared the blockage immediately before finish(), so a writer
that only ever retried at stop would have passed them — and a recording killed
after the storage came back would still be missing the part, which is the case
the retry exists for. There is now a test that never calls finish() at all.

The two-failures test checked the count but not the message, so reporting only
one of two missing parts would have passed. It names both now.

onlyCompletedPartsAreVisible claimed to prove the write is atomic and did not:
it looks at the directory between completed writes, so an in-place write passes
it too. Renamed to what it actually asserts — no temp file left behind — with
the reason a racing test is not worth having written down instead.

M1-09: strip absolute paths from the review artifacts, and SUMMARY 784f888ec0

The committed Codex artifacts cited files by absolute path, which publishes the
local username and directory layout into a public repository. Nothing had been
pushed, so the branch was rewritten rather than leaving them in reachable
history — every link is repo-relative now.

The generator was the actual fault and is fixed outside this repository: the
prompt asks for relative paths and the script strips $REPO/ and $HOME/ from
every artifact on the way out, because an instruction a model can miss is not a
control.

SUMMARY.md records every Codex finding across the four passes with its
disposition, what was validated and what was not, and the device tests.

M1-09: device results 6d4f4e3691

One three-minute recording on wes-iphone, three parts, 21 cues. Parts open on
their own, assemble into one signature with unbroken numbering and monotonic
times, and both seams are exact with nothing duplicated or dropped. FFmpeg
parses the assembled file clean.

Two things stay unvalidated and are written down as such. The recogniser spells
"ampersand" and "less than" out as words, so no reserved character ever reached
the cue text and the escaping path never fired on device. And filling the phone
to test the retry means hours of recording, so that path is covered against a
synthetic block only.

wk merged commit 31cb0c9515 into main

2026-08-07 14:18:37 -04:00

wk deleted branch m1-09-vtt-assembly

2026-08-07 14:18:37 -04:00

wk referenced this pull request from a commit

2026-08-07 14:18:39 -04:00

Merge pull request 'M1-09 follow-up: make the transcript parts actually assemble' (#24) from m1-09-vtt-assembly into main