Stash/stash-ios

Fork 0

M2-07: The track file — write it, and survive a crash #34

New issue

Closed

opened 2026-08-08 19:56:53 -04:00 by agent · 2 comments

agent commented

2026-08-08 19:56:53 -04:00

Member

Write <base>_track.gpx beside the memo, assemble it at stop, and recover it at launch.

Why it is durable rather than written once at stop

Decided 2026-08-08: the parts pattern, the same shape the transcript already uses — append as
it goes, join at stop, and let launch finish what a crash interrupted.

Audio survives a kill because of movieFragmentInterval. The transcript survives one because
parts are written every minute and assembled on the next launch. A track written only at stop
would be the single artifact in this app that a crash destroys completely, and that inconsistency
is what would surprise somebody later — including a future client reading these files.

M1-15 (#25) exists because a crash cost 16 seconds of speech. Losing a whole forty-minute route to
the same crash is the larger version of that bug, and it is cheaper to not build it than to fix it.

Scope

Points appended to part files during the recording, in the sibling parts folder that already
exists for transcripts, using the same padded numbering so the existing parsers keep working.
Assembly at stop produces one <base>_track.gpx.
The launch sweep that already recovers transcripts recovers tracks the same way: a capture with
location parts and no track is assembled from what survived.
A part that never landed leaves a gap. The transcript's answer — assemble anyway, keep the
folder, name the missing numbers — applies here for the same reason: a partial route is worth
having, and something on disk has to say it is partial.
Writes are throttled and atomic. A point per second for forty minutes is 2,400 points, and one
write per point during capture is write amplification while the audio path is what matters.

The file

Decided 2026-08-09: GPX, not a bespoke JSON schema. See the decision comment below. A GPX 1.1
<trk> with one <trkseg>, and a <trkpt> per fix.

This amends decision 3 from the M2 planning round, which called for relative seconds plus one
absolute start time in a header. GPX carries an absolute UTC <time> on every point natively, and
that is strictly better — the track is self-describing without a header convention, and every tool
that reads GPX already understands it. Alignment with VTT cue times is derived by subtracting the
recording start rather than stored, which removes a value that could disagree with the file.

Field mapping, to be confirmed against the GPX 1.1 schema rather than assumed:

lat / lon and <time> — native, and the reason for choosing GPX.
<ele> — native, altitude.
<hdop> — the nearest native home for horizontal accuracy. It is not the same quantity as
CLLocation.horizontalAccuracy; if it cannot be mapped honestly it goes in <extensions> under
its own name rather than being coerced into a field that means something else.
Speed and course — <extensions>. GPX 1.0 had them natively and 1.1 moved them out; an extension
namespace is the standard-conforming answer, and readers that do not know it ignore them, which
is the correct degradation.
<metadata><time> — the recording start, so the track still stands alone.

The exact element layout is the plan's to propose, and it gets written down in stash-docs as a
decision record before implementation. It is a format every future client has to read.

Done when

A completed recording with location on produces exactly three files.
A recording force-quit at minute 20 recovers a track covering roughly 20 minutes.
A recording with location off produces two files and no empty track.
A track with a missing part says so rather than reading as a whole route.
The schema is recorded in stash-docs.

Milestone 2 of the Stash v3 rebuild. M2 is still app-side — no sync, no account, no server. It
turns a list of memos into something you can open, listen to, read and see the route of. Sync
arrives in M3. There is no milestone/M2 label yet (creating org labels needs owner rights), so
the title carries the milestone.

Working agreement for every issue in this repo: feature branch, tested before the PR opens
(on-device where the label says so), then Wes reviews the PR and we walk the code together. Docs
update in the same commit. A PR that takes more than 20 minutes to review is too big — say so and
split it.

Write `<base>_track.gpx` beside the memo, assemble it at stop, and recover it at launch. ## Why it is durable rather than written once at stop **Decided 2026-08-08: the parts pattern**, the same shape the transcript already uses — append as it goes, join at stop, and let launch finish what a crash interrupted. Audio survives a kill because of `movieFragmentInterval`. The transcript survives one because parts are written every minute and assembled on the next launch. A track written only at stop would be the single artifact in this app that a crash destroys completely, and that inconsistency is what would surprise somebody later — including a future client reading these files. M1-15 (#25) exists because a crash cost 16 seconds of speech. Losing a whole forty-minute route to the same crash is the larger version of that bug, and it is cheaper to not build it than to fix it. ## Scope - Points appended to part files during the recording, in the sibling parts folder that already exists for transcripts, using the same padded numbering so the existing parsers keep working. - Assembly at stop produces one `<base>_track.gpx`. - The launch sweep that already recovers transcripts recovers tracks the same way: a capture with location parts and no track is assembled from what survived. - A part that never landed leaves a gap. The transcript's answer — assemble anyway, keep the folder, name the missing numbers — applies here for the same reason: a partial route is worth having, and something on disk has to say it is partial. - Writes are throttled and atomic. A point per second for forty minutes is 2,400 points, and one write per point during capture is write amplification while the audio path is what matters. ## The file **Decided 2026-08-09: GPX, not a bespoke JSON schema.** See the decision comment below. A GPX 1.1 `<trk>` with one `<trkseg>`, and a `<trkpt>` per fix. This **amends decision 3** from the M2 planning round, which called for relative seconds plus one absolute start time in a header. GPX carries an absolute UTC `<time>` on every point natively, and that is strictly better — the track is self-describing without a header convention, and every tool that reads GPX already understands it. Alignment with VTT cue times is derived by subtracting the recording start rather than stored, which removes a value that could disagree with the file. Field mapping, to be confirmed against the GPX 1.1 schema rather than assumed: - `lat` / `lon` and `<time>` — native, and the reason for choosing GPX. - `<ele>` — native, altitude. - `<hdop>` — the nearest native home for horizontal accuracy. It is **not the same quantity** as `CLLocation.horizontalAccuracy`; if it cannot be mapped honestly it goes in `<extensions>` under its own name rather than being coerced into a field that means something else. - Speed and course — `<extensions>`. GPX 1.0 had them natively and 1.1 moved them out; an extension namespace is the standard-conforming answer, and readers that do not know it ignore them, which is the correct degradation. - `<metadata><time>` — the recording start, so the track still stands alone. The exact element layout is the plan's to propose, and it gets written down in `stash-docs` as a decision record before implementation. It is a format every future client has to read. ## Done when - [ ] A completed recording with location on produces exactly three files. - [ ] A recording force-quit at minute 20 recovers a track covering roughly 20 minutes. - [ ] A recording with location off produces two files and no empty track. - [ ] A track with a missing part says so rather than reading as a whole route. - [ ] The schema is recorded in `stash-docs`. --- *Milestone 2 of the Stash v3 rebuild. M2 is still app-side — **no sync, no account, no server**. It turns a list of memos into something you can open, listen to, read and see the route of. Sync arrives in M3. There is no `milestone/M2` label yet (creating org labels needs owner rights), so the title carries the milestone.* *Working agreement for every issue in this repo: feature branch, tested before the PR opens (on-device where the label says so), then Wes reviews the PR and we walk the code together. Docs update in the same commit. A PR that takes more than 20 minutes to review is too big — say so and split it.*

agent added the

labels

2026-08-08 19:56:53 -04:00

agent commented

2026-08-09 11:25:51 -04:00

Author

Member

The file format is not settled, and a note from 2026-07-27 argues against the JSON this issue assumes.

The 8/6 walk memo said "JSON file" in passing. A note Wes wrote on 7/27, thinking about this exact feature, states a principle that points somewhere else:

"If one of the goals of Stash is that at the end of the day somebody can open that folder in any application on their computer and just read the raw files — because they're just markdown and VTT, which is just text — then they're not locked into the app forever... I'm very much against proprietary data formats."

By that standard a bespoke JSON schema is the weakest option on the table. It is readable, but nothing else opens it — every consumer has to be told what the keys mean, which is the definition of a proprietary format that happens to use a standard encoding.

GPX is the open standard for tracks and every mapping tool on earth reads it. Drop a .gpx beside the audio and the route opens in Garmin Connect, Gaia, QGIS, Strava, Apple's own tools — with no code written anywhere. It carries time, elevation and coordinates natively. GeoJSON is the other candidate: less universal for tracks specifically, better for tooling that speaks geo generally.

Costs worth weighing rather than dismissing: GPX is XML, so it is more verbose than JSON on disk for a 2,400-point walk; its <extensions> mechanism is where speed, course and horizontal accuracy would have to live, which is standard but less tidy than named JSON fields; and the decided timestamp basis for this issue is relative seconds from recording start, whereas GPX wants absolute times per point. That last one is not a conflict — absolute times are derivable from the header start, and GPX carrying them natively is arguably better.

This should be settled in stash-docs before implementation, alongside the schema this issue already defers there. It is a format every future client reads and the vault consumes, so it is expensive to revisit — and the same argument that made the transcript a .vtt rather than a bespoke format applies here without modification.

Raised because M2-10 (#39) will read every one of these files, and because the note that argued for it predates the scope memo that said JSON.

**The file format is not settled, and a note from 2026-07-27 argues against the JSON this issue assumes.** The 8/6 walk memo said "JSON file" in passing. A note Wes wrote on 7/27, thinking about this exact feature, states a principle that points somewhere else: > *"If one of the goals of Stash is that at the end of the day somebody can open that folder in any application on their computer and just read the raw files — because they're just markdown and VTT, which is just text — then they're not locked into the app forever... I'm very much against proprietary data formats."* By that standard a bespoke JSON schema is the weakest option on the table. It is readable, but nothing else opens it — every consumer has to be told what the keys mean, which is the definition of a proprietary format that happens to use a standard encoding. **GPX** is the open standard for tracks and every mapping tool on earth reads it. Drop a `.gpx` beside the audio and the route opens in Garmin Connect, Gaia, QGIS, Strava, Apple's own tools — with no code written anywhere. It carries time, elevation and coordinates natively. **GeoJSON** is the other candidate: less universal for tracks specifically, better for tooling that speaks geo generally. Costs worth weighing rather than dismissing: GPX is XML, so it is more verbose than JSON on disk for a 2,400-point walk; its `<extensions>` mechanism is where speed, course and horizontal accuracy would have to live, which is standard but less tidy than named JSON fields; and the decided timestamp basis for this issue is relative seconds from recording start, whereas GPX wants absolute times per point. That last one is not a conflict — absolute times are derivable from the header start, and GPX carrying them natively is arguably better. This should be settled in `stash-docs` before implementation, alongside the schema this issue already defers there. It is a format every future client reads and the vault consumes, so it is expensive to revisit — and the same argument that made the transcript a `.vtt` rather than a bespoke format applies here without modification. Raised because M2-10 (#39) will read every one of these files, and because the note that argued for it predates the scope memo that said JSON.

agent commented

2026-08-09 11:30:58 -04:00

Author

Member

Decided 2026-08-09: GPX. Wes: "GPX for sure."

Resolves the format question raised above. The track is <base>_track.gpx, not <base>_location.json.

The reasoning that decided it is Wes's own, from 7/27: the folder should open in any application on the computer, and he is "very much against proprietary data formats." A bespoke JSON schema is readable and still proprietary — nothing but Stash knows what the keys mean. GPX opens in Gaia, QGIS, Garmin Connect, Strava and Apple's own tools with no code written anywhere, which is the same bar the transcript already meets by being WebVTT rather than something invented.

What this changes elsewhere:

#32 (M2-05) — the third file is <base>_track.gpx. Patched.
Decision 3 from the M2 planning round is amended. Relative-seconds-plus-header is replaced by GPX's native absolute UTC <time> per point; alignment with VTT is derived, not stored.
Decision 4 is unchanged in intent, but the fields land differently: time and altitude are native, horizontal accuracy maps to <hdop> only if that can be done honestly, and speed and course go in <extensions> because GPX 1.1 moved them there.
#39 (M2-10) reads these files for the heat map, so it reads GPX.

Accepted cost: XML is more verbose on disk than JSON for a 2,400-point walk. Worth it for a format that needs no consumer written.

**Decided 2026-08-09: GPX.** Wes: *"GPX for sure."* Resolves the format question raised above. The track is `<base>_track.gpx`, not `<base>_location.json`. The reasoning that decided it is Wes's own, from 7/27: the folder should open in any application on the computer, and he is *"very much against proprietary data formats."* A bespoke JSON schema is readable and still proprietary — nothing but Stash knows what the keys mean. GPX opens in Gaia, QGIS, Garmin Connect, Strava and Apple's own tools with no code written anywhere, which is the same bar the transcript already meets by being WebVTT rather than something invented. **What this changes elsewhere:** - **#32 (M2-05)** — the third file is `<base>_track.gpx`. Patched. - **Decision 3 from the M2 planning round is amended.** Relative-seconds-plus-header is replaced by GPX's native absolute UTC `<time>` per point; alignment with VTT is derived, not stored. - **Decision 4 is unchanged in intent**, but the fields land differently: time and altitude are native, horizontal accuracy maps to `<hdop>` only if that can be done honestly, and speed and course go in `<extensions>` because GPX 1.1 moved them there. - **#39 (M2-10)** reads these files for the heat map, so it reads GPX. Accepted cost: XML is more verbose on disk than JSON for a 2,400-point walk. Worth it for a format that needs no consumer written.

agent referenced this issue

2026-08-09 11:30:59 -04:00

M2-06: Collect location while a recording runs #33

agent referenced this issue

2026-08-09 18:21:22 -04:00

M2-05: A memo is up to three files #46

agent referenced this issue from Stash/stash-docs

2026-08-09 18:21:23 -04:00

Decision: a memo is up to three files #7

agent referenced this issue

2026-08-09 18:21:55 -04:00

M2-05: A memo is up to three files #32