Opt-in auto-naming for voice memos via on-device Foundation Models #53

New issue

Closed

opened 2026-08-10 12:49:45 -04:00 by agent · 0 comments

agent commented

2026-08-10 12:49:45 -04:00

Member

Voice memos currently land with timestamp-hash filenames. Add an opt-in setting that generates a short human-readable title from the on-device transcript.

Approach

Use Apple's Foundation Models framework (iOS 26+). The ~3B Apple Intelligence model is already resident on qualifying devices, so this costs nothing in bundle size and adds no second inference runtime. @Generable constrains decoding to a typed struct, so the result is a String rather than prose that needs parsing.

import FoundationModels

@Generable
struct MemoTitle {
    @Guide(description: "3-6 word title, no punctuation, no quotes")
    var title: String
}

let session = LanguageModelSession(
    instructions: "You name voice memos. Return a short factual title describing what the recording is about."
)
let result = try await session.respond(to: transcript, generating: MemoTitle.self)

Scope

TitleGenerator service that takes VTT cue text and returns a title
SystemLanguageModel.default.availability check gating the settings toggle. The three unavailable cases (no supporting hardware / Apple Intelligence disabled / model still downloading) need distinct UI copy — do not collapse them into one "unavailable" state
Settings toggle, default off
Fallback for unavailable devices: NLTagger noun-phrase and named-entity extraction over the transcript, top few terms joined. No model, no download, works back to iOS 12. Produces keyword-style titles ("boiler contractor Spectrum quote") rather than sentences, which for a filename is fine
Last-resort fallback: truncate the first VTT cue or two to ~60 chars
Title generation must never block or fail the capture/sync path. A memo with a bad title is fine; a memo that didn't sync is not

Constraints

Context window is roughly 4k tokens. Long memos need chunking, or feed only the opening few minutes of the VTT — enough for a title either way
Everything stays on-device. This does not change the blind-relay property: the relay still never sees transcript text or titles in the clear
Device gate is Apple Intelligence hardware (iPhone 15 Pro and up), narrower than the app's floor. The NL fallback is what covers the rest, so it is not optional

Not validated

Title quality has not been tested against real recordings. Hold the PR until Wes has run it on-device against actual memos — that is the only way this feature's quality shows up.

Alternative considered and rejected

A Core ML conversion of Qwen3.5-0.8B (mlboydaisuke/qwen3.5-0.8B-CoreML). Rejected: 128-token max sequence length means titling from the opening ~90 words only, a 754 MB .mlpackage to ship or download for a five-word filename, and no bundled tokenizer.

Voice memos currently land with timestamp-hash filenames. Add an opt-in setting that generates a short human-readable title from the on-device transcript. ## Approach Use Apple's **Foundation Models** framework (iOS 26+). The ~3B Apple Intelligence model is already resident on qualifying devices, so this costs nothing in bundle size and adds no second inference runtime. `@Generable` constrains decoding to a typed struct, so the result is a `String` rather than prose that needs parsing. ```swift import FoundationModels @Generable struct MemoTitle { @Guide(description: "3-6 word title, no punctuation, no quotes") var title: String } let session = LanguageModelSession( instructions: "You name voice memos. Return a short factual title describing what the recording is about." ) let result = try await session.respond(to: transcript, generating: MemoTitle.self) ``` ## Scope - `TitleGenerator` service that takes VTT cue text and returns a title - `SystemLanguageModel.default.availability` check gating the settings toggle. The three unavailable cases (no supporting hardware / Apple Intelligence disabled / model still downloading) need distinct UI copy — do not collapse them into one "unavailable" state - Settings toggle, default off - Fallback for unavailable devices: `NLTagger` noun-phrase and named-entity extraction over the transcript, top few terms joined. No model, no download, works back to iOS 12. Produces keyword-style titles ("boiler contractor Spectrum quote") rather than sentences, which for a filename is fine - Last-resort fallback: truncate the first VTT cue or two to ~60 chars - Title generation must never block or fail the capture/sync path. A memo with a bad title is fine; a memo that didn't sync is not ## Constraints - Context window is roughly 4k tokens. Long memos need chunking, or feed only the opening few minutes of the VTT — enough for a title either way - Everything stays on-device. This does not change the blind-relay property: the relay still never sees transcript text or titles in the clear - Device gate is Apple Intelligence hardware (iPhone 15 Pro and up), narrower than the app's floor. The NL fallback is what covers the rest, so it is not optional ## Not validated Title quality has not been tested against real recordings. Hold the PR until Wes has run it on-device against actual memos — that is the only way this feature's quality shows up. ## Alternative considered and rejected A Core ML conversion of Qwen3.5-0.8B (`mlboydaisuke/qwen3.5-0.8B-CoreML`). Rejected: 128-token max sequence length means titling from the opening ~90 words only, a 754 MB `.mlpackage` to ship or download for a five-word filename, and no bundled tokenizer.

agent added the

area/voice-memos

type/feature

labels

2026-08-10 12:49:45 -04:00

agent referenced this issue

2026-08-11 07:29:47 -04:00

Edit a memo's title on the detail screen #58

agent referenced this issue from a pull request that will close it,

2026-08-11 11:53:59 -04:00

M3-53 + M3-58: name a memo, and let somebody change the name #61