iOS: viewers, Review Mode, dictation, and purchases

Code with Runestone and Tree-sitter #

File previews and code blocks use Runestone with 37 Tree-sitter grammars from the pinned plantocode-tree-sitter-languages package, from Swift, Kotlin, TypeScript, and Rust to SQL, LaTeX, and Diff. Compiling a Tree-sitter query is expensive, so PlanRunestoneLanguageProvider keeps prepared languages alive and shares them between previews and inline code blocks, and Runestone’s TextViewState is prepared away from the main thread. Swift is highlighted with SwiftSyntax, and jq, AWK, and code embedded in shell commands have their own highlighters. Highlight results are cached, 48 of them within 16 MiB, and a highlight that is already running is shared by every view that asks for it.

Command output chooses a rendering mode by size. Up to 8 KiB it is highlighted as one text, up to 512 KiB it is highlighted in virtualized rows, and anything larger renders as plain virtualized rows, so CoreText never shapes a whole large document on the main thread. Markdown rendering of output stops at 2 MiB.

Markdown, math, and Mermaid #

Chat Markdown renders with MarkdownUI on swift-cmark, and math renders with SwiftMath. Prepared Markdown is cached per timeline block, 256 entries within 16 MiB, and a long message collapses above 3,200 characters or 48 lines.

Mermaid diagrams render in a WKWebView with a non-persistent data store, using a bundled mermaid.min.js 11.16.0 whose SHA-256 is pinned in a manifest. The page runs with securityLevel strict and htmlLabels false and posts the diagram’s size back to the app. An LRU cache of 48 entries and 4 MiB keeps rendered SVG, so scrolling back to a diagram does not start the runtime again. A diagram source is capped at 50,000 characters.

HTML documents over a custom URL scheme #

An .html or .htm file opens as a live document. Every path under its ptc-document://<preview ID>.localhost host is read from one folder on the desktop: the project folder when the document lives inside the project, otherwise the document’s own folder. Relative links resolve inside that folder, the desktop refuses any path whose real location leaves it, and each response carries the MIME type the desktop derives from the file name.

All binary reads share one chunk reader. WorkspaceFileChunks pulls one chunk at a time and checks that the offset is the expected one, that the content version and size did not change, that the next offset follows without overflow, and that the chunk stays inside the file. Base64 decoding runs off the main thread, and a read that fails is not resumed.

Media, PDFs, images, and the preview cache #

Audio and video play through an AVAssetResourceLoader delegate that reads files.readMediaRange in 512 KiB chunks backed by a 128 MiB range cache. Desktop-backed playback keeps the relay open in the background through the audio background mode.

Previews are cached by an actor with a 512 MiB budget and 3-day retention, and cached entries are tied to the SHA-256 digest from the desktop’s preview metadata. PDFs render with PDFKit. Images decode at no more than 2,560 pixels on the long side for full views and 480 for cards, with at most two decodes at a time.

Review Mode builds a video from narration and screenshots #

Review Mode on the iPhone records your voice and takes a still screenshot each time you tap Capture. The first start asks for consent, and the prompt names Gemini as the service that analyzes the recording. Audio records with AVAudioRecorder as AAC at 44.1 kHz, mono, 128 kbps, in an audio session for spoken audio that allows Bluetooth headsets, ducks other audio, and uses the speaker. The review controls live in a separate pass-through window above the app.

A capture hides the controls, waits 120 milliseconds, draws the key window with drawHierarchy, and saves a JPEG at quality 0.82 together with its time offset. Recordings and images are written with file protection and deleted on logout.

Stop runs three steps in order. The phone cuts the audio at the midpoints between neighboring captures and uploads each segment to /api/audio/transcriptions, which transcribes it with openai/gpt-4o-transcribe. It then composes the MP4 with AVAssetWriter and adds the audio through a composition exported as passthrough when the format allows. Last, it uploads the video with its prompt to /api/llm/video/analyze, and the analysis comes back as text you review in the chat draft before anything reaches the agent.

Dictation #

Dictation records an m4a file. While it records, the app holds a background task and turns on proximity monitoring. The upload goes to /api/audio/transcriptions on the regional server with openai/gpt-4o-transcribe and up to 3 attempts, 0.8 and then 1.6 seconds apart. The prompt that carries your vocabulary comes from the desktop through system.voiceTranscriptionContext, and no provider key is stored on the phone.

StoreKit, the connected preview, and push #

PlanToCode Pro on the iPhone is a StoreKit 2 subscription checked on the device. The app loads the products, purchases through StoreKit, accepts only verified transactions, listens to Transaction.updates, reads current entitlements, and calls AppStore.sync to restore. No App Store receipt goes to the PlanToCode server.

The 10-day connected preview belongs to the server. GET api/billing/workspace-preview reads it and PUT starts it, and the app sends the PUT after the first successful RPC response from a desktop, as long as the region, the account, and the server URL of that claim are still current. A response with unexpected keys is rejected, and expiry is tracked on a monotonic clock.

The APNs token is registered with PUT api/devices/push-token, in up to 4 attempts that wait 15 to 60 seconds after a rate limit and 1 to 8 seconds after a transient error. In the foreground a banner appears only for notifications scoped to the signed-in account. A silent desktop_online push starts relay recovery, and an agent notification changes the open session only when you tap it.