Documents, speech, video, alerts, session lookup
The rest of the plantocode namespace: self-contained HTML documents, Gemini text-to-speech and video extraction under your own key, push alerts through the relay, and a read-only view of your sessions.
Checked against the source on 17 September 2026
On this page
What the agent can reach
- Tool callitem/tool/call
Codex asks the desktop to run a tool and waits for the reply. The handler reads only the tool name and the arguments the model wrote, including the session ID copied from the tool description.
- Local toolsbuild_app_capabilities
Always in the list: HTML files under agent-documents in the app data folder, a read-only look at appdata.db, and your Chrome through Browser Bridge, which stays off until you enable it.
- Conditional toolshas_gemini_api_key · show_notifications
The Gemini pair needs a Gemini key saved on this computer and calls Gemini straight from the desktop. The alert needs Desktop and phone notifications on and travels through the regional server.
- Fixed listdynamicTools
The desktop rebuilds the list for every run, but Codex takes it only when a thread starts. A resumed session keeps the tools and routing values it began with.
HTML documents
The agent can answer with a complete HTML document, with inline CSS, SVG, and scripts, that opens in the desktop preview pane and in the file viewer on your phone. It loads no external assets. The desktop serves it from a sandboxed origin, and the app accepts only two messages from the page: open a link, and page loaded.
The tool takes a title of at most 120 characters and a complete document of at most 2 MiB with doctype, html[lang], a title, a device-width viewport without zoom limits, main, and h1. Every SVG needs a viewBox and a title, aria-label, or aria-hidden; src, poster, and href may only be data: URLs or fragment IDs; script[src], link[href], iframe, object, embed, base, srcset, and meta refresh are rejected. The desktop prepends a Content-Security-Policy of default-src none with inline scripts and styles, data: images, fonts, and media, and no connect, frame, or form targets, then writes the file to agent-documents/<sessionId>/<documentId>/ under its data directory with a metadata.json, refuses to overwrite, and returns a Markdown link the agent has to include verbatim, because a tool result alone is not delivery. The result marks structure as passed and visual and factual review as not checked.
Speech and video with your Gemini key
Save a Gemini API key in Settings → API Keys and two more tools appear in the namespace in sessions that start after you save it. The key stays on the desktop, and the requests go from your computer straight to Gemini.
- Text-to-speech. The desktop splits the text at sentence boundaries into chunks of at most 1,800 characters, synthesizes three at a time with gemini-3.1-flash-tts-preview, retries each chunk twice with a growing backoff, and stitches the 24 kHz mono 16-bit PCM parts with 180 ms of silence between them into one WAV file under agent-speech in the cache directory. Director notes stay outside the spoken transcript, and a multi-part prompt tells the model not to read the part number aloud. Thirty prebuilt voices are available and Kore is the default.
- Video context. The source is a public YouTube URL, a public https video URL, or a local recording, with a tutorial or meeting intent. A YouTube ID must be 11 characters and a playlist parameter is rejected. gemini-3.1-pro-preview runs at temperature 0.2 against a JSON schema that is rendered to Markdown. Tutorial mode extracts concepts, procedures, commands, code, on-screen text, and caveats; meeting mode extracts decisions, requirements, blockers, action items, and open questions. The result is timestamped notes scoped to your goal, with gaps called out.
Video sources are exactly one of youtube_url, local_file, or video_url. A remote URL needs one of twelve supported video MIME types such as video/mp4, video/webm, video/quicktime, or video/mpeg, and Gemini must be able to fetch it directly. There are no fallbacks: a private, inaccessible, or ambiguous source fails explicitly.
Alerts written by the agent
- Requestedagent-notification-requested
The call returns as soon as the event is queued on a connected device link. The server broadcasts it and sends the pushes afterwards, on its own.
- Visible failureinitialize_connection
If the link is not ready, the desktop restarts it and tries 8 more times. With Allow Remote Access off or while signed out it never comes up, so the error arrives after about 2 seconds, or at once if the link never started since the app launched.
- Silent dropsend_agent_notification_push
After the relay nothing comes back. A server without APNs or FCM, an account with no registered phone, and a rejected token are only logged. A phone without notification permission shows nothing.
Turn on Desktop and phone notifications in Settings → General and sessions started after you turn it on get a tool for one push alert about the current session per call. The Agent notifications row below it selects the guidance the tool carries: Important limits it to results and blocking questions, Frequent adds meaningful milestones and failures but still tells the agent to avoid routine progress updates. Both policies expose the same tool; only its description differs. The 60-character title cap forces the outcome first.
Session lookup
maintain_sessions is a read-only view of appdata.db with three actions: guide, listSessions, and resolveSession. It joins sessions with the project workspace state, which marks the active session, and with the Codex session links, orders by last update, returns 25 rows by default and at most 200, filters by the hash of the project directory, and resolves a PlanToCode session to its Codex thread link. It cannot update session metadata or active-session state, and it is not a SQL tool.
Where the data goes
Browser control operates your Chrome, speech and video send text or video to Gemini under your key, and alerts travel through the regional server.