1. Core Components
Four subsystems run inside a single process: the HTTP/WebSocket hub, the transcript watcher, the client registry, and the pairing layer.
Async Web & WebSocket Hub
Powered by aiohttp, the hub serves the companion web app (PWA) over plain HTTP on 5141, alongside a parallel self-signed HTTPS server on 5142 to satisfy the browser secure-context requirement for microphone access. WebSocket connections at /ws carry persistent full-duplex traffic.
Client Registry & State Sync
The server keeps an active registry of connected WebSocket clients. When an agent turn completes or a tool runs, it broadcasts a serialized JSON event to every connected client, keeping the mobile PWA in step with desktop agent state.
Transcript Polling & Parsing
A background daemon thread tails local agent execution transcripts (JSONL). It separates terminal PLANNER_RESPONSE steps from intermediate tool invocations, extracts the agent's reply, and strips markdown syntax so the text reads cleanly through speech synthesis.
def _check_transcript_turn(self, path: Path):
# Polls JSONL transcript for new lines
# Detects PLANNER_RESPONSE where status == "DONE"
# Cleans markdown for speech
# Triggers WebSocket broadcastLocal Pairing & Secure Tunnels
To bridge desktop agent to phone, the companion generates a QR code pointing at the machine's LAN address. It bootstraps a self-signed ssl.SSLContext for local HTTPS, or provisions an ephemeral Cloudflare Quick Tunnel via cloudflared when the local network blocks peer-to-peer traffic entirely.
Hands-Free Workflow & VAD
The Companion PWA implements local Voice Activity Detection (VAD) to establish a continuous feedback loop across any supported workflow—including Antigravity, Claude, Cursor, and other autonomous coding agents or IDEs. When an agent turn completes and the server broadcasts the summary, the mobile client automatically plays the TTS audio, re-opens the microphone, and actively listens for the user's next command until silence is detected.
2. Network Surface
Every listener the companion opens, and why it exists.
| Port | Scheme | Serves | Rationale |
|---|---|---|---|
| 5141 | http | Companion PWA + /ws |
Desktop and same-machine clients, no cert prompt. |
| 5142 | https | Same app, TLS-wrapped | Browsers only grant getUserMedia in a secure context, so mobile mic capture requires TLS. |
| — | https | Cloudflare Quick Tunnel | Ephemeral public hostname for networks with client isolation. Optional. |
Endpoints
- /
- Companion PWA shell and static assets.
- /ws
- Full-duplex WebSocket event bus. One connection per client.
3. Data Flow
The agent never talks to the companion directly. The transcript file on disk is the integration seam — which is what keeps the companion agent-agnostic.
Why polling, not hooks. Reading the transcript file means the companion needs no cooperation from the agent runtime — no plugin, no patched binary, no IPC contract to keep in sync. Any agent that writes JSONL turns can be observed. The cost is poll latency instead of an instant push.
4. Event Protocol
| Type | Direction | Fires when |
|---|---|---|
| agent_turn_completed | hub → clients | The agent finishes its loop and hands control back to the user. |
| agent_working_step | hub → clients | Incrementally, as the agent executes background tools. |
agent_turn_completed
The terminal event of a turn. summary is the markdown-stripped text intended for speech synthesis; full_response preserves the original for on-screen display.
{
"type": "agent_turn_completed",
"conv_id": "db466be3-ba60...",
"agent_role": "antigravity",
"summary": "I have updated the repository settings to private.",
"full_response": "I've updated the repository settings to private...",
"origin": "desktop",
"timestamp": 1724430000.0
}| Field | Type | Meaning |
|---|---|---|
| conv_id | string | Conversation this turn belongs to. |
| agent_role | string | Which agent produced the turn; drives voice persona selection. |
| summary | string | Speech-ready text, markdown stripped. |
| full_response | string | Unmodified agent reply for display. |
| origin | string | Where the prompting turn came from — e.g. desktop or mobile. Lets a client suppress echo of its own input. |
| timestamp | float | Unix epoch seconds. |
agent_working_step
Progress telemetry emitted while the agent is mid-loop. Clients typically render these as a live activity feed rather than speaking them.
{
"type": "agent_working_step",
"conv_id": "db466be3-ba60...",
"step_index": 42,
"tool_name": "run_command",
"summary": "Check git remote",
"action": "Running command",
"status": "running",
"timestamp": 1724430005.0
}| Field | Type | Meaning |
|---|---|---|
| step_index | int | Monotonic position within the turn; use it to dedupe replays. |
| tool_name | string | Tool being invoked. |
| summary | string | Short human-readable intent. |
| action | string | Present-tense verb phrase for the UI. |
| status | string | Lifecycle state of the step, e.g. running. |
Compatibility. These payloads carry no schema version. Treat unknown type values as ignorable and unknown fields as additive — a client that hard-fails on either will break on the next release.
5. Pairing & Transport
Two ways to reach the desktop from a phone. The companion prefers the first and falls back to the second.
LAN + self-signed TLS
The companion resolves the machine's LAN IP, mints a certificate into an ssl.SSLContext, and encodes the https://<lan-ip>:5142 URL as a QR code. Audio never leaves the local network.
Trade-off: the certificate is untrusted, so the phone shows a one-time browser interstitial that must be accepted manually.
Cloudflare Quick Tunnel
When the network isolates clients from each other — guest Wi-Fi, enterprise APs, cellular — the companion shells out to cloudflared for an ephemeral public hostname with a valid certificate, and encodes that instead.
Trade-off: traffic transits Cloudflare, and the hostname is reachable by anyone who learns it for as long as the tunnel is up.
6. Security Model
What the current implementation guarantees — and, just as importantly, what it does not.
Holds today
- Transcript data is read locally; nothing is uploaded when running over LAN.
- Mobile microphone capture is gated behind TLS by browser policy.
- The tunnel is opt-in and ephemeral — it dies with the process.
Not yet enforced
- The
/wsendpoint has no authentication — any client that can reach the port receives every broadcast. - Self-signed certs give encryption without identity; the interstitial trains users to click through warnings.
- Broadcasts fan out to all clients rather than being scoped per conversation.
Operating guidance. Run the companion on networks you control. Treat an active Quick Tunnel as a public endpoint carrying your agent's output, and shut it down when you are done pairing. Paired-session tokens and per-conversation scoping are specified in