Video streaming

When you open the app, you see your avatar's game view live. Streaming only runs while someone is watching. The agent plays the same whether or not anyone watches.

How it works

flowchart LR
  C["Game canvas<br/>(headless Chromium botclient)"] --> E["WebCodecs VP8 encoder<br/>in the page, 30 fps"]
  E -->|"IPC"| API["API fan-out<br/>per-viewer rules"]
  API -->|"binary WebSocket"| D["App: WebCodecs decoder<br/>→ canvas"]
  C -. "fallback" .-> J["JPEG screencast"] -.-> API

• VP8, encoded inside the page. Chromium ships a VP8 encoder, so no extra package or process is needed. The server has no GPU, so encoding is in software. Constant bitrate, about 600 kbps.
• One encode per avatar, however many viewers. The same bytes go to every socket.
• Two temporal layers. Every other frame can be dropped without breaking the stream, which halves the frame rate for a slow link or for Data saver mode (about 15 fps, roughly 60 % of the bits), with no second encode.
• Keyframes every 5 seconds and on request (when a new viewer joins).

Fullscreen

The fullscreen button (under the thoughts button, top right) gives the game view the whole screen. Chat, thoughts and settings stay on top and usable; the status pill hides.

• Android and desktop: real fullscreen, and the phone turns to landscape where the browser allows it. Back or Esc leaves it, as does the button.
• iPhone: Safari has no fullscreen for a page, so the game view just fills the window; Back or the button leaves it. For a true full-screen app, use Share → Add to Home Screen (Android: Add to Home screen / Install app). The app opens without browser bars.

Turning the camera

Drag the picture to turn your avatar's camera: left and right turn around, up and down tilt the view. The 2004 client turns its camera with the arrow keys, so the server holds the matching arrow key in your avatar's game browser while you drag. There is no zoom, the client has none.

• Only arrow keys are ever sent, and in that client they can't type in chat or click anything. The agent doesn't use the camera at all (it walks and interacts through the SDK), so turning never disturbs what it's doing.
• Only the owner can turn it, and camera messages are rate-limited per connection. It works while the game view is streaming (not on the lite client used when nobody watches).
• One camera per avatar: everyone watching that avatar sees the same turn.

JPEG fallback

Browsers that can't decode VP8 with WebCodecs get JPEG frames instead. The app checks support first, and switches to JPEG on its own if decoding fails repeatedly. Supported for VP8: Chrome/Edge/Samsung Internet (desktop and Android), Firefox desktop 130+, and Safari 16.4+ where its decoder agrees (not yet tested on a real iPhone). Firefox on Android and older browsers use JPEG. WebCodecs needs HTTPS.

Slow connections

Each viewer's socket is handled on its own:

1. If the socket's send buffer starts filling up, that viewer gets only every other frame (15 fps).
2. If it fills completely, frames are dropped until the next keyframe (at most 5 s), so a viewer never gets a broken picture.
3. The encoder lowers its bitrate only when every viewer is struggling, so one slow phone doesn't lower the quality for others.

Warm re-watch

Closing the app and coming back within 3 minutes is instant. The browser stays warm (streaming off, rendering throttled to 5 fps), and the first frame of a re-watch arrives in about 50 ms on TEST. After 3 minutes unwatched, the avatar goes back to the light lite client, and the next watch is a cold start of about 5 seconds (a new Chromium and a login, which takes over the player already in the world).

The app also stops watching if its tab is hidden for more than 2 minutes, so a forgotten open tab doesn't keep a browser running.

Costs (measured on TEST)

Game client	Memory	CPU
Browser, watched (streaming 30 fps)	~660 MB	~84 % of a core
Browser, warm (unwatched)	~610 MB	~9 %
Lite client (nobody watching)	~65 MB	1–2 %

That is why watching is capped (6 streams at once on TEST, and at most 10 browsers counting warm ones). The least recently watched warm browser is released first.