watchparty/docs/architecture.md

Architecture

Goals

• Watch the same title, in sync, with other users of the same Emby server.
• Use Emby as the media source (catalog + streaming) without requiring changes to Emby itself or its official clients.
• Access gated to people with an Emby account on this server — no anonymous use, no separate identity system.

Non-goals

• Chat/reactions — deliberately out of scope.
• OIDC/SSO — an OIDC provider (LemonLDAP::NG) is available in principle, but the explicit choice was Emby-only auth: simpler, no extra infra dependency, and everyone already has an Emby account.
• Full library navigation (Shows → Seasons → Episodes) in v1 — search-based catalog only, to keep the first version achievable. Revisit if browsing turns out to matter more than searching in practice.

Why this replaced an Emby plugin

The first attempt (still in src/WatchParty/, kept for reference) was a real Emby Server plugin: sync via ISessionManager's existing remote-control commands, a settings page via IHasWebPages, and — after a fair amount of reverse-engineering — a real user-facing page integrated into Emby's own web client menu (PluginPageInfo.EnableInUserMenu, undocumented but confirmed working; required a data-controller AMD module to satisfy Emby's modern ViewManager, found by reading the target server's actual viewmanager.js).

That plugin builds and installs correctly. What it hit a hard wall on: a button inside the video player itself, or on an item's detail page. An Emby developer confirmed on their own forum that plugins have no API for this ("It's currently not possible for a plugin to do this"), and nothing found in research since suggests that's changed. The only way to get a button into Emby's own player would be patching Emby's shipped web client files directly inside the Docker container — outside the plugin entirely, guaranteed to break on every Emby image update, and requiring manual reapplication each time. That trade-off wasn't worth it.

Building a standalone app sidesteps the question entirely: it's our own player, so there's no "how do I add a button to someone else's UI" problem in the first place, and sync can be tighter than nudging Emby's remote-control commands ever allowed (see below).

Full trace of the plugin investigation (real errors, real fixes, sources consulted) is in the git history of this file and of src/WatchParty/ — worth reading if a future Emby SDK update ever makes in-player integration possible and this gets revisited.

Components

server/   Node.js + Express + ws. Owns party/room state and the realtime
          sync channel. Never touches a media byte - see "Media never
          flows through our backend" below.

client/   Static frontend, plain JS + ES modules, no build step (same bias
          as the earlier plugin's control panel). Talks to two different
          things and must not confuse them:
            - Emby directly (embyApi.js): catalog, images, streaming.
            - Our own backend (party.js): create/join/leave a party, and
              the WebSocket for realtime sync.

Auth and Emby access: the server proxies everything

Revised after the first version: the browser used to call Emby directly (catalog, images, streaming) and only used server/ for party coordination. That hit real CORS problems in practice (Emby/its reverse proxy don't answer cross-origin preflights the way that needed), and separately wasn't the wanted shape anyway — the browser holding a real Emby access token at all was more exposure than necessary. Current design:

• The browser sends a username/password once, to POST /api/auth/login on our own backend — never to Emby directly.
• Our backend (server/src/embyAuth.js) calls Emby's POST /Users/AuthenticateByName itself, using EMBY_SERVER_URL from its own environment (not something the user ever types — see .env.example). On success it mints its own opaque session id, stores {embyToken, userId, userName} server-side (in-memory, like party state), and hands the browser only that session id, as an httpOnly cookie (wp_session). The browser never sees or holds Emby's real access token.
• Every other request — catalog search, item details, images, HLS streaming, party actions — goes to our own origin only (/api/...), authenticated by that same cookie. server/src/embyProxy.js is what actually forwards these to Emby server-side and streams the response back, attaching the session's real Emby token itself.
• The /api/catalog/stream/:itemId/master.m3u8 and .../raw routes proxy HLS specifically: the master playlist is fetched from Emby server-side, and every URI line in it is rewritten to point back through .../raw (whether Emby wrote it as relative or absolute) — otherwise hls.js would follow the playlist's own URLs straight to Emby the moment it tried to fetch a segment, defeating the whole point. See rewritePlaylist in embyProxy.js.
• Images (<img src>) and video (hls.js/<video>) still can't carry a bearer header, but that's fine now: they're same-origin requests to our own /api/... paths, so the session cookie rides along automatically — no token in any URL anywhere, unlike the first version's ?api_key=....

This also means the client no longer needs to ask for a server URL at all — one less field, one less way to get it wrong. EMBY_SERVER_URL lives only in the backend's environment.

Control model: host drives, everyone else follows

Now that we own the player, the classic watch-party model (Teleparty, Amazon Watch Party, ...) fits better than the old plugin's "everyone keeps their own controls, shared reference position" — that design was specifically a concession to nudging pre-existing, independently-running Emby app sessions we didn't control. Here, there's only one player implementation, so:

• The party creator is the host. Their player has normal controls.
• Everyone else's <video> has controls hidden; a small notice says the host is in charge.
• Host → server → everyone else, over the WebSocket (server/src/wsHandler.js):
  • Event-driven: every play/pause/seeked on the host's video sends {type:"playstate", isPaused, positionTicks} immediately.
  • Heartbeat: also sent every 3s regardless, so latecomers/drift get corrected even without a discrete host action.
  • Followers apply pause/unpause immediately, and only seek if drift exceeds 1.5s (DRIFT_TOLERANCE_SECONDS in client/js/views/player.js) — avoids fighting small timing noise with constant re-seeks.
• If the host leaves, the server promotes another member (same fallback logic the old plugin used for its "reference" member).

Party lifecycle

In-memory only on the backend (server/src/partyManager.js) — a restart clears every party, which is fine: a party is meant to last one watch session, not persist for weeks. removeIdleParties sweeps anything with no activity for 4 hours.

Assumptions to verify

Confirmed against the real server (not a guess — actually exercised, see session history): POST /Users/AuthenticateByName really is reachable from Node's server-side fetch with no CORS issue at all (server-to-server has no CORS concept), returns a real 401 for bad credentials, and the whole login round-trip (browser → our backend → real Emby → real rejection → our error message) renders correctly end to end. The --theme-primary-color-* etc. custom properties client/css/app.css is built from were read directly off this exact server's live dark theme, not guessed.

Not yet exercised against the real server — a successful login hasn't happened yet (no real credentials used in testing), so nothing past that point is confirmed:

• /Users/{userId} for token validation was dropped — the proxy redesign no longer needs it at all (sessions are validated once at login and cached server-side, not re-checked against Emby per-request).
• Streaming: the first version hand-built /Videos/{id}/master.m3u8?VideoCodec=h264&AudioCodec=aac directly, and it broke in the field: Emby's server threw ArgumentNullException: Value cannot be null. (Parameter 'key') — almost certainly an internal dictionary keyed by something that URL never supplied (MediaSourceId, PlaySessionId, DeviceId are the usual suspects). Fixed by calling GET /Items/{id}/PlaybackInfo?UserId=... first and using the MediaSource's own TranscodingUrl — the same thing real Emby clients do — falling back to a hand-built URL (now at least carrying MediaSourceId/PlaySessionId/UserId from that response) only if no TranscodingUrl comes back. Not yet confirmed against a real response: whether TranscodingUrl is reliably present, and whether it's always HLS (a direct-play-eligible source might not transcode at all, which hls.js on the client wouldn't handle - see Known limitations).
• HLS playlist rewriting (rewritePlaylist in embyProxy.js) — the logic is straightforward (rewrite every non-# line) but has only been reasoned about, never run against a playlist Emby actually generated. If video loads the manifest but segments 404 or hang, start here.
• Response field names (AccessToken, User.Id, User.Name, ServerId, Items[].Id/Name/Overview/ProductionYear/Type, /Users/{id}/Views returning library folders the same shape as regular items) — long-standing Emby/Jellyfin convention, not independently re-confirmed for this Emby version.
• X-Emby-Authorization header format — same caveat as above.
• The Emby web client deep-link shape (/web/index.html#!/item?id=X&serverId=Y) used for "Voir sur Emby" — matches the long-standing convention, not confirmed against this exact build's routing.

Home page: library tabs, tiles, ongoing parties

client/js/views/browse.js (the home page, route /browse) is now:

• A "watch parties en cours" section listing every active party server-wide (GET /api/parties — deliberately not filtered to "your" parties, since the point is letting people discover and join one without needing a code passed to them separately), each with a direct "Rejoindre".
• A join-by-code fallback below it, for a party that isn't visible there for whatever reason.
• One tab per Emby library (GET /api/catalog/libraries, i.e. Emby's /Users/{id}/Views), each showing that library's top-level items (GET /api/catalog/libraries/:id/items, Recursive=false — Movies as movies, TV shows as Series tiles, not every episode flattened together).
• Tiles: poster + name, with a hover-revealed overlay carrying two actions - "Lancer une watch party" (only for directly-playable types: Movie/Episode/Video — a Series can't be streamed as-is, so it only gets the second action) and "Voir sur Emby" (opens Emby's own web client to that item in a new tab, via a embyUrl the backend builds from EMBY_SERVER_URL + the session's serverId, captured at login). :hover/:focus-within reveal the overlay on mouse/keyboard; a @media (hover: none) fallback keeps it always visible (smaller, inline) on touch devices, which have no hover state to trigger it at all.
• Full width, no max-width cap (.home), with a plain CSS grid (repeat(auto-fill, minmax(160px, 1fr))) doing all the responsive reflowing — no breakpoints needed.

Confirmed by actually driving a browser against this (see session history): tab switching swaps grid content correctly, the hover overlay shows/hides per-tile (not globally), a Series tile correctly omits the watch-party button, and the touch fallback ((hover: none), tested via mobile viewport emulation) shows both buttons inline without needing hover at all. That test used a mocked fetch (canned JSON standing in for /api/parties and /api/catalog/...) since exercising it for real still needs a logged-in session — so the rendering/interaction logic is verified, but the real Emby response shapes for /Views and library items are not (see below).

Known limitations

• Library tabs show top-level items only — no drilling into a Series' seasons/episodes yet (a Series tile links out to Emby instead, see above).
• Playback always goes through hls.js against whatever Emby's PlaybackInfo suggests. If that ever comes back as a direct-play source with no TranscodingUrl (no transcoding needed/possible), the client has no fallback path yet - it'll try to hand hls.js something that isn't HLS.
• One item per party; no shared queue.
• A party member needs a browser tab open and connected for sync to apply — there's no "catch up automatically when you come back" beyond whatever the next heartbeat corrects.
• No persistence: server restart or long idle time ends every party.