watchparty/docs/architecture.md

# Architecture

## Goals

- Watch the same title, in sync, with other users of the same Emby server.
- Use Emby as the media source (catalog + streaming) without requiring
  changes to Emby itself or its official clients.
- Access gated to people with an Emby account on this server — no anonymous
  use, no separate identity system.

## Non-goals

- Chat/reactions — deliberately out of scope.
- OIDC/SSO — an OIDC provider (LemonLDAP::NG) is available in principle, but
  the explicit choice was Emby-only auth: simpler, no extra infra dependency,
  and everyone already has an Emby account.
- Full library navigation (Shows → Seasons → Episodes) in v1 — search-based
  catalog only, to keep the first version achievable. Revisit if browsing
  turns out to matter more than searching in practice.

## Why this replaced an Emby plugin

The first attempt (still in [`src/WatchParty/`](../src/WatchParty/), kept for
reference) was a real Emby Server plugin: sync via `ISessionManager`'s
existing remote-control commands, a settings page via `IHasWebPages`, and —
after a fair amount of reverse-engineering — a real user-facing page
integrated into Emby's own web client menu (`PluginPageInfo.EnableInUserMenu`,
undocumented but confirmed working; required a `data-controller` AMD module
to satisfy Emby's modern `ViewManager`, found by reading the target server's
actual `viewmanager.js`).

That plugin builds and installs correctly. What it hit a hard wall on: **a
button inside the video player itself**, or on an item's detail page. An
Emby developer confirmed on their own forum that plugins have no API for
this ("It's currently not possible for a plugin to do this"), and nothing
found in research since suggests that's changed. The only way to get a
button into Emby's own player would be patching Emby's shipped web client
files directly inside the Docker container — outside the plugin entirely,
guaranteed to break on every Emby image update, and requiring manual
reapplication each time. That trade-off wasn't worth it.

Building a standalone app sidesteps the question entirely: it's our own
player, so there's no "how do I add a button to someone else's UI" problem
in the first place, and sync can be tighter than nudging Emby's
remote-control commands ever allowed (see below).

Full trace of the plugin investigation (real errors, real fixes, sources
consulted) is in the git history of this file and of `src/WatchParty/` —
worth reading if a future Emby SDK update ever makes in-player integration
possible and this gets revisited.

## Components

```
server/   Node.js + Express + Socket.IO. Owns party/room state and the
          realtime sync channel. Never touches a media byte - see "Media
          never flows through our backend" below.

client/   Static frontend, plain JS + ES modules, no build step (same bias
          as the earlier plugin's control panel). Talks to two different
          things and must not confuse them:
            - Emby directly (embyApi.js): catalog, images, streaming.
            - Our own backend (party.js): create/join/leave a party, and
              the Socket.IO connection for realtime sync.
```

### Auth and Emby access: the server proxies everything

Revised after the first version: the browser used to call Emby directly
(catalog, images, streaming) and only used `server/` for party coordination.
That hit real CORS problems in practice (Emby/its reverse proxy don't
answer cross-origin preflights the way that needed), and separately wasn't
the wanted shape anyway — the browser holding a real Emby access token at
all was more exposure than necessary. Current design:

- The browser sends a username/password **once**, to
  `POST /api/auth/login` on **our own backend** — never to Emby directly.
- Our backend (`server/src/embyAuth.js`) calls Emby's
  `POST /Users/AuthenticateByName` itself, using `EMBY_SERVER_URL` from its
  own environment (not something the user ever types — see `.env.example`).
  On success it mints its own opaque session id, stores
  `{embyToken, userId, userName}` server-side (in-memory, like party state),
  and hands the browser only that session id, as an `httpOnly` cookie
  (`wp_session`). The browser never sees or holds Emby's real access token.
- Every other request — catalog search, item details, images, HLS
  streaming, party actions — goes to **our own origin only**
  (`/api/...`), authenticated by that same cookie. `server/src/embyProxy.js`
  is what actually forwards these to Emby server-side and streams the
  response back, attaching the session's real Emby token itself.
- The `/api/catalog/stream/:itemId/master.m3u8` and `.../raw` routes proxy
  HLS specifically: the master playlist is fetched from Emby server-side,
  and every URI line in it is rewritten to point back through `.../raw`
  (whether Emby wrote it as relative or absolute) — otherwise hls.js would
  follow the playlist's own URLs straight to Emby the moment it tried to
  fetch a segment, defeating the whole point. See `rewritePlaylist` in
  `embyProxy.js`.
- Images (`<img src>`) and video (`hls.js`/`<video>`) still can't carry a
  bearer header, but that's fine now: they're same-origin requests to our
  own `/api/...` paths, so the session cookie rides along automatically —
  no token in any URL anywhere, unlike the first version's `?api_key=...`.

This also means the client no longer needs to ask for a server URL at all —
one less field, one less way to get it wrong. `EMBY_SERVER_URL` lives only
in the backend's environment.

### Control model: host drives, everyone else follows

Now that we own the player, the classic watch-party model (Teleparty,
Amazon Watch Party, ...) fits better than the old plugin's "everyone keeps
their own controls, shared reference position" — that design was specifically
a concession to nudging *pre-existing, independently-running* Emby app
sessions we didn't control. Here, there's only one player implementation,
so:

- The party creator is the host. Their player has normal controls.
- Everyone else's `<video>` has controls hidden; a small notice says the
  host is in charge.
- Host → server → everyone else, over Socket.IO
  (`server/src/socketHandler.js`):
  - Event-driven: every `play`/`pause`/`seeked` on the host's video sends
    `{type:"playstate", isPaused, positionTicks}` immediately.
  - Heartbeat: also sent every 3s regardless, so latecomers/drift get
    corrected even without a discrete host action.
  - Followers apply pause/unpause immediately, and only seek if drift
    exceeds 1.5s (`DRIFT_TOLERANCE_SECONDS` in `client/js/views/player.js`)
    — avoids fighting small timing noise with constant re-seeks.
- If the host leaves, the server promotes another member (same
  fallback logic the old plugin used for its "reference" member).

### Party lifecycle

In-memory only on the backend (`server/src/partyManager.js`) — a restart
clears every party, which is fine: a party is meant to last one watch
session, not persist for weeks. `removeIdleParties` sweeps anything with no
activity for 4 hours.

## Realtime transport: Socket.IO, and two real memory leaks

The realtime layer was originally a raw WebSocket (the `ws` package). Two
things prompted a rework — a requirement (use Socket.IO instead) and a
specific worry (memory leaks) — and auditing for the second turned up two
real, concrete leaks, independent of which transport library is used:

1. **Server-side: dead connections never got cleaned up.** A raw `ws`
   server has no idea a client is gone unless it receives a proper close
   frame. A laptop put to sleep, a network drop, a killed tab that never
   gets to send one — none of those fire `ws`'s `'close'` event, so the
   `PartyMember`'s socket reference (and everything it closes over) would
   sit in memory forever, and the party would never get cleaned up by
   anything (`removeIdleParties` only looks at `lastActivityUtc`, which a
   phantom connection doesn't affect). `ws`'s own docs recommend a manual
   ping/pong heartbeat to detect this - Socket.IO has one built in
   (`pingInterval`/`pingTimeout`), which is a real, not just cosmetic,
   reason to prefer it here, not only the requested API ergonomics.
2. **Client-side: leaving the player any way other than its own "Quitter"
   button leaked everything.** `router.js` only ever did
   `currentRoot.innerHTML = ''` between routes — that destroys DOM nodes,
   but nothing told the outgoing view's *JavaScript* to stop. Clicking a
   sidebar link or the browser's back button while watching left the
   player's 3-second heartbeat `setInterval`, its realtime connection, and
   its `hls.js` instance all running against a detached `<video>` element
   indefinitely - for as long as the tab stayed open. Fixed by having
   `route()` handlers optionally return a cleanup function; `router.js`
   calls whatever the previous view returned right before rendering the
   next one, regardless of how navigation happened. `player.js` returns one
   that clears the heartbeat, destroys `hls`, disconnects the socket, and
   removes the one `document`-level click listener it adds (named, not
   inline-anonymous, specifically so it's removable at all - an anonymous
   listener added fresh on every visit would itself have been a slow leak
   even with everything else fixed).

Both are now covered by the same class of test: drive the router through
several "join a party, then navigate away without clicking Quitter" cycles
and check nothing accumulates. Verified directly (see session history) by
instrumenting `document.addEventListener`/`removeEventListener` and
`setInterval`/`clearInterval` around three such cycles against a really-running
local server (Socket.IO connection included, not mocked) - adds and removes
matched exactly (3/3 listeners, 2/2 intervals across two cycles), nothing
left pending. The server-side heartbeat fix is Socket.IO's own
well-established mechanism, not custom code, so it wasn't separately
instrumented the same way.

`server/src/socketHandler.js` mirrors the old `wsHandler.js` protocol
closely (see its own top comment for the exact event list) - the one naming
difference is the error event, sent as `"errorMessage"` rather than
`"error"`, since Socket.IO reserves the latter for its own internal error
handling.

## Live lobby, host handoff, no join codes

Three related changes, all in service of "the home page should reflect
reality without a refresh":

- **The home page's party list is a live subscription, not a fetch.**
  `client/js/party.js`'s `connectLobby` joins a `"lobby"` Socket.IO room
  (`subscribeLobby` → `server/src/socketHandler.js`) and gets a fresh
  snapshot on subscribe, then again every time `broadcastLobby(io)` runs -
  which is now wired into every place party membership changes: creating a
  party, joining one (both in `index.js`'s REST handlers), and leaving
  (`applyLeave`, used by both the socket `leave`/`disconnect` handlers and
  the REST leave endpoint, so this - and the host-handoff notification below
  - only has to be right in one place). `home.js` returns this subscription
  as part of its router cleanup function, same as the player does for its
  own socket - an open lobby subscription left running after navigating away
  would have been exactly the same class of leak just fixed above.
- **Host and elapsed time are shown per party**, per the request. Host name
  and member count come from the same snapshot. Elapsed time doesn't: it
  ticks client-side from a `createdUtc` timestamp the server includes once
  (`home.js`'s `setInterval` re-renders every `.elapsed` label every
  second) - pushing a recomputed duration from the server every second for
  every viewer would be pure waste when the client can derive it from one
  number. Verified live (see session history): faking a `parties` push with
  a `createdUtc` two minutes in the past and waiting rendered "2:06", not a
  static value.
- **No join codes.** `PartyManager` no longer generates or tracks one - see
  its class comment. The home page's live list is now the only way to find
  a party; a code that has to be copied and typed out of band doesn't add
  anything on top of a list that's already always current. `POST
  /api/parties/:partyId/join` replaces the old `:joinCode` route.
- **Host handoff is live, not just correct in the data model.**
  `PartyManager.leave` already promoted another member to host when the
  host left and destroyed the party once empty - that part predates this
  change. What was missing: a player already open as a follower had no way
  to find out it became host mid-session. `applyLeave` now emits a personal
  `"hostChanged"` (to that one socket, not the room) when this happens;
  `player.js`'s `becomeHost()` - the same function `init()` calls when you
  already are the host - unlocks the controls live, no reload needed.
  `becomeHost` is written to tolerate being called more than once safely:
  `addEventListener` with the same function reference is a no-op if already
  attached, and the heartbeat `setInterval` is explicitly guarded
  (`if (!heartbeat)`) since `setInterval` has no such built-in idempotence -
  calling it twice would leak the first interval's id, the exact class of
  bug the rest of this pass was about fixing, not reintroducing.

One incidental bug found and fixed while touching this code: the home
page's "Rejoindre" button on a live party card used to `navigate()` straight
to the player **without ever calling the join endpoint first** - it worked
well enough to load the video and read basic party info (`GET
/api/parties/:id` never checked membership), but the realtime `"join"`
would be rejected server-side ("Tu n'es pas membre de cette party.") since
`socketHandler.js`'s join handler does check membership - meaning sync
silently never worked for anyone who joined this way. Fixed by awaiting
`joinParty()` before navigating, same as the library grid's tiles already
did correctly.

## Player: no page scroll

`.playerWrap` was `width:100%; height:100vh` sitting inside `.mainContent`,
which has its own padding - the two stacked (100vh *plus* padding) made the
whole shell taller than the viewport, so the page grew a scrollbar even
though the video itself was already full-height. Changed to
`position:fixed; inset:0` so it's sized against the real viewport
regardless of any ancestor's padding, and (like most players' theater mode)
covers the sidebar while watching - confirmed via computed style
(`position: fixed`, all four insets `0px`) rather than just asserted.

## Assumptions to verify

**Confirmed against the real server** (not a guess — actually exercised,
see session history): `POST /Users/AuthenticateByName` really is reachable
from Node's server-side `fetch` with no CORS issue at all (server-to-server
has no CORS concept), returns a real 401 for bad credentials, and the whole
login round-trip (browser → our backend → real Emby → real rejection → our
error message) renders correctly end to end. The `--theme-primary-color-*`
etc. custom properties `client/css/app.css` is built from were read directly
off this exact server's live dark theme, not guessed.

**Not yet exercised against the real server** — a successful login hasn't
happened yet (no real credentials used in testing), so nothing past that
point is confirmed:

- **`/Users/{userId}` for token validation** was dropped — the proxy
  redesign no longer needs it at all (sessions are validated once at login
  and cached server-side, not re-checked against Emby per-request).
- **Streaming**: the first version hand-built
  `/Videos/{id}/master.m3u8?VideoCodec=h264&AudioCodec=aac` directly, and it
  broke in the field: Emby's server threw `ArgumentNullException: Value
  cannot be null. (Parameter 'key')` — almost certainly an internal
  dictionary keyed by something that URL never supplied (`MediaSourceId`,
  `PlaySessionId`, `DeviceId` are the usual suspects). Fixed by calling
  `GET /Items/{id}/PlaybackInfo?UserId=...` first and using the
  `MediaSource`'s own `TranscodingUrl` — the same thing real Emby clients do
  — falling back to a hand-built URL (now at least carrying
  `MediaSourceId`/`PlaySessionId`/`UserId` from that response) only if no
  `TranscodingUrl` comes back. Not yet confirmed against a real response:
  whether `TranscodingUrl` is reliably present, and whether it's always HLS
  (a direct-play-eligible source might not transcode at all, which `hls.js`
  on the client wouldn't handle - see Known limitations).
- **HLS playlist rewriting** (`rewritePlaylist` in `embyProxy.js`) — the
  logic is straightforward (rewrite every non-`#` line) but has only been
  reasoned about, never run against a playlist Emby actually generated. If
  video loads the manifest but segments 404 or hang, start here.
- **Response field names** (`AccessToken`, `User.Id`, `User.Name`,
  `ServerId`, `Items[].Id/Name/Overview/ProductionYear/Type`,
  `/Users/{id}/Views` returning library folders the same shape as regular
  items) — long-standing Emby/Jellyfin convention, not independently
  re-confirmed for this Emby version.
- **`X-Emby-Authorization` header format** — same caveat as above.
- **The Emby web client deep-link shape**
  (`/web/index.html#!/item?id=X&serverId=Y`) used for "Voir sur Emby" —
  matches the long-standing convention, not confirmed against this exact
  build's routing.

## UI: Emby-like shell, per-type browsing, episode queue

Reworked to look and navigate more like Emby itself, and to treat each
collection type (movies / TV / music) as its own experience rather than one
generic grid:

- **`client/js/shell.js`** renders the persistent chrome once per login: a
  sidebar (collections from `GET /api/catalog/libraries`, one icon per
  `collectionType`) + a `#mainContent` element the router (`router.js`)
  swaps on every navigation. `router.js` was made idempotent
  (`start(root)` can be called again safely) specifically so login → shell
  boot can happen more than once in a page's lifetime without piling up
  duplicate `hashchange` listeners — the same class of bug the earlier Emby
  plugin work ran into with its page lifecycle, worth remembering.
- **`views/home.js`** (route `/home`): a horizontal carousel of every active
  party server-wide (`GET /api/parties`, deliberately not filtered to "your"
  parties — the point is letting people discover and join one without a
  code passed out of band) plus a join-by-code fallback, then one more
  carousel per library of its recently-added items
  (`GET /api/catalog/libraries/:id/recent`, Emby's own
  `/Users/{id}/Items/Latest`) — skipping music (see below).
- **`views/library.js`** (route `/library/:id`): dispatches purely on the
  library's `collectionType`. `movies`/`tvshows`/anything else gets a tile
  grid (`GET /api/catalog/libraries/:id/items`, `Recursive=false` — Movies
  as movies, TV as Series tiles); `music` gets an honest "not built yet"
  message instead of a half-working page (see Non-goals).
- **Tiles** (`client/js/tiles.js`, shared by the home carousels and the
  library grid so they can't drift apart): poster + name, hover-revealed
  overlay with "Lancer une watch party" (only for directly-playable types —
  a Series tile gets "Voir les épisodes" instead, linking to `/series/:id`)
  and "Voir sur Emby" (opens Emby's own web client via a `embyUrl` the
  backend builds from `EMBY_SERVER_URL` + the session's `serverId`).
  `:hover`/`:focus-within` reveal it on mouse/keyboard; `@media (hover: none)`
  keeps it always visible on touch devices, which have no hover state at all.
- **`views/series.js`** (route `/series/:id`): season tabs
  (`GET /api/catalog/series/:id/seasons`, Emby's `/Shows/{id}/Seasons`) and
  an episode list per season (`GET /api/catalog/series/:id/episodes`, Emby's
  `/Shows/{id}/Episodes?SeasonId=...`), each episode startable as its own
  party.

### Episode queue / "next episode"

Starting a party from an episode (`POST /api/parties`) now has the backend
fetch the rest of that series (`/Shows/{seriesId}/Episodes`, no season
filter) and store everything from the clicked episode onward as the party's
`queue` (`server/src/partyManager.js`). The host gets an "Épisode suivant"
button once `hasNext` is true; clicking it (or the video's `ended` event
firing) sends a `"next"` Socket.IO event
(`server/src/socketHandler.js`), host-only, which advances `queueIndex`,
resets every member's known position, and broadcasts `{type:"itemChanged",
itemId, itemName, ...}` — every client (including the host) reloads the
player against the new `itemId` on that message. A movie or a
standalone-started episode just gets a one-item queue, so this is a strict
superset of the old single-item behavior, not a separate code path.

### Player: language, subtitles, quality

`GET /api/catalog/item/:id/playback-options` reads `PlaybackInfo`'s
`MediaSources[0].MediaStreams` and returns the Audio/Subtitle entries as
`{index, label}` pairs for two `<select>`s in the player's host-only
controls. Picking a value reloads the stream
(`/api/catalog/stream/:id/master.m3u8?AudioStreamIndex=&SubtitleStreamIndex=&MaxStreamingBitrate=`)
at the same position (`hls.js` destroyed and rebuilt, `currentTime`/paused
state preserved across the reload) rather than restarting from zero.
"Quality" is a fixed set of `MaxStreamingBitrate` presets (Auto/1080p/720p/
480p) forwarded to `PlaybackInfo` — the same simplification Emby's/
Jellyfin's own web clients use; it's a transcode ceiling, not true
client-side adaptive bitrate switching. Subtitles picked this way get
burned into the transcoded video (Emby's doing, via `SubtitleStreamIndex`)
rather than rendered as a toggleable `<track>` — simpler to implement
correctly than sidecar WebVTT, at the cost of a re-transcode to turn them
on/off.

### Confirmed vs. not, for all of the above

Driving an actual browser against this (see session history) confirms the
**rendering and interaction logic**: sidebar highlighting, home carousels
(parties + one row per non-music library), library grid dispatch, and the
series page's season-tab switching + episode list all render and navigate
correctly. That test mocked every `/api/...` response (canned JSON), since
exercising it for real needs a logged-in session — so what's **not**
confirmed yet is the shape of the real Emby responses this now depends on:
`Views` items actually carrying a `CollectionType` field the way assumed,
`/Shows/{id}/Seasons` and `/Shows/{id}/Episodes`, `Items/Latest`'s
plain-array response shape (handled defensively — see `index.js` — but not
seen for real), and `PlaybackInfo`'s `MediaStreams` shape for the track
pickers. The queue/"next episode" WebSocket flow and the player's
track/quality reload are new protocol surface with **no test coverage at
all yet** (no way to exercise real HLS playback without a live session) —
the first things to check once a real party actually plays something.

## Non-goals (updated)

**Music** was requested (dedicated audio player, artist/album browsing, a
shared/collaborative playlist party members can add to together) but is
deliberately not in this pass: it's a genuinely different subsystem — a
different player paradigm, full library-depth browsing down to the track,
and real-time collaborative queue state that doesn't fit the existing
"host drives, everyone follows" model at all (multiple people adding to one
shared list is a different kind of shared state than one host's playback
position). The sidebar and library routing already treat `music` as a first
-class `collectionType` specifically so slotting in a real implementation
later doesn't need another structural pass — see `library.js`'s early
return. Right now it just shows an honest placeholder.

## Known limitations

- Playback always goes through `hls.js` against whatever Emby's
  `PlaybackInfo` suggests. If that ever comes back as a direct-play source
  with no `TranscodingUrl` (no transcoding needed/possible), the client has
  no fallback path yet - it'll try to hand hls.js something that isn't HLS.
- One item per party; no shared queue.
- A party member needs a browser tab open and connected for sync to apply —
  there's no "catch up automatically when you come back" beyond whatever
  the next heartbeat corrects.
- No persistence: server restart or long idle time ends every party.