watchparty/docs/architecture.md

# Architecture

## Goals

- Watch the same title, in sync, with other users of the same Emby server.
- Use Emby as the media source (catalog + streaming) without requiring
  changes to Emby itself or its official clients.
- Access gated to people with an Emby account on this server — no anonymous
  use, no separate identity system.

## Non-goals

- Chat/reactions — deliberately out of scope.
- OIDC/SSO — an OIDC provider (LemonLDAP::NG) is available in principle, but
  the explicit choice was Emby-only auth: simpler, no extra infra dependency,
  and everyone already has an Emby account.
- Full library navigation (Shows → Seasons → Episodes) in v1 — search-based
  catalog only, to keep the first version achievable. Revisit if browsing
  turns out to matter more than searching in practice.

## Why this replaced an Emby plugin

The first attempt (still in [`src/WatchParty/`](../src/WatchParty/), kept for
reference) was a real Emby Server plugin: sync via `ISessionManager`'s
existing remote-control commands, a settings page via `IHasWebPages`, and —
after a fair amount of reverse-engineering — a real user-facing page
integrated into Emby's own web client menu (`PluginPageInfo.EnableInUserMenu`,
undocumented but confirmed working; required a `data-controller` AMD module
to satisfy Emby's modern `ViewManager`, found by reading the target server's
actual `viewmanager.js`).

That plugin builds and installs correctly. What it hit a hard wall on: **a
button inside the video player itself**, or on an item's detail page. An
Emby developer confirmed on their own forum that plugins have no API for
this ("It's currently not possible for a plugin to do this"), and nothing
found in research since suggests that's changed. The only way to get a
button into Emby's own player would be patching Emby's shipped web client
files directly inside the Docker container — outside the plugin entirely,
guaranteed to break on every Emby image update, and requiring manual
reapplication each time. That trade-off wasn't worth it.

Building a standalone app sidesteps the question entirely: it's our own
player, so there's no "how do I add a button to someone else's UI" problem
in the first place, and sync can be tighter than nudging Emby's
remote-control commands ever allowed (see below).

Full trace of the plugin investigation (real errors, real fixes, sources
consulted) is in the git history of this file and of `src/WatchParty/` —
worth reading if a future Emby SDK update ever makes in-player integration
possible and this gets revisited.

## Components

```
server/   Node.js + Express + ws. Owns party/room state and the realtime
          sync channel. Never touches a media byte - see "Media never
          flows through our backend" below.

client/   Static frontend, plain JS + ES modules, no build step (same bias
          as the earlier plugin's control panel). Talks to two different
          things and must not confuse them:
            - Emby directly (embyApi.js): catalog, images, streaming.
            - Our own backend (party.js): create/join/leave a party, and
              the WebSocket for realtime sync.
```

### Auth and Emby access: the server proxies everything

Revised after the first version: the browser used to call Emby directly
(catalog, images, streaming) and only used `server/` for party coordination.
That hit real CORS problems in practice (Emby/its reverse proxy don't
answer cross-origin preflights the way that needed), and separately wasn't
the wanted shape anyway — the browser holding a real Emby access token at
all was more exposure than necessary. Current design:

- The browser sends a username/password **once**, to
  `POST /api/auth/login` on **our own backend** — never to Emby directly.
- Our backend (`server/src/embyAuth.js`) calls Emby's
  `POST /Users/AuthenticateByName` itself, using `EMBY_SERVER_URL` from its
  own environment (not something the user ever types — see `.env.example`).
  On success it mints its own opaque session id, stores
  `{embyToken, userId, userName}` server-side (in-memory, like party state),
  and hands the browser only that session id, as an `httpOnly` cookie
  (`wp_session`). The browser never sees or holds Emby's real access token.
- Every other request — catalog search, item details, images, HLS
  streaming, party actions — goes to **our own origin only**
  (`/api/...`), authenticated by that same cookie. `server/src/embyProxy.js`
  is what actually forwards these to Emby server-side and streams the
  response back, attaching the session's real Emby token itself.
- The `/api/catalog/stream/:itemId/master.m3u8` and `.../raw` routes proxy
  HLS specifically: the master playlist is fetched from Emby server-side,
  and every URI line in it is rewritten to point back through `.../raw`
  (whether Emby wrote it as relative or absolute) — otherwise hls.js would
  follow the playlist's own URLs straight to Emby the moment it tried to
  fetch a segment, defeating the whole point. See `rewritePlaylist` in
  `embyProxy.js`.
- Images (`<img src>`) and video (`hls.js`/`<video>`) still can't carry a
  bearer header, but that's fine now: they're same-origin requests to our
  own `/api/...` paths, so the session cookie rides along automatically —
  no token in any URL anywhere, unlike the first version's `?api_key=...`.

This also means the client no longer needs to ask for a server URL at all —
one less field, one less way to get it wrong. `EMBY_SERVER_URL` lives only
in the backend's environment.

### Control model: host drives, everyone else follows

Now that we own the player, the classic watch-party model (Teleparty,
Amazon Watch Party, ...) fits better than the old plugin's "everyone keeps
their own controls, shared reference position" — that design was specifically
a concession to nudging *pre-existing, independently-running* Emby app
sessions we didn't control. Here, there's only one player implementation,
so:

- The party creator is the host. Their player has normal controls.
- Everyone else's `<video>` has controls hidden; a small notice says the
  host is in charge.
- Host → server → everyone else, over the WebSocket
  (`server/src/wsHandler.js`):
  - Event-driven: every `play`/`pause`/`seeked` on the host's video sends
    `{type:"playstate", isPaused, positionTicks}` immediately.
  - Heartbeat: also sent every 3s regardless, so latecomers/drift get
    corrected even without a discrete host action.
  - Followers apply pause/unpause immediately, and only seek if drift
    exceeds 1.5s (`DRIFT_TOLERANCE_SECONDS` in `client/js/views/player.js`)
    — avoids fighting small timing noise with constant re-seeks.
- If the host leaves, the server promotes another member (same
  fallback logic the old plugin used for its "reference" member).

### Party lifecycle

In-memory only on the backend (`server/src/partyManager.js`) — a restart
clears every party, which is fine: a party is meant to last one watch
session, not persist for weeks. `removeIdleParties` sweeps anything with no
activity for 4 hours.

## Assumptions to verify

**Confirmed against the real server** (not a guess — actually exercised,
see session history): `POST /Users/AuthenticateByName` really is reachable
from Node's server-side `fetch` with no CORS issue at all (server-to-server
has no CORS concept), returns a real 401 for bad credentials, and the whole
login round-trip (browser → our backend → real Emby → real rejection → our
error message) renders correctly end to end. The `--theme-primary-color-*`
etc. custom properties `client/css/app.css` is built from were read directly
off this exact server's live dark theme, not guessed.

**Not yet exercised against the real server** — a successful login hasn't
happened yet (no real credentials used in testing), so nothing past that
point is confirmed:

- **`/Users/{userId}` for token validation** was dropped — the proxy
  redesign no longer needs it at all (sessions are validated once at login
  and cached server-side, not re-checked against Emby per-request).
- **Streaming**: the first version hand-built
  `/Videos/{id}/master.m3u8?VideoCodec=h264&AudioCodec=aac` directly, and it
  broke in the field: Emby's server threw `ArgumentNullException: Value
  cannot be null. (Parameter 'key')` — almost certainly an internal
  dictionary keyed by something that URL never supplied (`MediaSourceId`,
  `PlaySessionId`, `DeviceId` are the usual suspects). Fixed by calling
  `GET /Items/{id}/PlaybackInfo?UserId=...` first and using the
  `MediaSource`'s own `TranscodingUrl` — the same thing real Emby clients do
  — falling back to a hand-built URL (now at least carrying
  `MediaSourceId`/`PlaySessionId`/`UserId` from that response) only if no
  `TranscodingUrl` comes back. Not yet confirmed against a real response:
  whether `TranscodingUrl` is reliably present, and whether it's always HLS
  (a direct-play-eligible source might not transcode at all, which `hls.js`
  on the client wouldn't handle - see Known limitations).
- **HLS playlist rewriting** (`rewritePlaylist` in `embyProxy.js`) — the
  logic is straightforward (rewrite every non-`#` line) but has only been
  reasoned about, never run against a playlist Emby actually generated. If
  video loads the manifest but segments 404 or hang, start here.
- **Response field names** (`AccessToken`, `User.Id`, `User.Name`,
  `ServerId`, `Items[].Id/Name/Overview/ProductionYear/Type`,
  `/Users/{id}/Views` returning library folders the same shape as regular
  items) — long-standing Emby/Jellyfin convention, not independently
  re-confirmed for this Emby version.
- **`X-Emby-Authorization` header format** — same caveat as above.
- **The Emby web client deep-link shape**
  (`/web/index.html#!/item?id=X&serverId=Y`) used for "Voir sur Emby" —
  matches the long-standing convention, not confirmed against this exact
  build's routing.

## UI: Emby-like shell, per-type browsing, episode queue

Reworked to look and navigate more like Emby itself, and to treat each
collection type (movies / TV / music) as its own experience rather than one
generic grid:

- **`client/js/shell.js`** renders the persistent chrome once per login: a
  sidebar (collections from `GET /api/catalog/libraries`, one icon per
  `collectionType`) + a `#mainContent` element the router (`router.js`)
  swaps on every navigation. `router.js` was made idempotent
  (`start(root)` can be called again safely) specifically so login → shell
  boot can happen more than once in a page's lifetime without piling up
  duplicate `hashchange` listeners — the same class of bug the earlier Emby
  plugin work ran into with its page lifecycle, worth remembering.
- **`views/home.js`** (route `/home`): a horizontal carousel of every active
  party server-wide (`GET /api/parties`, deliberately not filtered to "your"
  parties — the point is letting people discover and join one without a
  code passed out of band) plus a join-by-code fallback, then one more
  carousel per library of its recently-added items
  (`GET /api/catalog/libraries/:id/recent`, Emby's own
  `/Users/{id}/Items/Latest`) — skipping music (see below).
- **`views/library.js`** (route `/library/:id`): dispatches purely on the
  library's `collectionType`. `movies`/`tvshows`/anything else gets a tile
  grid (`GET /api/catalog/libraries/:id/items`, `Recursive=false` — Movies
  as movies, TV as Series tiles); `music` gets an honest "not built yet"
  message instead of a half-working page (see Non-goals).
- **Tiles** (`client/js/tiles.js`, shared by the home carousels and the
  library grid so they can't drift apart): poster + name, hover-revealed
  overlay with "Lancer une watch party" (only for directly-playable types —
  a Series tile gets "Voir les épisodes" instead, linking to `/series/:id`)
  and "Voir sur Emby" (opens Emby's own web client via a `embyUrl` the
  backend builds from `EMBY_SERVER_URL` + the session's `serverId`).
  `:hover`/`:focus-within` reveal it on mouse/keyboard; `@media (hover: none)`
  keeps it always visible on touch devices, which have no hover state at all.
- **`views/series.js`** (route `/series/:id`): season tabs
  (`GET /api/catalog/series/:id/seasons`, Emby's `/Shows/{id}/Seasons`) and
  an episode list per season (`GET /api/catalog/series/:id/episodes`, Emby's
  `/Shows/{id}/Episodes?SeasonId=...`), each episode startable as its own
  party.

### Episode queue / "next episode"

Starting a party from an episode (`POST /api/parties`) now has the backend
fetch the rest of that series (`/Shows/{seriesId}/Episodes`, no season
filter) and store everything from the clicked episode onward as the party's
`queue` (`server/src/partyManager.js`). The host gets an "Épisode suivant"
button once `hasNext` is true; clicking it (or the video's `ended` event
firing) sends a `{type:"next"}` WebSocket message
(`server/src/wsHandler.js`), host-only, which advances `queueIndex`,
resets every member's known position, and broadcasts `{type:"itemChanged",
itemId, itemName, ...}` — every client (including the host) reloads the
player against the new `itemId` on that message. A movie or a
standalone-started episode just gets a one-item queue, so this is a strict
superset of the old single-item behavior, not a separate code path.

### Player: language, subtitles, quality

`GET /api/catalog/item/:id/playback-options` reads `PlaybackInfo`'s
`MediaSources[0].MediaStreams` and returns the Audio/Subtitle entries as
`{index, label}` pairs for two `<select>`s in the player's host-only
controls. Picking a value reloads the stream
(`/api/catalog/stream/:id/master.m3u8?AudioStreamIndex=&SubtitleStreamIndex=&MaxStreamingBitrate=`)
at the same position (`hls.js` destroyed and rebuilt, `currentTime`/paused
state preserved across the reload) rather than restarting from zero.
"Quality" is a fixed set of `MaxStreamingBitrate` presets (Auto/1080p/720p/
480p) forwarded to `PlaybackInfo` — the same simplification Emby's/
Jellyfin's own web clients use; it's a transcode ceiling, not true
client-side adaptive bitrate switching. Subtitles picked this way get
burned into the transcoded video (Emby's doing, via `SubtitleStreamIndex`)
rather than rendered as a toggleable `<track>` — simpler to implement
correctly than sidecar WebVTT, at the cost of a re-transcode to turn them
on/off.

### Confirmed vs. not, for all of the above

Driving an actual browser against this (see session history) confirms the
**rendering and interaction logic**: sidebar highlighting, home carousels
(parties + one row per non-music library), library grid dispatch, and the
series page's season-tab switching + episode list all render and navigate
correctly. That test mocked every `/api/...` response (canned JSON), since
exercising it for real needs a logged-in session — so what's **not**
confirmed yet is the shape of the real Emby responses this now depends on:
`Views` items actually carrying a `CollectionType` field the way assumed,
`/Shows/{id}/Seasons` and `/Shows/{id}/Episodes`, `Items/Latest`'s
plain-array response shape (handled defensively — see `index.js` — but not
seen for real), and `PlaybackInfo`'s `MediaStreams` shape for the track
pickers. The queue/"next episode" WebSocket flow and the player's
track/quality reload are new protocol surface with **no test coverage at
all yet** (no way to exercise real HLS playback without a live session) —
the first things to check once a real party actually plays something.

## Non-goals (updated)

**Music** was requested (dedicated audio player, artist/album browsing, a
shared/collaborative playlist party members can add to together) but is
deliberately not in this pass: it's a genuinely different subsystem — a
different player paradigm, full library-depth browsing down to the track,
and real-time collaborative queue state that doesn't fit the existing
"host drives, everyone follows" model at all (multiple people adding to one
shared list is a different kind of shared state than one host's playback
position). The sidebar and library routing already treat `music` as a first
-class `collectionType` specifically so slotting in a real implementation
later doesn't need another structural pass — see `library.js`'s early
return. Right now it just shows an honest placeholder.

## Known limitations

- Playback always goes through `hls.js` against whatever Emby's
  `PlaybackInfo` suggests. If that ever comes back as a direct-play source
  with no `TranscodingUrl` (no transcoding needed/possible), the client has
  no fallback path yet - it'll try to hand hls.js something that isn't HLS.
- One item per party; no shared queue.
- A party member needs a browser tab open and connected for sync to apply —
  there's no "catch up automatically when you come back" beyond whatever
  the next heartbeat corrects.
- No persistence: server restart or long idle time ends every party.