watchparty/README.md
Nicolas 26da8fc5dc
All checks were successful
Publier l'image Docker / test (push) Successful in 25s
Publier l'image Docker / publish (push) Successful in 30s
Self-hosted ffmpeg pipeline for PGS (image) subtitle burn-in
Asked for directly, right after the client-side <track> redesign closed
every *text* subtitle case: PGS/VobSub/DVB subtitles (Blu-ray/DVD rips)
are bitmaps, not text - there's nothing for Emby's Stream.vtt conversion
to extract, so a native <track> fundamentally can't render one. Rather
than keep asking Emby's own transcode to burn these in (the exact thing
that kept failing in earlier sessions - missing fontconfig, then an
unrelated "Error starting ffmpeg"), this runs ffmpeg ourselves: a PGS
stream decodes like a second video track, and ffmpeg's `overlay` filter
composites it directly - no text rendering, no libass, no fontconfig
anywhere in this path.

One shared stream per PARTY, not per viewer - explicitly asked for after
the initial plan: every member plugs into the exact same server-emitted
stream, mirroring "host drives, everyone follows" rather than a
per-viewer model. Picking a burn-in subtitle is therefore a host-only,
party-wide choice, like play/pause/next already are.

- server/src/services/TranscodeService.ts (new): one ffmpeg session per
  partyId. getOrStartSession negotiates PlaybackInfo (via the new
  CatalogService.getMediaSource), converts Emby's absolute
  MediaStreams[].Index into ffmpeg's type-relative 0:s:N/0🅰️N
  (absoluteToRelativeIndex - pure, unit-tested), and spawns ffmpeg
  against Emby's own direct/static file endpoint (TODO(verify) - no
  Emby server reachable from this dev environment, see the function's
  own comment). Output is real progressive HLS to a temp dir, served by
  two new routes that poll briefly for the next file. endSession/
  reapIdleSessions handle teardown; PartyService's destroy path and
  advanceQueue both end a party's session immediately, the idle sweep
  is the backstop.
- server/src/services/PartyService.ts: Party/PartyPublicState gain
  burnInSubtitleIndex/burnInAudioIndex (null together = no burn-in);
  advanceQueue clears them (a new item's indices mean nothing on the
  old one).
- server/src/realtime/SocketGateway.ts: new host-only "setBurnInSubtitle"
  handler broadcasting "burnInChanged" - same shape as "next"/
  "itemChanged".
- server/src/services/CatalogService.ts: new getMediaSource() (id/
  container/streams together - what TranscodeService needs); extended
  the local PlaybackInfoResponse type with Container.
- server/src/Logger.ts: new Logger.transcode().
- server/src/index.ts: GET /api/catalog/burn/:partyId/master.m3u8 and
  .../:segment (membership-checked, same pattern as the party REST
  routes); a dedicated idle-reap interval for TranscodeService.
- server/Dockerfile: `apk add ffmpeg` in the runtime stage - no font
  packages needed, unlike Emby's own text-subtitle burn-in.
- client/src/api/party.ts, client/src/api/embyApi.ts: setBurnInSubtitle/
  onBurnInChanged on PartyConnection, burnInStreamUrl().
- client/src/views/player/Player.{html,css,ts}: the subtitle picker's
  image-track options go from unconditionally disabled to host-only
  selectable, labeled to say picking one restarts the stream for
  everyone. Every member (host included) only ever reacts to the
  resulting "burnInChanged" broadcast - never applies a pick
  optimistically, same discipline the rest of the player already uses.
  Audio/quality are disabled while a burn-in is active (one fixed
  encode); a follower's subtitle select is disabled too (nothing left
  for them to meaningfully pick).
- server/test/transcodeService.test.ts (new): absoluteToRelativeIndex,
  session start/reuse/replace, endSession, reapIdleSessions -
  child_process.spawn mocked via a small explicit test seam
  (setSpawnForTesting), never invoking a real ffmpeg binary.
- server/test/socketGateway.test.ts, partyService.test.ts,
  catalogService.test.ts: host-only gating + broadcast for
  setBurnInSubtitle, advanceQueue's reset, toPublicState exposing the
  new fields, getMediaSource.
- docs/architecture.md, README.md: design + an explicit "assumptions to
  verify" note (the direct-source URL shape, first and foremost).

Verified: full typecheck+test+build on both projects (56/56 server
tests). No Emby server or ffmpeg binary is reachable from this dev
environment, so the actual command against a real PGS file is
unverified - flagged explicitly in the docs, same "TODO(verify)" spirit
already used elsewhere in this codebase for an Emby response shape that
can't be checked locally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-14 18:05:08 +02:00

13 KiB

Watch Party for Emby

A standalone web app for watching the same title, in sync, with other users of the same Emby server. It reads your Emby catalog and streams your Emby media, but it's its own app with its own player — not an Emby plugin.

See docs/architecture.md for the full design, including why this replaced an earlier Emby-plugin attempt (still in src/WatchParty/ for reference — see below).

How it works

  1. Sign in with your Emby account (just username/password — the server address is fixed server-side, see Deploying below).
  2. A sidebar lists your Emby libraries (movies, TV, music, music videos — anything else Emby might have, like photos or a boxset collection, isn't shown); the home page shows every ongoing watch party live (host, how long it's been running, member count) plus a "recently added" row per library, Emby-style. No join codes — if a party is running, it's already on the list, updating in real time.
  3. Pick a movie/episode tile and hit "Lancer une watch party" (or drill into a Series' seasons/episodes first) — you become the host — or hit "Rejoindre" on any party already listed on the home page. Already connected to that same party from another device? You're asked to confirm before it gets disconnected — reconnecting from the same device (a reload, a second tab) never asks.
  4. The host's play/pause/seek drives everyone else's player automatically; starting from an episode queues the rest of the series so the host can just hit "Épisode suivant" instead of restarting a party each time. If the host leaves, the party hands off to another member automatically (their player unlocks live, no reload needed) — only closing once everyone's gone.
  5. Everyone present shows up as a small avatar overlay on the video (their initials, on the player itself). If you're not the host, a small toolbar lets you ask the host to pause or resume — everyone sees the request, including the host, without needing chat.
  6. Subtitles render natively in the browser for any text-based track. A Blu-ray/DVD-style image subtitle (PGS/VobSub/DVB) is different — the host can pick one to have it burned into the video by this app's own ffmpeg, shared live by everyone in the party (audio/quality selection is fixed while that's active, and it restarts the stream for everyone, so it's a host-only choice, like play/pause already is).
  7. Music works differently, on purpose: pick a track (browsing Artist → Album → Track) to start a listening party. There's no host for playback — anyone can add more tracks to the shared queue at any time, anyone can pause/resume, and skipping the current track needs a majority vote from whoever's currently listening. The player itself is a persistent bar docked at the bottom of the screen, not a page — the sidebar and the rest of the app stay fully usable while it plays, so adding more tracks is just browsing to an album and clicking a track (its button reads "Ajouter à la file" instead of "Écouter ensemble" once a party's already going). Starting or joining a video watch party always leaves the listen party first — the two don't run at once.

Everything — catalog, images, video, audio, sync — flows through this app's own backend, which proxies whatever it needs from Emby server-side. The browser never talks to Emby directly and never holds an Emby access token; it only ever has a session cookie for this app.

Status

Login, and the rendering/navigation logic for every page (sidebar, home, library, series, artist/album, both players), have been exercised end to end (login against the real Emby server with intentionally-wrong credentials; the pages against mocked API responses; the music queue's timer/vote/pause logic against the real server modules directly - no Emby involved there at all). See docs/architecture.md for exactly what that does and doesn't confirm — the real Emby response shapes this now depends on (library CollectionType, seasons/episodes, artists/ albums/tracks, PlaybackInfo's media streams) and actual HLS playback (video or audio) are still unverified.

The TypeScript rewrite itself is separately verified: both projects type-check in strict mode, the server has a real automated test suite (server/test/*.test.ts, Node's built-in test runner, see Running locally below) covering the party lifecycle, the music queue's timer/vote/pause behavior, the socket protocol, and the catalog/streaming overrides, and the compiled+bundled client was driven through every page and both players end to end (mocked network/Socket.IO) in a real browser. CI (Forgejo Actions) runs that same typecheck+test+build sequence on every push and gates the Docker image publish on it passing — see docs/architecture.md. Not verified: an actual docker build of the new multi-stage Dockerfile - this dev environment has no Docker - so treat the first real build as the thing to double-check, even though it mirrors exactly the npm run build commands already confirmed working locally for both projects. Same caveat, doubled, for the self-hosted ffmpeg burn-in pipeline (see docs/architecture.md): no Emby server or ffmpeg binary is reachable here either, so while its pure logic (stream-index conversion, session lifecycle) is unit-tested with child_process.spawn mocked, the actual command against a real PGS file - and the exact shape of Emby's direct-stream URL it depends on - has never run for real. Treat the first real attempt at a Blu-ray-style subtitle as the thing to double-check here too.

Project layout

Both server/ and client/ are TypeScript now (see docs/architecture.md for the why and the migration itself) - each its own project with its own package.json/tsconfig.json/build step; server/src and client/src are the only source of truth, server/dist/client/dist are generated (gitignored) and never hand-edited.

server/           Node.js + Express + Socket.IO backend: proxies ALL Emby
                  access (auth, catalog, images, HLS streaming) and
                  coordinates party/room state + realtime sync. The
                  browser only ever talks to this. TypeScript, compiled by
                  tsc to server/dist.
  src/index.ts                    Express app + route wiring only - thin,
                                   delegates to the services below
  src/config.ts                   env-derived config (EMBY_SERVER_URL, PORT)
  src/types.ts                    shared interfaces (Party, ItemDto, the
                                   Socket.IO event maps, ...)
  src/Logger.ts                   party/stream lifecycle logging
  src/emby/EmbyClient.ts          low-level Emby HTTP + response proxying (incl. HLS)
  src/services/AuthService.ts     server-side login + session store
  src/services/CatalogService.ts  all Emby browsing/streaming + DTO shaping
  src/services/PartyService.ts    in-memory party lifecycle (video queue, host handoff)
  src/services/MusicQueueService.ts  participative music queue - see docs/architecture.md
  src/realtime/SocketGateway.ts   Socket.IO protocol (see comments in the file)

client/           Frontend: TypeScript, esbuild-bundled to client/dist
                  (one app.js + one app.css - see client/build.mjs).
                  Talks only to server/ - never to Emby directly. Each
                  view is its own folder colocating its .ts/.html/.css.
  src/app.ts                 entry point - registers routes, boots the app
  src/router.ts              tiny hash router (unchanged in spirit - see docs)
  src/types.ts                shared interfaces (client-side copies of the
                               server's DTO shapes - see docs/architecture.md)
  src/api/embyApi.ts          calls our own /api/catalog/* + auth routes
  src/api/party.ts            our own backend's party REST + Socket.IO client
  src/tiles.ts + tiles.css    poster-tile rendering shared by home/library/artist
  src/styles/                 base.css (global reset/vars/primitives) +
                               small shared component styles (presence,
                               detail header, track list)
  src/shell/Shell.*            persistent sidebar + #mainContent the router swaps
  src/musicBar/MusicBar.*      persistent bottom-bar listen party - NOT a route,
                                mounted once by Shell.ts so it survives navigation
                                (see docs/architecture.md's "Music bar" section)
  src/views/login/Login.*      sign-in form
  src/views/home/Home.*        live parties carousel + per-library "recently added"
  src/views/library/Library.*  one library's tile grid (or artist grid, for music)
  src/views/series/Series.*    Series detail + season tabs + episode list
  src/views/artist/Artist.*    one artist's albums
  src/views/album/Album.*      one album's tracks - each track starts or queues
                                onto the active listen party (see musicBar/ above)
  src/views/player/Player.*    the /party/:id route's dispatcher (video takeover,
                                or hands off to musicBar/ + bounces back for music)

src/WatchParty/   The earlier Emby-plugin attempt (C#, unrelated to the
                  TypeScript above). Builds fine and installs, but Emby
                  has no supported extension point for a player button,
                  and even menu integration required reverse-engineering
                  undocumented internals - see docs/architecture.md for
                  the full story. Kept for reference; not the active
                  direction.

docs/architecture.md   full design doc for the standalone app

Running locally

Both projects need npm install once; the server needs the client already built at least once (it serves client/dist as static files).

cd client && npm install && npm run build && cd ..
cd server && npm install && npm run build
EMBY_SERVER_URL=https://your-emby-server npm start

Then open http://localhost:3000.

For active development, client's npm run watch rebuilds on save, and server's npm run dev runs the server straight from TypeScript via tsx (no separate compile step needed while iterating) - run both in separate terminals.

Server tests (server/test/*.test.ts, Node's built-in test runner, no extra dependencies or compile step):

cd server && npm test

server's npm run typecheck also covers server/test/, not just src/ (see tsconfig.test.json).

Deploying

Build the Docker image from the repo root (it needs both server/ and client/) - the Dockerfile compiles both TypeScript projects itself in a build stage, so nothing needs building beforehand:

docker build -f server/Dockerfile -t watchparty .
docker run -d --name watchparty -p 3000:3000 -e EMBY_SERVER_URL=https://theater.media.kyuno.fr watchparty

Then put it behind your existing reverse proxy like everything else (theater.media.kyuno.fr, git.dev.kyuno.fr, ...), e.g. watchparty.kyuno.fr → http://<container>:3000.

Roadmap

  • Design + scaffold the standalone app
  • Proxy all Emby access server-side; Emby-themed UI (sidebar, carousels)
  • TV show navigation (Series → Seasons → Episodes) + episode auto-queue
  • Player language/subtitle/quality pickers
  • Socket.IO instead of raw WebSockets; fixed two real memory leaks (dead-connection cleanup server-side, view teardown on navigation client-side) — see docs/architecture.md
  • Live "watch parties en cours" list (host, elapsed time, member count), live host handoff, no join codes
  • Presence overlay (avatars) + a request toolbar (pause/resume) in the player — see docs/architecture.md
  • Device detection: reconnecting from the same device is silent, a genuinely different device asks to confirm before disconnecting the other one — see docs/architecture.md
  • Server-side logging of the party lifecycle (create/join/leave/destroy, host handoff, play/pause, pause requests) — see docs/architecture.md
  • Music: dedicated player, Artist/Album/Track browsing, participative queue (free add, vote-to-skip, no host) — see docs/architecture.md
  • Full TypeScript rewrite (client + server), component-style client views (.ts/.html/.css each), server-side singleton services per feature — see docs/architecture.md
  • Automated server test suite (node:test) + CI job gating the Docker publish on typecheck+test+build passing — see docs/architecture.md
  • Music player as a persistent bottom bar instead of a page — browse and queue tracks while listening; starting a video party always leaves the listen party first — see docs/architecture.md
  • Self-hosted ffmpeg burn-in for PGS/image subtitles (Emby's own transcode can't render these as text) — one shared stream per party, host-driven — see docs/architecture.md
  • Try a full party end-to-end with two real accounts (still the main gap)
  • Tune transcoding target / support direct play when possible

No chat, no OIDC — deliberately out of scope (Emby-only auth was the explicit choice; see docs/architecture.md).

License

MIT — see LICENSE.