watchparty/docs/architecture.md

# Architecture

## Goals

- Watch the same title, at (roughly) the same time, as other users on the
  same Emby server.
- Don't require anyone to change how they watch — no new client, no browser
  extension, no modified apps. People keep using whatever Emby app they
  already use.
- Ship as a single Emby Server plugin.

## Non-goals (for v1)

- Frame-accurate sync. A few seconds of drift between corrections is
  acceptable — that's inherent to the "generic remote control" approach (see
  below), and is a conscious tradeoff, not an oversight.
- A shared/forced "host controls everyone" model. The chosen model (see
  "Control model" below) is: everyone keeps their own play/pause/seek
  controls; the party only shares a reference position to resync against.
- Cross-server parties, chat, reactions. Possible later, not in v1 — see
  Roadmap in the README.

## The key idea: reuse Emby's existing remote-control commands

Emby already has an authoritative-session model for "cast" / "remote control
another device" style features: `ISessionManager` tracks every active
session's playback state, and a session can be commanded via
`SendPlaystateCommand` (`Play`, `Pause`, `Unpause`, `Seek`, ...) or
`SendGeneralCommand`. This already exists in Emby Server and, to whatever
extent each client supports being remote-controlled at all (most official
clients do, since it's how casting/remote-play works), in the clients too.

So instead of building a new synchronized player, this plugin is mostly a
coordinator sitting on top of that existing mechanism:

1. Track which sessions belong to which "party".
2. Watch each member's reported position/pause state (the same data Emby's
   own "Now Playing" / dashboard already reads).
3. When a member drifts too far from the party's reference position, or its
   pause state disagrees with the reference, send the same kind of
   Seek/Pause/Unpause command Emby would send for ordinary remote control.

Sources consulted while designing this:
- [How to build a Server Plugin](https://github.com/MediaBrowser/Emby/wiki/How-to-build-a-Server-Plugin) — project setup, `BasePlugin<T>`, `IServerEntryPoint`.
- [Creating Api Endpoints](https://github.com/MediaBrowser/Emby/wiki/Creating-Api-Endpoints) — the `IService`/`[Route]`/`IReturn<T>` REST pattern.
- [Emby plugin dev docs](https://dev.emby.media/doc/plugins/dev/index.html) — confirms `netstandard2.0`, .NET Core/Mono runtimes.
- [`MediaBrowser.Server.Core` on NuGet](https://www.nuget.org/packages/mediabrowser.server.core) — package to reference; latest published at research time was `4.9.1.90`.
- Emby community thread on [remote-controlling seek](https://emby.media/community/topic/119432-emby-remote-control-another-client-to-seek-tracks/), and Jellyfin's (Emby-derived) [`SessionManager.cs`](https://github.com/jellyfin/jellyfin/blob/master/Emby.Server.Implementations/Session/SessionManager.cs) — shape of `SendPlaystateCommand(controllingSessionId, sessionId, PlaystateRequest, CancellationToken)` and `PlaystateRequest { Command, SeekPositionTicks, ControllingUserId }`.

## Control model

Design decision: **each participant keeps their own player and controls; the
party only shares a reference position to resync against.** Concretely:

- Nobody's play/pause/seek is disabled. You control your own playback
  exactly like you always do.
- One member is the *reference* (the host, by default — falls back to
  whoever's left if the host leaves). The server treats the reference
  session's reported position/pause state as ground truth for the party.
- Other members get corrected against the reference:
  - **Pause/unpause is mirrored immediately** (configurable —
    `SyncPausePlay`, on by default): if the reference pauses, everyone gets a
    `Pause` command; if it unpauses, everyone gets `Unpause`.
  - **Position drift beyond a threshold is corrected with a `Seek`**, on a
    cooldown so a shaky connection doesn't cause a stream of seek commands
    (`DriftThresholdSeconds`, `ResyncCooldownSeconds`).
  - Anyone can also hit **"Resync me now"** in the control panel for an
    immediate correction, bypassing the cooldown.

This deliberately does *not* try to stop someone from pausing or seeking
locally — it just keeps pulling things back toward the reference. That fits
"loose sync via existing remote-control commands" much better than trying to
fight every local action.

## Sync algorithm

Runs on a timer (`SyncTickSeconds`, default 5s) inside `PartySyncEntryPoint`,
per active party:

```
reference = party.Members[party.ReferenceUserId]
referencePos = reference.EstimatedPositionTicks(now)   # extrapolated if playing

for member in party.Members, excluding reference:
    if SyncPausePlay and member.IsPaused != reference.IsPaused:
        send Pause/Unpause to member's session
        continue

    if now - member.LastResyncUtc < ResyncCooldownSeconds:
        continue   # give the last correction time to take effect

    drift = abs(member.EstimatedPositionTicks(now) - referencePos)
    if drift > DriftThresholdSeconds:
        send Seek(referencePos) to member's session
```

`EstimatedPositionTicks` extrapolates from the last known position using
elapsed wall-clock time when playing, so members aren't marked "drifted" just
because their last progress event was a few seconds ago.

Position/pause state itself comes from Emby's own session events
(`ISessionManager.PlaybackStart` / `PlaybackProgress` / `PlaybackStopped`) —
the plugin doesn't poll clients itself.

## Which session is "mine"?

The trickiest part of gluing "a user" to "a specific playback session",
since one user can have several active sessions (phone + TV + browser tab).

v1's blunt heuristic (`WatchPartyService.FindCallerSession`): when you
create/join a party, the server looks at your currently active sessions,
keeps the ones actually playing something (and, when joining, playing the
*same item* as the party), and picks the most recently active one. Good
enough for "one person, one device, actively watching" — the expected case —
but it's a real limitation if you're doing something else on a second device
at the same time. Documented here rather than hidden.

## Components

```
src/WatchParty/
  Plugin.cs           BasePlugin<PluginConfiguration> + both plugin pages (see GetPages())
  Configuration/
    PluginConfiguration.cs   defaults (drift threshold, cooldown, etc.)
    configPage.html          admin settings page (Dashboard > Plugins > Watch Party)
    partyPage.html           the actual feature: create/join, member list, resync/leave -
                             EnableInUserMenu=true, so regular users see it too
  Parties/
    Party.cs            party id, join code, item id, host/reference user, members
    PartyMember.cs       per-member session id + last known position/pause + EstimatedPositionTicks
    PartyManager.cs      in-memory store (ConcurrentDictionary); create/join/leave/lookup
  Sync/
    PartySyncEntryPoint.cs   IServerEntryPoint: session event wiring + the tick loop above
  Api/
    WatchPartyService.cs     REST endpoints (create/join/leave/get/resync), called from
                             partyPage.html via the Emby web client's own ApiClient
```

Parties are **in-memory only** — a server restart clears them. That's fine
for v1: a party is meant to live for one watch session, not persist for
weeks.

The party UI is a real Emby plugin page (`Configuration/partyPage.html`,
registered in `Plugin.cs` via `IHasWebPages.GetPages()`), not a separate
web app. `PluginPageInfo` turns out to support exactly this: setting
`EnableInUserMenu = true` (`Plugin.cs`) surfaces the page to regular users, not
just admins — the same mechanism as the admin settings page
(`configPage.html`), just with different visibility flags. This was not
obvious from the official docs (which only cover the admin/dashboard case) —
confirmed instead via reflection on the real `PluginPageInfo` type (it has
`EnableInMainMenu`, `EnableInUserMenu`, `MenuSection`, `MenuIcon`,
`IsMainConfigPage`) and an Emby community thread showing another plugin using
`EnableInUserMenu = true` in practice.

Two consequences of doing it this way instead of a standalone page:
- No separate login: the page runs inside the already-authenticated Emby web
  client and just calls the global `ApiClient`, the same as the admin
  settings page already did. There's no server URL / username / password
  form, no token stored anywhere by this plugin.
- **Known limitation**: this reaches the **Web browser and Electron desktop
  client only**. Official mobile/TV apps allowlist which plugin pages they
  render client-side (independent of what the server exposes), so the page
  won't show up there without changes on the client side that aren't
  realistic to make from this plugin. Playback sync itself (the
  `SendPlaystateCommand` nudges) still works on any client regardless — it's
  only the *page to create/join a party* that's Web/Electron-only.

An earlier version of this plugin shipped a fully standalone web app
(`web/control-panel/`, its own login form, served via custom
`GET /WatchParty/app` routes) to sidestep exactly this uncertainty. It was
removed once `EnableInUserMenu` was confirmed to work — see git history if
you ever need the standalone version back (e.g. to reach a client where the
integrated page doesn't show up).

## Assumptions to verify

Update: the .NET SDK is now installed and `src/WatchParty` **builds clean
(0 errors, 0 warnings) against the real `MediaBrowser.Server.Core` /
`MediaBrowser.Common` 4.9.1.90 assemblies**, restored from NuGet. Most of the
list below turned out right on the first guess; a few didn't, and were fixed
by reading the actual assemblies (via .NET reflection) and then letting the
compiler confirm. Keeping the history here since it's a useful map of where
this SDK surprises you.

**Confirmed correct as originally written** (compiled with no changes needed):
- `BasePlugin<T>` constructor shape, `IServerEntryPoint`, `netstandard2.0` +
  `MediaBrowser.Server.Core` package — all from the official wiki.
- `IHasWebPages` (not `IHasUIPages` — that name does exist too, in
  `MediaBrowser.Model.Plugins.UI`, but `IHasWebPages` in
  `MediaBrowser.Model.Plugins` is the one that's actually wired up here) and
  `PluginPageInfo.Name`/`.EmbeddedResourcePath`.
- `ISessionManager.SendPlaystateCommand(string controllingSessionId, string
  sessionId, PlaystateRequest, CancellationToken)`, `SendGeneralCommand`, and
  `PlaystateRequest { Command, SeekPositionTicks }` / `PlaystateCommand.{Pause,
  Unpause, Seek}`.
- `ILogManager.GetLogger(name)` / `ILogger.Debug(...)` /
  `.ErrorException(...)` — still the live pattern on 4.9.x, not replaced by
  `Microsoft.Extensions.Logging`.
- `IRequiresRequest` / `IService` / `[Route]` / `IReturn<T>` for the REST
  endpoints.
- `SessionInfo.PlayState.PositionTicks` (`long?`) and `.IsPaused` (`bool`).

**Wrong on the first guess, fixed once the real assemblies were available:**
- **Namespaces**, not names: `PlaybackProgressEventArgs`/`PlaybackStopEventArgs`
  live in `MediaBrowser.Controller.Library`, not `...Session`.
  `IAuthorizationContext`/`AuthorizationInfo` live in `MediaBrowser.Controller.Net`.
- **ID types**: item/user/session ids (`SessionInfo.Id`, `SessionInfo.UserId`,
  `BaseItemDto.Id`) are plain **`string`**, not `Guid`, on this SDK version.
  `Party`/`PartyMember` were changed from `Guid` to `string` accordingly.
  (`Party.Id` itself stays a `Guid` — that's an id this plugin mints, not an
  Emby one.)
- **Reading the calling user is a two-step, not one property**:
  `IAuthorizationContext.GetAuthorizationInfo(Request).UserId` is a `long`
  (an internal row id — not the id used anywhere else). The one that matches
  `SessionInfo.UserId` etc. is `.User.Id` — except that one comes back as a
  `Guid`, so `WatchPartyService.CallerUserId()` reformats it with
  `.ToString("N")` to match the string convention used elsewhere. See the
  comment there.

**Still only compile-checked, not verified against a running server**
(nothing here fails to build — these are runtime behaviors the compiler
can't check):
- Whether `.ToString("N")` (no dashes) is really the right string format to
  match `SessionInfo.UserId` — plausible, matches Emby/Jellyfin convention
  elsewhere, not observed directly.
- HTTP error mapping — endpoints throw plain
  `ArgumentException`/`InvalidOperationException`; Emby likely has a more
  specific exception type that maps to a proper 404/400 instead of a
  generic 500.
- Static file content types — `GET /WatchParty/app` etc. return a bare
  string; `IHasResultFactory` looks like the right extension point for a
  real `Content-Type`/caching headers, not wired up yet.
- `controllingSessionId` passed as `null` to `SendPlaystateCommand` — compiles,
  unknown whether Emby's runtime wants a real session/user there instead.
- `X-Emby-Authorization` header fields and `/Users/AuthenticateByName`'s `Pw`
  field name in `web/control-panel/app.js` — it's JS, nothing compiles it;
  long-standing Emby/Jellyfin convention, not exercised against a real server.
- The dashboard settings page (`configPage.html`) actually rendering/saving
  correctly — embedding compiles, but the page's own JS (`ApiClient`,
  `Dashboard` globals) only exists inside a running Emby dashboard.
- **`partyPage.html`**: whether `ApiClient.ajax({...})` really behaves like
  the jQuery-style options object it's called with here (`type`, `url`,
  `dataType`) — that's the long-standing Emby/Jellyfin web client convention,
  not confirmed against this specific build. Whether `EnableInMainMenu`/
  `MenuIcon = "group"` place the page where you'd expect (main nav vs. a
  user submenu) is also cosmetic-only until seen live — easy to adjust once
  visible.

The remaining way to close these out: install the plugin DLL on a real Emby
Server 4.9.x and exercise it with two accounts.

## Known limitations

- Sync is best-effort and loose by design (see "Non-goals"). Don't expect
  frame-accuracy.
- Depends on the member's *own* client actually honoring remote Seek/Pause
  commands smoothly — most official Emby apps do (it's how casting/remote
  control works today), but behavior can vary by platform, and a Seek can
  cause a visible re-buffer.
- One item per party; no shared "up next" queue.
- No chat/reactions in v1 (kept out on purpose — see the design conversation
  that led to this repo).
- A user with two simultaneously active sessions on the same item is
  resolved by "most recently active", which can guess wrong.