From 8eb047047493f1d341abb82133779dc2ed5518e3 Mon Sep 17 00:00:00 2001 From: Nicolas Date: Thu, 10 Sep 2026 19:29:44 +0200 Subject: [PATCH] =?UTF-8?q?Notifications=20par=20webhook=20(upload=20final?= =?UTF-8?q?is=C3=A9=20/=20=C3=A9chec)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Nouvelle clé `webhooks` : liste de points de terminaison POST notifiés à chaque upload finalisé (`upload`, direct comme fragmenté) et à chaque requête `/upload*` terminée en 4xx/5xx (`error`). Chaque entrée : `url` (http/https), `events` (défaut : les deux), `headers` (auth du récepteur), `secret` (signature HMAC-SHA256 dans `X-MediaDrop-Signature`), `timeoutMs` (défaut 5 s, max 30 s). Corps JSON structuré + champs `text`/`content` portant un résumé lisible, pour fonctionner tel quel avec Slack / Discord / ntfy ou un récepteur custom. Livraison en parallèle, sans bloquer la réponse d'upload ; une nouvelle tentative sur erreur réseau / 429 / 5xx ; échecs journalisés en `warn` ; l'arrêt du serveur attend les envois en cours. Sans webhook configuré, le notifieur est une non-opération. Co-Authored-By: Claude Sonnet 5 --- README.md | 64 +++++++++ config.example.json | 1 + config.schema.json | 39 +++++ src/app.ts | 6 +- src/config.ts | 106 ++++++++++++++ src/error-handler.ts | 106 ++++++++------ src/fastify.d.ts | 3 + src/lib/notifications.ts | 211 +++++++++++++++++++++++++++ src/lib/upload-sessions.ts | 2 + src/routes/upload-chunked.ts | 16 ++- src/routes/upload.ts | 12 +- tests/config.test.ts | 63 +++++++++ tests/helpers.ts | 1 + tests/notifications.test.ts | 217 ++++++++++++++++++++++++++++ tests/webhooks-integration.test.ts | 220 +++++++++++++++++++++++++++++ 15 files changed, 1023 insertions(+), 44 deletions(-) create mode 100644 src/lib/notifications.ts create mode 100644 tests/notifications.test.ts create mode 100644 tests/webhooks-integration.test.ts diff --git a/README.md b/README.md index cde7823..68d7d5a 100644 --- a/README.md +++ b/README.md @@ -198,6 +198,7 @@ Un seul fichier JSON (voir [`config.example.json`](config.example.json), schéma | `logLevel` | `info` | `fatal` \| `error` \| `warn` \| `info` \| `debug` \| `trace` \| `silent` | | `chunkPoolSize` | `30` | Morceaux en vol simultanés (upload fragmenté), répartis entre fichiers actifs ; `1`–`256`. Utile en HTTP/2 | | `allowedMimeTypes` | _(liste média)_ | `"all"`, ou tableau de types exacts et/ou jokers de préfixe (`"video/*"`, `"*"`) ; absent => liste par défaut | +| `webhooks` | `[]` | Liste de webhooks POST notifiés à chaque upload finalisé et/ou chaque échec (voir §Notifications) | | `session.key` | — | **Obligatoire.** 32 octets base64 (`openssl rand -base64 32`) | | `session.maxAgeHours` | `12` | Durée de vie absolue de la session | | `oidc.issuer` | — | **Obligatoire.** Issuer OIDC | @@ -207,6 +208,68 @@ Un seul fichier JSON (voir [`config.example.json`](config.example.json), schéma Seule variable d'environnement : `MEDIADROP_CONFIG` (chemin du fichier, défaut `./config.json`). +## Notifications (webhooks) + +`webhooks` est une liste ; chaque entrée reçoit un `POST` JSON **à chaque upload +finalisé** (`upload` — dépôt direct comme fragmenté) et **à chaque échec d'upload** +(`error` — toute requête `/upload*` terminée en 4xx/5xx). + +```json +"webhooks": [ + { + "url": "https://exemple.fr/hooks/mediadrop", + "events": ["upload", "error"], + "headers": { "Authorization": "Bearer …" }, + "secret": "un-secret-partagé", + "timeoutMs": 5000 + } +] +``` + +| Champ | Défaut | Rôle | +| ----------- | -------------------- | ----------------------------------------------------------- | +| `url` | — | **Obligatoire.** Endpoint HTTP(S) appelé en POST | +| `events` | `["upload","error"]` | Sous-ensemble d'événements souscrits | +| `headers` | `{}` | En-têtes ajoutés à la requête (auth du récepteur…) | +| `secret` | — | Active la signature `X-MediaDrop-Signature` (voir plus bas) | +| `timeoutMs` | `5000` | Délai d'attente par tentative (`1`–`30000`) | + +**Requête émise** — `content-type: application/json`, `User-Agent: mediaDrop-webhook`, +`X-MediaDrop-Event: upload|error`, plus les `headers` configurés, plus +`X-MediaDrop-Signature` si `secret`. Corps `upload` : + +```json +{ + "event": "upload", + "timestamp": "2026-09-10T18:30:00.000Z", + "target": "videos", + "storedName": "vacances/plage/DSC_001.mp4", + "originalName": "DSC_001.mp4", + "size": 20480, + "mimeType": "video/mp4", + "user": "auth0|abc", + "via": "chunked", + "text": "✅ Upload terminé : vacances/plage/DSC_001.mp4 (20 Ko) — cible « videos »", + "content": "✅ Upload terminé : vacances/plage/DSC_001.mp4 (20 Ko) — cible « videos »" +} +``` + +Corps `error` : `{ event, timestamp, message, statusCode, method, path, target?, user?, text, content }`. +`text` et `content` portent le même résumé lisible (`text` pour Slack, `content` pour +Discord) — l'endpoint peut aussi être un webhook entrant Slack/Discord/ntfy tel quel. + +**Livraison** : en parallèle, sans bloquer la réponse d'upload ; une nouvelle tentative +sur erreur réseau / `429` / `5xx` ; les échecs définitifs sont seulement journalisés +(`warn`). L'arrêt du serveur attend les envois en cours. + +**Vérifier la signature** (récepteur) — HMAC-SHA256 du corps **brut** avec le `secret` : + +```js +const expected = 'sha256=' + crypto.createHmac('sha256', secret).update(rawBody).digest('hex'); +const ok = + sig.length === expected.length && crypto.timingSafeEqual(Buffer.from(sig), Buffer.from(expected)); +``` + ## Scripts | Commande | Effet | @@ -260,6 +323,7 @@ src/ lib/ storage.ts écriture disque, nom sûr, plafond, morceaux (writeChunkAt) upload-sessions.ts registre en mémoire des uploads fragmentés (+ purge TTL) + notifications.ts envoi des webhooks (upload finalisé / échec), signature HMAC targets.ts résolution cible / chemin relatif (partagé par les deux routes) media-types.ts liste blanche des types + extensions + jokers paths.ts assainissement des chemins relatifs (dépôt de dossier) diff --git a/config.example.json b/config.example.json index 3be3ea1..8e31aa7 100644 --- a/config.example.json +++ b/config.example.json @@ -11,6 +11,7 @@ "logLevel": "info", "chunkPoolSize": 30, "allowedMimeTypes": ["image/jpeg", "image/png", "image/webp", "video/mp4", "audio/mpeg"], + "webhooks": [], "session": { "key": "REMPLACER_PAR_32_OCTETS_BASE64_openssl_rand_base64_32", "maxAgeHours": 12 diff --git a/config.schema.json b/config.schema.json index 2427316..d89f734 100644 --- a/config.schema.json +++ b/config.schema.json @@ -50,6 +50,45 @@ "default": 30, "description": "Nombre max de requêtes de morceaux en vol, réparti par le navigateur entre les fichiers fragmentés actifs (1 fichier => tout le pool, N fichiers => une part chacun). Efficace surtout en HTTP/2." }, + "webhooks": { + "type": "array", + "description": "Points de terminaison POST notifiés à chaque upload finalisé et/ou chaque échec d'upload.", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["url"], + "properties": { + "url": { + "type": "string", + "format": "uri", + "description": "URL HTTP(S) appelée en POST (corps JSON)." + }, + "events": { + "type": "array", + "items": { "enum": ["upload", "error"] }, + "uniqueItems": true, + "default": ["upload", "error"], + "description": "Événements souscrits. Défaut : les deux." + }, + "headers": { + "type": "object", + "additionalProperties": { "type": "string" }, + "description": "En-têtes ajoutés à chaque requête (auth du récepteur, etc.)." + }, + "secret": { + "type": "string", + "description": "Si présent : en-tête X-MediaDrop-Signature = sha256=HMAC-SHA256(corps brut)." + }, + "timeoutMs": { + "type": "integer", + "minimum": 1, + "maximum": 30000, + "default": 5000, + "description": "Délai d'attente par tentative." + } + } + } + }, "allowedMimeTypes": { "description": "\"all\", ou liste de types exacts et/ou jokers de préfixe (\"video/*\", \"*/*\", \"*\"). Absent => liste média par défaut.", "oneOf": [ diff --git a/src/app.ts b/src/app.ts index 1caad5a..5765a06 100644 --- a/src/app.ts +++ b/src/app.ts @@ -7,6 +7,7 @@ import secureSession from '@fastify/secure-session'; import type { AppConfig } from './config.js'; import { registerErrorHandler } from './error-handler.js'; +import { createNotifier } from './lib/notifications.js'; import { ensureUploadDir, removeFile } from './lib/storage.js'; import { createUploadSessionStore } from './lib/upload-sessions.js'; import { createOidcClient, type OidcClient } from './oidc.js'; @@ -51,12 +52,15 @@ export async function buildApp( const uploadSessions = createUploadSessionStore((session) => { void removeFile(session.partPath); }); + const notifier = createNotifier(config.webhooks, app.log); app.decorate('appConfig', config); app.decorate('oidcClient', oidcClient); app.decorate('uploadSessions', uploadSessions); - app.addHook('onClose', () => { + app.decorate('notifier', notifier); + app.addHook('onClose', async () => { uploadSessions.stop(); + await notifier.close(); }); // Corps brut (flux) pour les morceaux d'upload fragmenté ; écrit en streaming. diff --git a/src/config.ts b/src/config.ts index fe36a1e..37f991b 100644 --- a/src/config.ts +++ b/src/config.ts @@ -24,6 +24,24 @@ export interface OidcConfig { scopes: string[]; } +/** Événements qu'un webhook peut recevoir. */ +export const WEBHOOK_EVENTS = ['upload', 'error'] as const; +export type WebhookEvent = (typeof WEBHOOK_EVENTS)[number]; + +/** Un point de terminaison notifié à chaque upload finalisé et/ou chaque échec. */ +export interface WebhookConfig { + /** URL HTTP(S) appelée en POST (corps JSON). */ + url: string; + /** Événements souscrits (défaut : les deux). */ + events: readonly WebhookEvent[]; + /** En-têtes ajoutés à chaque requête (auth du récepteur, etc.). */ + headers: Readonly>; + /** Si présent : en-tête `X-MediaDrop-Signature: sha256=`. */ + secret?: string; + /** Délai d'attente par tentative, en millisecondes. */ + timeoutMs: number; +} + /** Dossier nommé dans lequel déposer des fichiers. */ export interface DropTarget { /** Identifiant stable, transmis par le client (`?target=...`). */ @@ -58,6 +76,8 @@ export interface AppConfig { * N fichiers => une part chacun). Transmis au navigateur via `GET /config`. */ chunkPoolSize: number; + /** Webhooks notifiés à chaque upload finalisé et à chaque échec d'upload. */ + webhooks: readonly WebhookConfig[]; session: SessionConfig; oidc: OidcConfig; } @@ -78,6 +98,9 @@ const DEFAULTS = { /** Garde-fou : au-delà, la mémoire tampon et le nombre de sockets deviennent déraisonnables. */ const MAX_CHUNK_POOL_SIZE = 256; +const DEFAULT_WEBHOOK_TIMEOUT_MS = 5000; +const MAX_WEBHOOK_TIMEOUT_MS = 30_000; + const TARGET_NAME_PATTERN = /^[A-Za-z0-9._-]+$/; function asRecord(value: unknown, label: string): Record { @@ -243,6 +266,88 @@ function parseOidc(value: unknown): OidcConfig { }; } +function parseWebhookUrl(value: unknown, name: string): string { + const raw = requiredString(value, name); + let parsed: URL; + try { + parsed = new URL(raw); + } catch { + throw new ConfigError(`${name} doit être une URL absolue (reçu : "${raw}")`); + } + if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') { + throw new ConfigError(`${name} doit utiliser le schéma http ou https`); + } + return raw; +} + +function parseWebhookEvents(value: unknown, name: string): readonly WebhookEvent[] { + if (value === undefined || value === null) { + return [...WEBHOOK_EVENTS]; + } + if (!Array.isArray(value)) { + throw new ConfigError(`${name} doit être un tableau`); + } + const seen = new Set(); + for (const entry of value) { + if (typeof entry !== 'string' || !(WEBHOOK_EVENTS as readonly string[]).includes(entry)) { + throw new ConfigError( + `${name} : événement inconnu « ${String(entry)} » (attendu : upload, error)`, + ); + } + seen.add(entry as WebhookEvent); + } + if (seen.size === 0) { + throw new ConfigError(`${name} ne peut pas être vide`); + } + return [...seen]; +} + +function parseWebhookHeaders(value: unknown, name: string): Readonly> { + if (value === undefined || value === null) { + return {}; + } + const raw = asRecord(value, name); + const headers: Record = {}; + for (const [key, headerValue] of Object.entries(raw)) { + if (typeof headerValue !== 'string') { + throw new ConfigError(`${name}.${key} doit être une chaîne`); + } + headers[key] = headerValue; + } + return headers; +} + +function parseWebhookTimeout(value: unknown, name: string): number { + const parsed = optionalPositiveInt(value, DEFAULT_WEBHOOK_TIMEOUT_MS, name); + if (parsed > MAX_WEBHOOK_TIMEOUT_MS) { + throw new ConfigError( + `${name} ne doit pas dépasser ${String(MAX_WEBHOOK_TIMEOUT_MS)} ms (reçu : ${String(parsed)})`, + ); + } + return parsed; +} + +function parseWebhooks(value: unknown): readonly WebhookConfig[] { + if (value === undefined || value === null) { + return []; + } + if (!Array.isArray(value)) { + throw new ConfigError('webhooks doit être un tableau'); + } + return value.map((entry, index): WebhookConfig => { + const label = `webhooks[${String(index)}]`; + const raw = asRecord(entry, label); + const secret = optionalString(raw.secret, `${label}.secret`); + return { + url: parseWebhookUrl(raw.url, `${label}.url`), + events: parseWebhookEvents(raw.events, `${label}.events`), + headers: parseWebhookHeaders(raw.headers, `${label}.headers`), + ...(secret !== undefined ? { secret } : {}), + timeoutMs: parseWebhookTimeout(raw.timeoutMs, `${label}.timeoutMs`), + }; + }); +} + function parseTargets(value: unknown): DropTargets { if (!Array.isArray(value) || value.length === 0) { throw new ConfigError('targets doit être un tableau non vide'); @@ -297,6 +402,7 @@ export function parseConfig(raw: unknown): AppConfig { allowedMimeTypes: parseMimePolicy(root.allowedMimeTypes), logLevel: parseLogLevel(root.logLevel), chunkPoolSize: parseChunkPoolSize(root.chunkPoolSize), + webhooks: parseWebhooks(root.webhooks), session: parseSession(root.session), oidc: parseOidc(root.oidc), }; diff --git a/src/error-handler.ts b/src/error-handler.ts index cac5c55..bedd3b4 100644 --- a/src/error-handler.ts +++ b/src/error-handler.ts @@ -1,6 +1,7 @@ -import type { FastifyError, FastifyInstance, FastifyReply, FastifyRequest } from 'fastify'; +import type { FastifyError, FastifyInstance, FastifyRequest } from 'fastify'; import { AppError } from './errors.js'; +import { getSessionUser } from './session.js'; interface ErrorBody { error: string; @@ -9,6 +10,54 @@ interface ErrorBody { const TOO_LARGE_MESSAGE = 'Le fichier dépasse la taille maximale autorisée'; +/** Détermine le statut HTTP et le corps `{ error, message }` d'une erreur. */ +function resolveError( + error: FastifyError, + request: FastifyRequest, +): { status: number; body: ErrorBody } { + if (error instanceof AppError) { + return { status: error.statusCode, body: { error: error.name, message: error.message } }; + } + + switch (error.code) { + case 'FST_REQ_FILE_TOO_LARGE': + return { status: 413, body: { error: 'PayloadTooLargeError', message: TOO_LARGE_MESSAGE } }; + case 'FST_FILES_LIMIT': + return { + status: 400, + body: { error: 'ValidationError', message: 'Un seul fichier par requête' }, + }; + case 'FST_INVALID_MULTIPART_CONTENT_TYPE': + return { + status: 400, + body: { error: 'ValidationError', message: 'Requête multipart/form-data attendue' }, + }; + default: + break; + } + + if (error.validation) { + return { status: 400, body: { error: 'ValidationError', message: error.message } }; + } + + request.log.error({ err: error }, 'Erreur non gérée'); + return { status: 500, body: { error: 'InternalServerError', message: 'Erreur interne' } }; +} + +/** Chemin de la requête sans la chaîne de query. */ +function pathOf(request: FastifyRequest): string { + return request.url.split('?', 1)[0] ?? request.url; +} + +function requestedTarget(request: FastifyRequest): string | undefined { + const query = request.query as { target?: unknown }; + return typeof query.target === 'string' ? query.target : undefined; +} + +function isUploadPath(path: string): boolean { + return path === '/upload' || path.startsWith('/upload/'); +} + /** * Installe le gestionnaire d'erreurs global. * @@ -16,48 +65,25 @@ const TOO_LARGE_MESSAGE = 'Le fichier dépasse la taille maximale autorisée'; * - Erreurs `@fastify/multipart` connues -> 4xx explicites. * - Erreurs de validation de schéma Fastify -> 400. * - Tout le reste -> 500 générique (détails seulement dans les logs). + * + * Toute réponse 4xx/5xx sur une route `/upload*` déclenche une notification + * webhook `error`. */ export function registerErrorHandler(app: FastifyInstance): void { - app.setErrorHandler((error: FastifyError, request: FastifyRequest, reply: FastifyReply): void => { - if (error instanceof AppError) { - reply - .status(error.statusCode) - .send({ error: error.name, message: error.message } satisfies ErrorBody); - return; - } + app.setErrorHandler((error: FastifyError, request, reply): void => { + const { status, body } = resolveError(error, request); + reply.status(status).send(body); - switch (error.code) { - case 'FST_REQ_FILE_TOO_LARGE': - reply - .status(413) - .send({ error: 'PayloadTooLargeError', message: TOO_LARGE_MESSAGE } satisfies ErrorBody); - return; - case 'FST_FILES_LIMIT': - reply.status(400).send({ - error: 'ValidationError', - message: 'Un seul fichier par requête', - } satisfies ErrorBody); - return; - case 'FST_INVALID_MULTIPART_CONTENT_TYPE': - reply.status(400).send({ - error: 'ValidationError', - message: 'Requête multipart/form-data attendue', - } satisfies ErrorBody); - return; - default: - break; + const path = pathOf(request); + if (status >= 400 && isUploadPath(path)) { + app.notifier.uploadFailed({ + message: body.message, + statusCode: status, + method: request.method, + path, + target: requestedTarget(request), + user: getSessionUser(request)?.sub, + }); } - - if (error.validation) { - reply - .status(400) - .send({ error: 'ValidationError', message: error.message } satisfies ErrorBody); - return; - } - - request.log.error({ err: error }, 'Erreur non gérée'); - reply - .status(500) - .send({ error: 'InternalServerError', message: 'Erreur interne' } satisfies ErrorBody); }); } diff --git a/src/fastify.d.ts b/src/fastify.d.ts index 2177f5a..a0b9df7 100644 --- a/src/fastify.d.ts +++ b/src/fastify.d.ts @@ -1,6 +1,7 @@ import type { AppConfig } from './config.js'; import type { OidcClient } from './oidc.js'; import type { LoginTransaction, SessionUser } from './session.js'; +import type { Notifier } from './lib/notifications.js'; import type { UploadSessionStore } from './lib/upload-sessions.js'; declare module 'fastify' { @@ -11,6 +12,8 @@ declare module 'fastify' { oidcClient: OidcClient; /** Registre des uploads fragmentés en cours. */ uploadSessions: UploadSessionStore; + /** Envoi des notifications webhook (upload finalisé / échec). */ + notifier: Notifier; } } diff --git a/src/lib/notifications.ts b/src/lib/notifications.ts new file mode 100644 index 0000000..5473b03 --- /dev/null +++ b/src/lib/notifications.ts @@ -0,0 +1,211 @@ +import { createHmac } from 'node:crypto'; + +import type { WebhookConfig, WebhookEvent } from '../config.js'; + +/** Sous-ensemble du logger Fastify utilisé par le notifieur. */ +export interface NotifierLogger { + debug(details: Record, message: string): void; + warn(details: Record, message: string): void; +} + +/** Données d'un upload finalisé, avant enrichissement par le notifieur. */ +export interface UploadCompletedInput { + target: string; + /** Nom simple, ou chemin relatif `dossier/sous/fichier.ext` (mode dossier). */ + storedName: string; + originalName?: string; + size: number; + mimeType: string; + /** `sub` de l'utilisateur, si la session est lisible. */ + user?: string; + /** Chemin ayant produit l'upload. */ + via: 'direct' | 'chunked'; +} + +/** Données d'un échec d'upload (route `/upload*` terminée en 4xx/5xx). */ +export interface UploadFailedInput { + message: string; + statusCode: number; + method: string; + path: string; + target?: string; + user?: string; +} + +export interface Notifier { + /** Notifie les webhooks abonnés à `upload`. Ne lève jamais ; n'attend pas la livraison. */ + uploadCompleted(input: UploadCompletedInput): void; + /** Notifie les webhooks abonnés à `error`. Ne lève jamais ; n'attend pas la livraison. */ + uploadFailed(input: UploadFailedInput): void; + /** Attend la fin des livraisons en cours (arrêt propre). */ + close(): Promise; +} + +const MAX_ATTEMPTS = 2; +const RETRY_DELAY_MS = 500; +const USER_AGENT = 'mediaDrop-webhook'; +const SIZE_UNITS = ['o', 'Ko', 'Mo', 'Go', 'To'] as const; + +function formatBytes(bytes: number): string { + let value = bytes; + let unit = 0; + while (value >= 1024 && unit < SIZE_UNITS.length - 1) { + value /= 1024; + unit += 1; + } + const shown = value < 10 && unit > 0 ? value.toFixed(1) : String(Math.round(value)); + return `${shown} ${SIZE_UNITS[unit] ?? 'o'}`; +} + +function delay(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +function errorText(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +function uploadSummary(input: UploadCompletedInput): string { + return `✅ Upload terminé : ${input.storedName} (${formatBytes(input.size)}) — cible « ${input.target} »`; +} + +function errorSummary(input: UploadFailedInput): string { + return `⚠️ Échec d'upload (${String(input.statusCode)}) : ${input.message} [${input.method} ${input.path}]`; +} + +function uploadBody(input: UploadCompletedInput): Record { + const summary = uploadSummary(input); + return { + event: 'upload' satisfies WebhookEvent, + timestamp: new Date().toISOString(), + target: input.target, + storedName: input.storedName, + ...(input.originalName !== undefined ? { originalName: input.originalName } : {}), + size: input.size, + mimeType: input.mimeType, + ...(input.user !== undefined ? { user: input.user } : {}), + via: input.via, + // `text` (Slack) et `content` (Discord) : même résumé lisible. + text: summary, + content: summary, + }; +} + +function errorBody(input: UploadFailedInput): Record { + const summary = errorSummary(input); + return { + event: 'error' satisfies WebhookEvent, + timestamp: new Date().toISOString(), + message: input.message, + statusCode: input.statusCode, + method: input.method, + path: input.path, + ...(input.target !== undefined ? { target: input.target } : {}), + ...(input.user !== undefined ? { user: input.user } : {}), + text: summary, + content: summary, + }; +} + +function buildHeaders( + webhook: WebhookConfig, + event: WebhookEvent, + body: string, +): Record { + const headers: Record = { + ...webhook.headers, + 'content-type': 'application/json', + 'user-agent': USER_AGENT, + 'x-mediadrop-event': event, + }; + if (webhook.secret !== undefined) { + const digest = createHmac('sha256', webhook.secret).update(body).digest('hex'); + headers['x-mediadrop-signature'] = `sha256=${digest}`; + } + return headers; +} + +async function deliver( + webhook: WebhookConfig, + event: WebhookEvent, + body: string, + log: NotifierLogger, +): Promise { + const headers = buildHeaders(webhook, event, body); + + for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt += 1) { + try { + const response = await fetch(webhook.url, { + method: 'POST', + headers, + body, + signal: AbortSignal.timeout(webhook.timeoutMs), + }); + // Vider le corps pour libérer la connexion (keep-alive). + await response.text().catch(() => ''); + + if (response.ok) { + log.debug({ url: webhook.url, status: response.status, event }, 'Webhook livré'); + return; + } + const retriable = response.status === 429 || response.status >= 500; + if (!retriable) { + log.warn({ url: webhook.url, status: response.status, event }, 'Webhook rejeté (abandon)'); + return; + } + log.warn({ url: webhook.url, status: response.status, event, attempt }, 'Webhook en échec'); + } catch (error) { + log.warn({ url: webhook.url, err: errorText(error), event, attempt }, 'Webhook injoignable'); + } + + if (attempt < MAX_ATTEMPTS) { + await delay(RETRY_DELAY_MS); + } + } +} + +/** + * Crée un notifieur qui POSTe un corps JSON à chaque webhook abonné, à chaque + * upload finalisé (`upload`) et à chaque échec d'upload (`error`). + * + * Livraison en parallèle, sans bloquer la réponse HTTP : chaque webhook a son + * propre délai d'attente, une nouvelle tentative sur erreur réseau / 429 / 5xx, + * et les échecs sont seulement journalisés. `close()` attend les envois en cours. + * Sans webhook configuré, toutes les méthodes sont des non-opérations. + */ +export function createNotifier(webhooks: readonly WebhookConfig[], log: NotifierLogger): Notifier { + if (webhooks.length === 0) { + return { + uploadCompleted(): void {}, + uploadFailed(): void {}, + close: (): Promise => Promise.resolve(), + }; + } + + const pending = new Set>(); + + const dispatch = (event: WebhookEvent, body: Record): void => { + const json = JSON.stringify(body); + for (const webhook of webhooks) { + if (!webhook.events.includes(event)) { + continue; + } + const task = deliver(webhook, event, json, log).finally(() => { + pending.delete(task); + }); + pending.add(task); + } + }; + + return { + uploadCompleted(input): void { + dispatch('upload', uploadBody(input)); + }, + uploadFailed(input): void { + dispatch('error', errorBody(input)); + }, + close: async (): Promise => { + await Promise.allSettled([...pending]); + }, + }; +} diff --git a/src/lib/upload-sessions.ts b/src/lib/upload-sessions.ts index cf25ead..fc0b0c5 100644 --- a/src/lib/upload-sessions.ts +++ b/src/lib/upload-sessions.ts @@ -7,6 +7,8 @@ import { TooManyUploadsError, UploadSessionNotFoundError } from '../errors.js'; export interface UploadSession { id: string; target: DropTarget; + /** Nom de fichier annoncé par le client (pour les notifications). */ + originalName: string; /** Nom simple, ou chemin relatif `dossier/sous/fichier.ext` (mode dossier). */ storedName: string; mimeType: string; diff --git a/src/routes/upload-chunked.ts b/src/routes/upload-chunked.ts index 1906bba..5eeeb19 100644 --- a/src/routes/upload-chunked.ts +++ b/src/routes/upload-chunked.ts @@ -13,7 +13,7 @@ import { writeChunkAt, } from '../lib/storage.js'; import { resolvePlacement, resolveTarget } from '../lib/targets.js'; -import { requireSession } from '../session.js'; +import { getSessionUser, requireSession } from '../session.js'; const MIN_CHUNK = 1024 * 1024; const MAX_CHUNK = 32 * 1024 * 1024; @@ -95,9 +95,10 @@ export const chunkedUploadRoutes: FastifyPluginCallback = (app, _options, done) const chunkSize = clampChunkSize(body.chunkSize); const chunkCount = Math.ceil(totalSize / chunkSize); + const originalName = nonEmptyString(body.name, 'fichier'); const { storedName, destination } = await resolveDestination({ targetDir: target.dir, - originalName: nonEmptyString(body.name, 'fichier'), + originalName, mimeType, ...(placement !== undefined ? { placement } : {}), }); @@ -106,6 +107,7 @@ export const chunkedUploadRoutes: FastifyPluginCallback = (app, _options, done) const session = app.uploadSessions.create({ target, + originalName, storedName, mimeType, partPath, @@ -162,6 +164,16 @@ export const chunkedUploadRoutes: FastifyPluginCallback = (app, _options, done) await finalizeFile(session.partPath, session.finalPath); app.uploadSessions.remove(id); + app.notifier.uploadCompleted({ + target: session.target.name, + storedName: session.storedName, + originalName: session.originalName, + size: session.totalSize, + mimeType: session.mimeType, + user: getSessionUser(request)?.sub, + via: 'chunked', + }); + reply.status(201); return { target: session.target.name, diff --git a/src/routes/upload.ts b/src/routes/upload.ts index f1fb474..5c59842 100644 --- a/src/routes/upload.ts +++ b/src/routes/upload.ts @@ -4,7 +4,7 @@ import type { DropTarget } from '../config.js'; import { ValidationError } from '../errors.js'; import { isAllowedMedia } from '../lib/media-types.js'; import { resolvePlacement, resolveTarget } from '../lib/targets.js'; -import { requireSession } from '../session.js'; +import { getSessionUser, requireSession } from '../session.js'; import { saveFile, type StoredFile } from '../lib/storage.js'; const FIELD_NAME = 'file'; @@ -93,6 +93,16 @@ export const uploadRoutes: FastifyPluginCallback = (app, _options, done) => { ...(placement !== undefined ? { placement } : {}), }); + app.notifier.uploadCompleted({ + target: target.name, + storedName: stored.storedName, + originalName: stored.originalName, + size: stored.size, + mimeType: stored.mimeType, + user: getSessionUser(request)?.sub, + via: 'direct', + }); + reply.status(201); return toResponse(target, stored); }, diff --git a/tests/config.test.ts b/tests/config.test.ts index 72dc6d4..e218152 100644 --- a/tests/config.test.ts +++ b/tests/config.test.ts @@ -130,6 +130,69 @@ describe('parseConfig', () => { expect(() => parseConfig({ ...BASE, chunkPoolSize: 257 })).toThrow(ConfigError); }); + it('webhooks : absent => liste vide, valeurs par défaut appliquées', () => { + expect(parseConfig(structuredClone(BASE)).webhooks).toEqual([]); + expect( + parseConfig({ ...BASE, webhooks: [{ url: 'https://hook.example/mediadrop' }] }).webhooks, + ).toEqual([ + { + url: 'https://hook.example/mediadrop', + events: ['upload', 'error'], + headers: {}, + timeoutMs: 5000, + }, + ]); + }); + + it('webhooks : lit events, headers, secret et timeoutMs', () => { + expect( + parseConfig({ + ...BASE, + webhooks: [ + { + url: 'http://192.168.1.5:9000/notify', + events: ['error', 'error'], + headers: { 'X-Token': 'abc' }, + secret: 's3cr3t', + timeoutMs: 1500, + }, + ], + }).webhooks, + ).toEqual([ + { + url: 'http://192.168.1.5:9000/notify', + events: ['error'], + headers: { 'X-Token': 'abc' }, + secret: 's3cr3t', + timeoutMs: 1500, + }, + ]); + }); + + it('webhooks : rejette les entrées invalides', () => { + expect(() => parseConfig({ ...BASE, webhooks: {} })).toThrow(ConfigError); + expect(() => parseConfig({ ...BASE, webhooks: [{}] })).toThrow(ConfigError); + expect(() => parseConfig({ ...BASE, webhooks: [{ url: 'pas-une-url' }] })).toThrow(ConfigError); + expect(() => parseConfig({ ...BASE, webhooks: [{ url: 'ftp://x.fr/hook' }] })).toThrow( + ConfigError, + ); + expect(() => + parseConfig({ ...BASE, webhooks: [{ url: 'https://x.fr/h', events: ['boom'] }] }), + ).toThrow(ConfigError); + expect(() => + parseConfig({ ...BASE, webhooks: [{ url: 'https://x.fr/h', events: [] }] }), + ).toThrow(ConfigError); + expect(() => + parseConfig({ ...BASE, webhooks: [{ url: 'https://x.fr/h', headers: { a: 1 } }] }), + ).toThrow(ConfigError); + expect(() => + parseConfig({ ...BASE, webhooks: [{ url: 'https://x.fr/h', timeoutMs: -1 }] }), + ).toThrow(ConfigError); + expect(() => + parseConfig({ ...BASE, webhooks: [{ url: 'https://x.fr/h', timeoutMs: 60000 }] }), + ).toThrow(ConfigError); + }); + it('interprète allowedMimeTypes', () => { expect( parseConfig({ ...BASE, allowedMimeTypes: ['image/png', 'IMAGE/WEBP'] }).allowedMimeTypes, diff --git a/tests/helpers.ts b/tests/helpers.ts index 0e45d33..997ef73 100644 --- a/tests/helpers.ts +++ b/tests/helpers.ts @@ -46,6 +46,7 @@ export function makeConfig(overrides: Partial = {}): AppConfig { allowedMimeTypes: ['image/png', 'image/jpeg'], logLevel: 'silent', chunkPoolSize: 30, + webhooks: [], session: { key: Buffer.alloc(32, 7), maxAgeSeconds: 3600 }, oidc: { issuer: 'https://idp.test', diff --git a/tests/notifications.test.ts b/tests/notifications.test.ts new file mode 100644 index 0000000..021c08b --- /dev/null +++ b/tests/notifications.test.ts @@ -0,0 +1,217 @@ +import { createHmac } from 'node:crypto'; + +import Fastify from 'fastify'; +import { afterEach, describe, expect, it } from 'vitest'; + +import type { WebhookConfig } from '../src/config.js'; +import { createNotifier, type NotifierLogger } from '../src/lib/notifications.js'; + +const silentLog: NotifierLogger = { + debug: () => {}, + warn: () => {}, +}; + +interface ReceivedCall { + headers: Record; + raw: string; + body: Record; +} + +interface Receiver { + url: string; + calls: ReceivedCall[]; + setStatus: (status: number) => void; + close: () => Promise; +} + +async function startReceiver(): Promise { + const calls: ReceivedCall[] = []; + let status = 200; + const app = Fastify({ logger: false }); + // Corps brut : on vérifie la signature sur les octets exacts reçus. + app.addContentTypeParser('application/json', { parseAs: 'string' }, (_req, body, done) => { + done(null, body); + }); + app.post('/hook', (request, reply) => { + const raw = request.body as string; + calls.push({ + headers: request.headers, + raw, + body: JSON.parse(raw) as Record, + }); + return reply.status(status).send({ ok: status < 400 }); + }); + await app.listen({ port: 0, host: '127.0.0.1' }); + const address = app.server.address(); + if (address === null || typeof address === 'string') { + await app.close(); + throw new Error('adresse du récepteur inattendue'); + } + return { + url: `http://127.0.0.1:${String(address.port)}/hook`, + calls, + setStatus: (next: number) => { + status = next; + }, + close: () => app.close(), + }; +} + +function webhook(url: string, over: Partial = {}): WebhookConfig { + return { url, events: ['upload', 'error'], headers: {}, timeoutMs: 2000, ...over }; +} + +const UPLOAD_INPUT = { + target: 'videos', + storedName: 'vac/plage/clip.mp4', + originalName: 'clip.mp4', + size: 5 * 1024 * 1024, + mimeType: 'video/mp4', + user: 'user-123', + via: 'chunked' as const, +}; + +let receiver: Receiver; + +afterEach(async () => { + await receiver.close(); +}); + +describe('createNotifier', () => { + it('POSTe un corps structuré avec résumé lisible sur upload finalisé', async () => { + receiver = await startReceiver(); + const notifier = createNotifier([webhook(receiver.url)], silentLog); + + notifier.uploadCompleted(UPLOAD_INPUT); + await notifier.close(); + + expect(receiver.calls).toHaveLength(1); + const call = receiver.calls[0]!; + expect(call.headers['content-type']).toMatch(/^application\/json/); + expect(call.headers['x-mediadrop-event']).toBe('upload'); + expect(call.headers['user-agent']).toBe('mediaDrop-webhook'); + expect(call.body).toMatchObject({ + event: 'upload', + target: 'videos', + storedName: 'vac/plage/clip.mp4', + originalName: 'clip.mp4', + size: 5 * 1024 * 1024, + mimeType: 'video/mp4', + user: 'user-123', + via: 'chunked', + }); + expect(typeof call.body.timestamp).toBe('string'); + expect(call.body.text).toBe(call.body.content); + expect(call.body.text).toContain('clip.mp4'); + expect(call.body.text).toContain('5.0 Mo'); + }); + + it('envoie un corps `error` avec statut, méthode et chemin', async () => { + receiver = await startReceiver(); + const notifier = createNotifier([webhook(receiver.url)], silentLog); + + notifier.uploadFailed({ + message: 'Type de média non autorisé : application/zip', + statusCode: 400, + method: 'POST', + path: '/upload', + target: 'videos', + }); + await notifier.close(); + + expect(receiver.calls[0]!.body).toMatchObject({ + event: 'error', + message: 'Type de média non autorisé : application/zip', + statusCode: 400, + method: 'POST', + path: '/upload', + target: 'videos', + }); + expect(receiver.calls[0]!.headers['x-mediadrop-event']).toBe('error'); + }); + + it('respecte le filtre `events` de chaque webhook', async () => { + receiver = await startReceiver(); + const notifier = createNotifier([webhook(receiver.url, { events: ['error'] })], silentLog); + + notifier.uploadCompleted(UPLOAD_INPUT); + notifier.uploadFailed({ message: 'boom', statusCode: 500, method: 'PUT', path: '/upload/x/0' }); + await notifier.close(); + + expect(receiver.calls).toHaveLength(1); + expect(receiver.calls[0]!.body.event).toBe('error'); + }); + + it('signe le corps avec HMAC-SHA256 quand `secret` est défini', async () => { + receiver = await startReceiver(); + const notifier = createNotifier([webhook(receiver.url, { secret: 'top-secret' })], silentLog); + + notifier.uploadCompleted(UPLOAD_INPUT); + await notifier.close(); + + const call = receiver.calls[0]!; + const expected = 'sha256=' + createHmac('sha256', 'top-secret').update(call.raw).digest('hex'); + expect(call.headers['x-mediadrop-signature']).toBe(expected); + }); + + it('transmet les en-têtes personnalisés', async () => { + receiver = await startReceiver(); + const notifier = createNotifier( + [webhook(receiver.url, { headers: { authorization: 'Bearer jeton', 'x-src': 'md' } })], + silentLog, + ); + + notifier.uploadCompleted(UPLOAD_INPUT); + await notifier.close(); + + expect(receiver.calls[0]!.headers.authorization).toBe('Bearer jeton'); + expect(receiver.calls[0]!.headers['x-src']).toBe('md'); + }); + + it('réessaie une fois sur 5xx puis abandonne sans lever', async () => { + receiver = await startReceiver(); + receiver.setStatus(503); + const notifier = createNotifier([webhook(receiver.url)], silentLog); + + notifier.uploadCompleted(UPLOAD_INPUT); + await expect(notifier.close()).resolves.toBeUndefined(); + + expect(receiver.calls).toHaveLength(2); + }); + + it('ne réessaie pas sur une erreur cliente 4xx', async () => { + receiver = await startReceiver(); + receiver.setStatus(404); + const notifier = createNotifier([webhook(receiver.url)], silentLog); + + notifier.uploadCompleted(UPLOAD_INPUT); + await notifier.close(); + + expect(receiver.calls).toHaveLength(1); + }); + + it('livre à plusieurs webhooks en parallèle', async () => { + receiver = await startReceiver(); + const notifier = createNotifier( + [webhook(receiver.url), webhook(receiver.url, { headers: { 'x-n': '2' } })], + silentLog, + ); + + notifier.uploadCompleted(UPLOAD_INPUT); + await notifier.close(); + + expect(receiver.calls).toHaveLength(2); + }); + + it('sans webhook configuré : non-opération, close() résout', async () => { + receiver = await startReceiver(); + const notifier = createNotifier([], silentLog); + + expect(() => { + notifier.uploadCompleted(UPLOAD_INPUT); + notifier.uploadFailed({ message: 'x', statusCode: 500, method: 'POST', path: '/upload' }); + }).not.toThrow(); + await expect(notifier.close()).resolves.toBeUndefined(); + expect(receiver.calls).toHaveLength(0); + }); +}); diff --git a/tests/webhooks-integration.test.ts b/tests/webhooks-integration.test.ts new file mode 100644 index 0000000..5969856 --- /dev/null +++ b/tests/webhooks-integration.test.ts @@ -0,0 +1,220 @@ +import { randomBytes } from 'node:crypto'; + +import Fastify, { type FastifyInstance } from 'fastify'; +import FormData from 'form-data'; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { buildApp } from '../src/app.js'; +import type { OidcConfig } from '../src/config.js'; +import { makeConfig, makeTempDir, removeDir, startStubIdp, type StubIdp } from './helpers.js'; + +const MIB = 1024 * 1024; + +interface HookCall { + event: string; + body: Record; +} + +interface Receiver { + url: string; + calls: HookCall[]; + close: () => Promise; +} + +async function startReceiver(): Promise { + const calls: HookCall[] = []; + const app = Fastify({ logger: false }); + app.post('/hook', (request, reply) => { + const body = request.body as Record; + calls.push({ event: String(body.event), body }); + return reply.status(204).send(); + }); + await app.listen({ port: 0, host: '127.0.0.1' }); + const address = app.server.address(); + if (address === null || typeof address === 'string') { + await app.close(); + throw new Error('adresse du récepteur inattendue'); + } + return { + url: `http://127.0.0.1:${String(address.port)}/hook`, + calls, + close: () => app.close(), + }; +} + +function oidcOverride(ref: StubIdp): OidcConfig { + return { + issuer: ref.issuer, + clientId: ref.clientId, + clientSecret: ref.clientSecret, + scopes: ['openid', 'profile', 'email'], + }; +} + +function sessionCookie(res: Response): string { + return (res.headers.get('set-cookie') ?? '').split(';')[0] ?? ''; +} + +let idp: StubIdp; +let receiver: Receiver; +let app: FastifyInstance; +let baseUrl: string; +let dir: string; +let cookie: string; + +async function loginViaHttp(): Promise { + const start = await fetch(`${baseUrl}/auth/login`, { redirect: 'manual' }); + const txn = sessionCookie(start); + const authUrl = new URL(start.headers.get('location') ?? ''); + idp.setNextIdToken({ nonce: authUrl.searchParams.get('nonce') ?? '' }); + const state = authUrl.searchParams.get('state') ?? ''; + const cb = await fetch(`${baseUrl}/auth/callback?code=x&state=${encodeURIComponent(state)}`, { + redirect: 'manual', + headers: { cookie: txn }, + }); + return sessionCookie(cb); +} + +beforeAll(async () => { + idp = await startStubIdp(); +}); + +afterAll(async () => { + await idp.close(); +}); + +beforeEach(async () => { + receiver = await startReceiver(); + dir = await makeTempDir(); + app = await buildApp( + makeConfig({ + targets: [{ name: 'videos', label: 'Vidéos', dir, allowDirectories: true }], + maxFileSizeBytes: 8 * MIB, + allowedMimeTypes: ['video/*', 'image/png'], + oidc: oidcOverride(idp), + webhooks: [{ url: receiver.url, events: ['upload', 'error'], headers: {}, timeoutMs: 2000 }], + }), + { logger: false }, + ); + await app.listen({ port: 0, host: '127.0.0.1' }); + const address = app.server.address(); + if (address === null || typeof address === 'string') throw new Error('adresse inattendue'); + baseUrl = `http://127.0.0.1:${String(address.port)}`; + cookie = await loginViaHttp(); +}); + +afterEach(async () => { + await app.close(); // vide les livraisons webhook en attente (onClose -> notifier.close) + await receiver.close(); + await removeDir(dir); +}); + +describe('webhooks — intégration', () => { + it('notifie `upload` après un dépôt direct', async () => { + const form = new FormData(); + form.append('file', randomBytes(2048), { filename: 'clip.mp4', contentType: 'video/mp4' }); + const res = await fetch(`${baseUrl}/upload?target=videos`, { + method: 'POST', + headers: { ...form.getHeaders(), cookie }, + body: form.getBuffer(), + }); + expect(res.status).toBe(201); + + await app.close(); + const uploads = receiver.calls.filter((c) => c.event === 'upload'); + expect(uploads).toHaveLength(1); + expect(uploads[0]!.body).toMatchObject({ + event: 'upload', + target: 'videos', + originalName: 'clip.mp4', + mimeType: 'video/mp4', + via: 'direct', + user: 'user-123', + }); + }); + + it('notifie `error` quand un dépôt échoue (type refusé)', async () => { + const form = new FormData(); + form.append('file', Buffer.from('PK'), { + filename: 'x.zip', + contentType: 'application/zip', + }); + const res = await fetch(`${baseUrl}/upload?target=videos`, { + method: 'POST', + headers: { ...form.getHeaders(), cookie }, + body: form.getBuffer(), + }); + expect(res.status).toBe(400); + + await app.close(); + const errors = receiver.calls.filter((c) => c.event === 'error'); + expect(errors).toHaveLength(1); + expect(errors[0]!.body).toMatchObject({ + event: 'error', + statusCode: 400, + method: 'POST', + path: '/upload', + target: 'videos', + }); + }); + + it('notifie `upload` (via: chunked) après finalisation fragmentée', async () => { + const data = randomBytes(200 * 1024); + const init = await fetch(`${baseUrl}/upload/init`, { + method: 'POST', + headers: { 'content-type': 'application/json', cookie }, + body: JSON.stringify({ + target: 'videos', + path: 'a/clip.mp4', + name: 'clip.mp4', + size: data.length, + type: 'video/mp4', + chunkSize: MIB, + }), + }); + const { id } = (await init.json()) as { id: string }; + await fetch(`${baseUrl}/upload/${id}/0`, { + method: 'PUT', + headers: { 'content-type': 'application/octet-stream', cookie }, + body: new Uint8Array(data), + }); + const done = await fetch(`${baseUrl}/upload/${id}/complete`, { + method: 'POST', + headers: { cookie }, + }); + expect(done.status).toBe(201); + + await app.close(); + const uploads = receiver.calls.filter((c) => c.event === 'upload'); + expect(uploads).toHaveLength(1); + expect(uploads[0]!.body).toMatchObject({ + event: 'upload', + target: 'videos', + storedName: 'a/clip.mp4', + originalName: 'clip.mp4', + via: 'chunked', + }); + }); + + it('ne notifie rien si aucun webhook n’est configuré', async () => { + const quiet = await buildApp( + makeConfig({ + targets: [{ name: 'videos', label: 'Vidéos', dir, allowDirectories: true }], + allowedMimeTypes: ['video/*'], + oidc: oidcOverride(idp), + webhooks: [], + }), + { logger: false }, + ); + await quiet.listen({ port: 0, host: '127.0.0.1' }); + const quietAddress = quiet.server.address(); + if (quietAddress === null || typeof quietAddress === 'string') throw new Error('adresse'); + const quietUrl = `http://127.0.0.1:${String(quietAddress.port)}`; + + const before = receiver.calls.length; + const res = await fetch(`${quietUrl}/upload?target=videos`, { method: 'POST', body: 'x' }); + expect(res.status).toBe(401); + await quiet.close(); + expect(receiver.calls.length).toBe(before); + }); +});