kyuno/batchCooking
1
0
Fork 0
Code Issues 5 Pull requests Projects Releases Packages Wiki Activity Actions 1

feat(recipes): rapatrie le corpus NLP cote Python et l'enrichit de 48 techniques

Browse source 
Changement d'architecture demande par l'utilisateur : le dataset
d'entrainement (TECH_STEP_TRAINING_DATA) quitte apps/api pour vivre
entierement dans services/tech-step-intent-service
(intent_service/training_data.py). Ce service est desormais autonome :
il s'entraine lui-meme une seule fois, a son propre demarrage
(PipelineRegistry.initialize, dans le lifespan FastAPI), sans plus
dependre d'un POST /v1/train pousse par apps/api (route supprimee).
apps/api ne connait plus aucune technique/synonyme, uniquement le
resultat de POST /v1/process.

Corpus enrichi avec les 48 techniques du lexique fourni (Arroser,
Appertiser, Braiser, Caraméliser, Confire, Julienne/Brunoise/Mirepoix/
Paysanne, Cuire à blanc/au bain-marie/à l'étouffée, Déglacer variantes,
Emulsionner, Glacer, Pocher, Réduire, Suer, Zester, etc.), soit 74
techniques au total (26 + 48). Integration complete bout en bout :
- reference-seed-data.ts : 48 nouvelles entrees TECH_STEPS
- apps/web/locales/fr/translation.json : libelles francais correspondants
- "Mitonner" fondu comme synonyme de simmer (pas une technique distincte,
  sa propre definition le dit)
- "Blanchir un oeuf" (whiskPale) distingue de "Blanchir un legume"
  (blanch, existant) via des synonymes en phrase complete plutot qu'au
  mot nu — filter_spans (deja en place) resout la collision par
  specificite

Impact performance mesure : le corpus elargi (74 classes vs 26) rend
l'entrainement bien plus lent a nombre d'iterations egal (150 iterations
depassait 17 minutes par run de test) — reduit a 40 iterations apres
mesures repetees en local (~200s/locale, ~400s pour fr+en combines).
docker-compose.yml (healthcheck start_period 600s), CI (timeout curl
600s) et le README du service documentent ce nouveau temps de demarrage.
CONFIDENCE_THRESHOLD recalibre a 0.2 par verification manuelle (0.75 puis
0.45 ne tenaient plus compte tenu du nombre de classes) — marque
explicitement comme placeholder en attendant une vraie repasse de
calibrate-tech-step-threshold.ts (necessite Postgres, indisponible dans
cet environnement).

Verifie : 28/28 tests pytest du service (suite complete re-ecrite pour
s'entrainer une seule fois par session sur le vrai corpus, fixture
partagee dans conftest.py), lint + build complets du monorepo. La suite
Mocha d'apps/api reste a confirmer via CI (le root hook mocha n'attend
plus l'entrainement, seulement CI's propre attente sur /health).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Nicolas 2026-08-25 22:59:13 +02:00
parent 74a0052431
commit 065ef2a31a
33 changed files with 2199 additions and 1393 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
.github/workflows/ci.yml vendored
View file

@ -94,7 +94,15 @@ jobs:
working-directory: services/tech-step-intent-service
run: |
uv run uvicorn intent_service.main:app --host 0.0.0.0 --port 8000 &
timeout 60 bash -c 'until curl -sf http://localhost:8000/health > /dev/null; do sleep 1; done'
# `/health` only returns 200 once this service has finished
# training itself from scratch (no model ever persisted to disk —
# see its own README) — measured at ~200s per locale (~400s for
# fr+en combined) against the current ~74-technique corpus, so
# this wait is generous rather than the fast "base models only"
# check it used to be before that service trained itself at
# startup (see docker-compose.yml's healthcheck for the same
# reasoning).
timeout 600 bash -c 'until curl -sf http://localhost:8000/health > /dev/null; do sleep 2; done'
- run: pnpm install --frozen-lockfile
- run: pnpm --filter api exec prisma migrate deploy

6
README.md
View file

@ -103,8 +103,10 @@ pnpm --filter api exec prisma migrate dev
pnpm --filter api prisma:seed
# Microservice de détection des techniques (spaCy) — requis, `pnpm dev:api`
# ne peut plus détecter aucune technique de cuisine sans lui (voir son
# propre README pour le détail)
# ne peut plus détecter aucune technique de cuisine sans lui. Lance-le en
# premier et laisse-le tourner : il s'entraîne lui-même à chaque démarrage
# (~7 minutes pour le corpus actuel, voir son propre README) avant de
# répondre quoi que ce soit sur /health.
cd services/tech-step-intent-service
uv sync
cp .env.example .env # édite-le : même INTENT_SERVICE_SECRET que apps/api/.env

62
apps/api/src/db/reference-seed-data.ts
View file

@ -62,11 +62,12 @@ export const UNITS: Array<{ uid: string; type: UnitType; toBaseFactor: number }>
//
// Just a flat list of stable ids here — the actual matching data (per-
// locale synonym lists + example phrasings the classifier trains on) lives
// in `lib/recipe-matching/tech-step-training-data.ts`'s
// `TECH_STEP_TRAINING_DATA`, not here: unlike this list, it's read by
// `TechStepClassifierService`'s training pass, not the seed script, so it
// doesn't belong alongside the rest of this file's DB-seeded reference
// data. Every entry here must have a matching entry there.
// in `services/tech-step-intent-service/intent_service/training_data.py`'s
// `TECH_STEP_TRAINING_DATA`, not here: it's owned and trained entirely by
// that separate Python service (see its own README), not read by this
// seed script at all, so it doesn't belong alongside the rest of this
// file's DB-seeded reference data. Every entry here must have a matching
// entry there.
export const TECH_STEPS: string[] = [
"cook",
"fry",
@ -94,6 +95,57 @@ export const TECH_STEPS: string[] = [
"bake",
"plate",
"coat",
// Lexique de techniques ajouté par la suite — voir
// `services/tech-step-intent-service/intent_service/training_data.py`
// pour les synonymes/phrases d'exemple de chacune.
"baste",
"appertize",
"whiskPale",
"goldenBrown",
"braise",
"truss",
"caramelize",
"score",
"lineMold",
"clarify",
"compote",
"concasse",
"confit",
"julienne",
"brunoise",
"mirepoix",
"paysanne",
"blindBake",
"bainMarie",
"smother",
"decant",
"dilute",
"punchDown",
"disgorge",
"loosen",
"shellEgg",
"scald",
"pod",
"emulsify",
"hollowOut",
"shock",
"setGel",
"glaze",
"thicken",
"filet",
"proof",
"peelBlanch",
"whipUp",
"moisten",
"pasteurize",
"poach",
"reduce",
"rubIn",
"dustWithFlour",
"sweat",
"sift",
"toast",
"zest",
];
// The 14 allergens EU Regulation 1169/2011 (Annex II) requires food

35
apps/api/src/lib/recipe-matching/intent-service-client.ts
View file

@ -2,11 +2,14 @@ import { env } from "../../config/env.js";
/**
* Thin fetch wrapper around `services/tech-step-intent-service`'s HTTP
* contract (`/v1/train`, `/v1/process`) — the microservice
* contract (`POST /v1/process`) — the microservice
* {@link TechStepClassifierService} (`tech-step-matcher.ts`) delegates NER +
* intent classification to, in place of the `node-nlp` `NlpManager` it used
* to own directly. See that service's own README for the full contract and
* why it never touches Postgres itself.
* why it never touches Postgres itself — it also owns its own training
* corpus now (`training_data.py`), trained once at its own startup, so
* `apps/api` never pushes anything to it; `process()` below is this
* client's only method.
*
* Authenticated with `INTENT_SERVICE_SECRET` — the inverse direction of
* `requireInternalWorker`'s `INTERNAL_WORKER_SECRET` (this time `apps/api`
@ -21,20 +24,13 @@ export interface IntentServiceEntity {
end: number;
}
/** The full result of a `POST /v1/process` call — mirrors `ProcessResponse` (Python `schemas.py`). `intent` is `null` only when `locale` was never trained or `text` is blank; otherwise always a real `uid` (the Python service's `textcat` has no "None" sentinel, unlike node-nlp — see that service's README). */
/** The full result of a `POST /v1/process` call — mirrors `ProcessResponse` (Python `schemas.py`). `intent` is `null` only when `locale` isn't one this service trains for, or `text` is blank; otherwise always a real `uid` (the Python service's `textcat` has no "None" sentinel, unlike node-nlp — see that service's README). */
export interface IntentServiceProcessResult {
entities: IntentServiceEntity[];
intent: string | null;
score: number;
}
/** One technique's training data for one locale, as sent to `POST /v1/train` — mirrors `TrainEntryPayload` (Python `schemas.py`), itself shaped after `TechStepLocaleTrainingData` (`tech-step-training-data.ts`). */
export interface IntentServiceTrainEntry {
uid: string;
synonyms: string[];
utterances: string[];
}
/**
* Client for `services/tech-step-intent-service` — a real class (not a
* plain object of functions) per this repo's service-style-logic
@ -81,25 +77,6 @@ export class IntentServiceClient {
}
}
/**
* (Re)trains the intent service's pipeline for `locale` from `entries` —
* called once per locale by `TechStepClassifierService._train`, itself
* memoized so this only ever runs once per server process (see that
* method's own doc comment). Reconstructs the whole pipeline server-side,
* never a partial/incremental update — same "always retrains fresh from
* the one source of truth" posture the old in-process `NlpManager` had.
*/
public async train(locale: string, entries: IntentServiceTrainEntry[]): Promise<void> {
try {
await this._request("/v1/train", {
method: "POST",
body: JSON.stringify({ locale, entries }),
});
} catch (err) {
throw err;
}
}
/**
* Equivalent to the old `NlpManager.process(locale, text)` — returns every
* candidate technique mention (NER) plus the intent classifier's verdict

6
apps/api/src/lib/recipe-matching/tech-step-eval-dataset.ts
View file

@ -2,8 +2,8 @@
* Hand-labeled evaluation set for {@link techStepClassifier} — what
* `tech-step-eval.test.ts` runs the real classifier against to compute
* precision/recall/F1 (`tech-step-evaluator.ts`), the objective gate any
* future change to `tech-step-training-data.ts` must clear (see that
* module's own doc comment).
* future change to `services/tech-step-intent-service`'s `training_data.py`
* must clear (see that module's own doc comment).
*
* Deliberately *not* reusing `TECH_STEP_TRAINING_DATA`'s own `utterances`
* verbatim — scoring the classifier against the exact sentences it was
@ -246,7 +246,7 @@ export const TECH_STEP_EVAL_DATASET: TechStepEvalCase[] = [
// --- Documented false-positive traps, re-verified with fresh wording ---
// `brown`'s EN synonyms are verb forms only ("browned"/"browning"), not
// bare "brown" — precisely so this doesn't false-positive (see that
// entry's own comment in tech-step-training-data.ts).
// entry's own comment in training_data.py).
{
description: "This recipe calls for two tablespoons of brown sugar.",
locale: "en",

5
apps/api/src/lib/recipe-matching/tech-step-evaluator.ts
View file

@ -3,8 +3,9 @@
* hand-labeled evaluation set (`tech-step-eval-dataset.ts`) — the objective
* counterpart to the "inspected by eye" verdict every corpus change used to
* get before this module existed. Every future edit to
* `tech-step-training-data.ts` (including the LLM-assisted suggestions the
* worker in `services/tech-step-llm-worker` proposes) is expected to run
* `services/tech-step-intent-service`'s `training_data.py` (including the
* LLM-assisted suggestions the worker in `services/tech-step-llm-worker`
* proposes) is expected to run
* through `tech-step-eval.test.ts`'s regression gate, which calls
* {@link computeTechStepMetrics} — a corpus change that raises recall on one
* technique but silently tanks another's precision should fail loudly here,

156
apps/api/src/lib/recipe-matching/tech-step-matcher.ts
View file

@ -1,6 +1,5 @@
import { prisma } from "../../db/prisma.js";
import { intentServiceClient } from "./intent-service-client.js";
import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
/**
* Auto-detects which cooking techniques (`TechStep`) a free-text recipe
@ -15,13 +14,14 @@ import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
* generalize past its own vocabulary — a step describing melting butter as
* "jusqu'à ce que le beurre ait disparu dans la poêle" mentions no verb any
* regex could anchor on, yet unmistakably *means* `melt`. Replaced with a
* small hybrid pipeline (originally built on `node-nlp`, now delegated to
* `services/tech-step-intent-service` — a spaCy-based microservice, see
* {@link IntentServiceClient} and that service's own README):
* small hybrid pipeline (originally built on `node-nlp`, now entirely
* delegated to `services/tech-step-intent-service` — a spaCy-based
* microservice, see {@link IntentServiceClient} and that service's own
* README):
*
* 1. **NER** (the intent service's `PhraseMatcher`, built from `synonyms` in
* `TECH_STEP_TRAINING_DATA`) finds every *candidate* technique mention in
* the whole description, each with its exact character span —
* 1. **NER** (the intent service's `PhraseMatcher`, built from its own
* `training_data.py`'s `synonyms`) finds every *candidate* technique
* mention in the whole description, each with its exact character span —
* mechanically the same job the old regexes did, just as flat synonym
* lists instead of hand-written patterns. This step alone is *not* the
* final answer — see step 3.
@ -31,7 +31,7 @@ import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
* and `melt`) needs each judged on its own surrounding context, not the
* whole step lumped into one classification.
* 3. **NLP intent classification** (the intent service's `textcat`, trained
* on `TECH_STEP_TRAINING_DATA`'s `utterances`) then classifies each
* on its own `training_data.py`'s `utterances`) then classifies each
* clause on its own — this is what actually delivers "meaning, not
* keywords": the classifier was deliberately trained on paraphrases that
* never use the technique's own verb (e.g. "jusqu'à ce que le beurre ait
@ -50,12 +50,12 @@ import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
*
* `normalizeText` and {@link splitIntoClauses} are pure (no DB/model
* access) so they stay unit-testable in isolation (see
* `test/tech-step-matcher.test.ts`); the classifier itself needs a one-time
* training pass (`_ensureTrained`, a `POST /v1/train` call per locale to the
* intent service) plus a `TechStep.key -> id` lookup from the DB, both
* memoized on the shared {@link techStepClassifier} singleton rather than
* repeated per call — training is the expensive part, never worth redoing
* per request let alone per step.
* `test/tech-step-matcher.test.ts`); this class only ever needs a
* `TechStep.key -> id` lookup from the DB, memoized on the shared
* {@link techStepClassifier} singleton rather than repeated per call — the
* NLP model itself trains once, inside `services/tech-step-intent-service`'s
* own startup, entirely independently of this class (see that service's
* README — this repo no longer pushes any corpus to it over HTTP).
*/
/**
@ -242,32 +242,29 @@ export function splitIntoClauses(
* `TECH_STEP_TRAINING_DATA` — see `test/tech-step-matcher.test.ts` for the
* cases this threshold was picked to pass.
*
* Recalibrated to `0.45` for the migration off `node-nlp` to
* `services/tech-step-intent-service` (spaCy `textcat`, exclusive classes
* over ~26 techniques) — its score distribution is meaningfully different
* from node-nlp's own classifier. With that service's training tuned for
* real confidence rather than just correct argmax (see
* `_TRAINING_ITERATIONS`/`_TRAINING_DROPOUT` in
* `services/tech-step-intent-service/intent_service/locale_pipeline.py`),
* genuine matches score comfortably above `0.45` with real margin: `melt`
* scores `~0.95` on "jusqu'à ce que le beurre ait disparu dans la poêle"
* (the exact motivating no-keyword case this pipeline exists for, no NER
* anchor to fall back to), `preheat` `~0.90` on "mettre la poêle sur feu
* vif", down to `~0.51` for the weakest real case seen (`bake`, anchored).
* The noise floor stays far below all of them: English recipe text run
* through the French classifier (must find *nothing*, confirmed by
* `recipe-translation.test.ts`'s own locale-isolation test) scores `~0.05`
* for every technique — indistinguishable from the ~1/26 uniform baseline
* over this many exclusive classes. Cross-checked against
* `apps/api/src/scripts/calibrate-tech-step-threshold.ts`'s sweep over
* `TECH_STEP_EVAL_DATASET`: aggregate F1 climbs to its plateau (`0.987`)
* starting at `0.45` and stays flat through `0.95`, so this is the lowest
* threshold that already captures every gain available from trusting the
* classifier more — a higher value would only ever risk discarding a
* genuine anchor-less match like the two above, never buy back any
* precision.
* Recalibrated for the migration off `node-nlp` to
* `services/tech-step-intent-service` (spaCy `textcat`, exclusive classes)
* — its score distribution is meaningfully different from node-nlp's own
* classifier, and shifts again every time the corpus' technique count
* changes (more exclusive classes generally means a *lower* natural
* confidence ceiling, softmax mass spread thinner).
*
* Currently `0.2`, set against the corpus as expanded to ~74 techniques
* (`services/tech-step-intent-service/intent_service/training_data.py`,
* `_TRAINING_ITERATIONS = 40`) from manual spot-checks, not yet a real
* `calibrate-tech-step-threshold.ts` sweep against
* `TECH_STEP_EVAL_DATASET` (needs Postgres — see that script's own doc
* comment): observed real-case scores ranged `0.25`-`0.89` (`simmer`
* lowest, still correct and anchored anyway; `melt` highest, the
* motivating anchor-less case), against a noise floor around `0.02`
* (English text through the French classifier). `0.2` sits with margin
* above the noise floor and below every real case seen so far, but **this
* is a placeholder pending the real eval-dataset sweep** — do not treat it
* as load-bearing precision the way the previous `0.45` (calibrated
* against the ~26-technique corpus, `TECH_STEP_EVAL_DATASET` F1 plateauing
* exactly there) was.
*/
export const CONFIDENCE_THRESHOLD = 0.45;
export const CONFIDENCE_THRESHOLD = 0.2;
/**
* One clause's full classification detail — the finer-grained sibling of
@ -295,29 +292,34 @@ export interface TechStepClauseClassification {
}
/**
* Owns the trained state behind {@link matchTechStepSpans} — a real class
* (not a plain object of functions) per this repo's service-style-logic
* convention, even though it's only ever used as the one shared
* {@link techStepClassifier} singleton below: it holds real state (the
* memoized training/lookup promises), not just grouped stateless helpers.
* The actual NER/intent-classification model lives in
* `services/tech-step-intent-service` (a separate process) — this class's
* own state is just what it needs to talk to that service correctly
* (whether training has been kicked off yet, and the `TechStep.key -> id`
* lookup that service's `uid`s must still be resolved through).
* Owns the `TechStep.key -> id` lookup behind {@link matchTechStepSpans} —
* a real class (not a plain object of functions) per this repo's
* service-style-logic convention, even though it's only ever used as the
* one shared {@link techStepClassifier} singleton below: it holds real
* state (the memoized lookup promise), not just grouped stateless helpers.
* The actual NER/intent-classification model lives entirely in
* `services/tech-step-intent-service` (a separate process, trained from
* its own `training_data.py` at its own startup) — this class never
* trains or pushes anything to it, it only calls `POST /v1/process` and
* resolves whatever `uid` comes back to a local DB id.
*/
export class TechStepClassifierService {
/** Memoized training pass — `undefined` until the first call starts it, after which every caller (concurrent or not) awaits the same promise rather than retraining. */
private _trained: Promise<void> | undefined;
/** Memoized `TechStep.key -> id` lookup — training data only knows techniques by their stable `uid`/`key`, resolved to the real DB id once, alongside training. */
/** Memoized `TechStep.key -> id` lookup — resolved from the DB once, reused by every call rather than queried per request. `undefined` until the first call starts loading it, after which every caller (concurrent or not) awaits the same promise. */
private _techStepIdsLoaded: Promise<void> | undefined;
private _techStepIdByUid: Map<string, number> | undefined;
/**
* Forces training (a `POST /v1/train` call per locale to
* `services/tech-step-intent-service`) to happen now, synchronously with
* Forces the `TechStep.key -> id` lookup to load now, synchronously with
* server startup (see `server.ts`, which also retries this against a
* not-yet-ready intent service), rather than stalling whichever request
* happens to be first to save/preview a recipe.
* not-yet-reachable intent service), rather than stalling whichever
* request happens to be first to save/preview a recipe. Doesn't wait on
* `services/tech-step-intent-service` finishing its own training — that
* service is only ever considered "up" by Docker Compose/CI once it
* already is (see that service's `GET /health`), so by the time this
* runs in a real deployment it's already trained; a request racing an
* intent service that's genuinely still starting just gets an empty
* match list back (see `IntentServiceProcessResult`'s own doc comment),
* not an error.
*/
public async warmUp(): Promise<void> {
try {
@ -341,7 +343,7 @@ export class TechStepClassifierService {
*/
public async matchTechStepSpans(description: string, locale: string): Promise<TechStepMatch[]> {
try {
await this._ensureTrained();
await this._ensureTechStepIdsLoaded();
if (description.trim().length === 0) return [];
// The intent service only ever returns enum-style candidates (its own
@ -405,7 +407,7 @@ export class TechStepClassifierService {
locale: string,
): Promise<TechStepClauseClassification[]> {
try {
await this._ensureTrained();
await this._ensureTechStepIdsLoaded();
if (description.trim().length === 0) return [];
const nerResult = await intentServiceClient.process(locale, description);
@ -491,46 +493,34 @@ export class TechStepClassifierService {
}
/**
* Trains `services/tech-step-intent-service` from
* {@link TECH_STEP_TRAINING_DATA} and resolves the `uid -> TechStep.id`
* lookup, both exactly once — memoized on `_trained` so a burst of
* concurrent calls (several steps of the same recipe save, awaited via
* the same event loop tick) all await the one in-flight training pass
* rather than each kicking off their own.
* Resolves the `uid -> TechStep.id` lookup exactly once — memoized on
* `_techStepIdsLoaded` so a burst of concurrent calls (several steps of
* the same recipe save, awaited via the same event loop tick) all await
* the one in-flight DB query rather than each firing their own.
*/
private async _ensureTrained(): Promise<void> {
if (this._trained === undefined) {
this._trained = this._train();
private async _ensureTechStepIdsLoaded(): Promise<void> {
if (this._techStepIdsLoaded === undefined) {
this._techStepIdsLoaded = this._loadTechStepIds();
}
try {
await this._trained;
await this._techStepIdsLoaded;
} catch (err) {
// A failed training pass must be retried by the *next* call, not
// leave every future call permanently rejecting against a stale
// failed promise.
this._trained = undefined;
// A failed load must be retried by the *next* call, not leave every
// future call permanently rejecting against a stale failed promise.
this._techStepIdsLoaded = undefined;
throw err;
}
}
private async _train(): Promise<void> {
private async _loadTechStepIds(): Promise<void> {
try {
const techSteps = await prisma.techStep.findMany({ select: { id: true, key: true } });
this._techStepIdByUid = new Map(techSteps.map((techStep) => [techStep.key, techStep.id]));
for (const locale of ["fr", "en"] as const) {
const entries = TECH_STEP_TRAINING_DATA.map((entry) => ({
uid: entry.uid,
synonyms: entry[locale].synonyms,
utterances: entry[locale].utterances,
}));
await intentServiceClient.train(locale, entries);
}
} catch (err) {
throw err; // see matchTechStepSpans()'s catch comment above
}
}
}
/** Single shared instance — training is expensive enough (a couple of minutes total, both locales combined — see `services/tech-step-intent-service`'s own `_TRAINING_ITERATIONS`) that every caller must reuse the one already-trained model, never spin up their own. */
/** Single shared instance — every caller reuses the one memoized `TechStep.key -> id` lookup rather than re-querying the DB. The actual model training (expensive — a couple of minutes, both locales combined) happens entirely inside `services/tech-step-intent-service`'s own startup, not here — see that service's `_TRAINING_ITERATIONS`. */
export const techStepClassifier = new TechStepClassifierService();

911
apps/api/src/lib/recipe-matching/tech-step-training-data.ts
View file

@ -1,911 +0,0 @@
/**
* Training corpus for {@link TechStepClassifierService} (`tech-step-matcher.ts`)
* — one entry per `TechStep` (`uid` matches `reference-seed-data.ts`'s
* `TECH_STEPS`, which still owns the reference `TechStep` rows themselves;
* this file replaces `TECH_STEPS[].mappings`' regex expressions as the
* *matching* data source).
*
* Two distinct kinds of content per technique/locale, feeding two distinct
* mechanisms of the classifier (see that file's doc comment for why both
* are needed):
*
* - `synonyms` — short literal words/set phrases, fed to node-nlp's NER
* (enum entities). Mechanically equivalent to the old regexes' verb-form
* alternations, just spelled out as plain words instead of a pattern
* (node-nlp's own stemmer/fuzzy matching already covers minor
* conjugation/typo variance that the regexes had to enumerate by hand).
* Used only to find *candidate* technique mentions and cut a step into
* clauses around them — never the final answer on their own.
* - `utterances` — full example clauses, fed to node-nlp's NLP Manager as
* training documents for the intent classifier. Deliberately mixes
* keyword-anchored phrasings (reinforces the obvious case) with
* paraphrases that never use the technique's own verb at all (e.g.
* "jusqu'à ce que le beurre ait disparu" for `melt`) — this second kind
* is what actually delivers on "comprendre le sens, pas juste les mots
* clés" (see the PR this file was introduced in): a clause reaching the
* classifier gets labeled by what it's trained to recognize as *meaning*
* this technique, not by which literal word triggered its extraction.
*
* Kept as static in-code data (not DB rows, unlike the old
* `TechStepMapping` table) because nothing needs to query/edit it at
* runtime — it only ever feeds one thing, the classifier's one-time
* training pass (see `TechStepClassifierService._ensureTrained`) — same
* reasoning `INGREDIENT_LABELS_EN` (`packages/shared`) is a plain object,
* not a database table.
*/
/** One technique's matching data for one locale — see this file's doc comment for what each list feeds. */
export interface TechStepLocaleTrainingData {
synonyms: string[];
utterances: string[];
}
/** One technique's full training entry — `uid` must match a `TECH_STEPS[].uid` in `reference-seed-data.ts`. */
export interface TechStepTrainingEntry {
uid: string;
fr: TechStepLocaleTrainingData;
en: TechStepLocaleTrainingData;
}
export const TECH_STEP_TRAINING_DATA: TechStepTrainingEntry[] = [
{
uid: "cook",
fr: {
synonyms: [
"cuire",
"cuisez",
"cuisant",
"cuisson",
"cuit",
"cuite",
"cuites",
"cuits",
"cuisiner",
"cuisinez",
"cuisiné",
"cuisinée",
"faire cuire",
"laisser cuire",
],
utterances: [
"faire cuire à feu moyen",
"laisser cuire jusqu'à ce que ce soit prêt",
"la cuisson dure environ dix minutes",
"jusqu'à ce que la viande ne soit plus rose au centre",
"poursuivre la cuisson à couvert",
// Two real recipe clauses found misclassified (as `preheat` and
// `panFry` respectively, both above the confidence threshold) once
// real, longer, comma-heavy sentences started reaching the
// classifier — neither error came from a missing keyword (both
// clauses' own NER anchor, "laisser cuire"/"faire cuire", was
// already right), just the classifier's low-heat/occasional-
// stirring phrasing not resembling anything short and clean-cut it
// had actually been trained on.
"baisser le feu et laisser cuire à découvert encore un quart d'heure",
"faire cuire à feu doux en remuant de temps en temps",
],
},
en: {
// NOT "cooked through"/"cooking through" — both are word-prefix
// extensions of "cooked"/"cooking" above, so any text containing them
// matches BOTH the short and long form as separate overlapping NER
// candidates, corrupting clause-splitting (confirmed via "It should
// be cooking through evenly", which spuriously grew a second,
// wrongly-classified `roast` candidate). See this pattern flagged
// throughout the file wherever it was found — the fix is always to
// drop the longer, redundant form rather than keep both.
synonyms: ["cook", "cooks", "cooked", "cooking"],
utterances: [
"cook over medium heat",
"cook until done",
"cooking takes about ten minutes",
"until no longer pink in the middle",
"continue cooking covered",
],
},
},
{
uid: "fry",
fr: {
synonyms: [
"frire",
"frit",
"frite",
"frites",
"friture",
"faire frire",
"faites frire",
"bain de friture",
"huile de friture",
],
utterances: [
"faire frire dans l'huile chaude",
"plonger dans la friture",
"jusqu'à ce que ce soit doré et croustillant à l'extérieur",
"l'huile doit être bien chaude avant d'y plonger les morceaux",
],
},
en: {
// NOT "frying oil" — a word-prefix extension of "frying" above (see
// the `cook` entry's comment for why that duplicates/corrupts NER
// candidates; here it was even worse, misclassifying as `preheat`).
synonyms: ["fry", "fries", "fried", "frying", "deep fry", "deep-fried", "deep frying"],
utterances: [
"fry in hot oil",
"deep fry until golden",
"until crisp and golden on the outside",
"the oil should be very hot before adding the pieces",
],
},
},
{
uid: "melt",
fr: {
synonyms: [
"fondre",
"fondu",
"fondue",
"fondues",
"faire fondre",
"faites fondre",
// Also a plausible way to say "melt" (heating something — usually
// a fat — until it liquefies), not just a `preheat` phrasing —
// restores what the regex-based system anchored on before this
// pipeline replaced it.
"faire chauffer",
"faites chauffer",
"liquéfier",
"liquéfiez",
"liquéfié",
"faire liquéfier",
],
utterances: [
"faire fondre le beurre",
"jusqu'à ce que le beurre ait disparu dans la poêle",
"le beurre doit être complètement liquide",
"laisser le fromage devenir tout liquide sur feu doux",
],
},
en: {
synonyms: ["melt", "melts", "melted", "melting", "liquefy", "liquefied"],
utterances: [
"melt the butter",
"until the butter has completely disappeared into the pan",
"the butter should be fully liquid",
"let the cheese turn completely liquid over low heat",
],
},
},
{
uid: "deglaze",
fr: {
// NOT "déglacer la poêle"/"déglacer le fond de cuisson" — both are
// word-prefix extensions of "déglacer" above (see `cook`'s comment
// for why that duplicates NER candidates).
synonyms: ["déglacer", "déglacez", "déglacé", "déglacée", "déglaçage"],
utterances: [
"déglacer avec le vin blanc",
"verser le vin dans la poêle chaude pour décoller les sucs",
"gratter les sucs de cuisson au fond de la casserole avec un peu de bouillon",
],
},
en: {
// NOT "deglaze the pan" — a word-prefix extension of "deglaze" above
// (see `cook`'s comment for why that duplicates NER candidates).
synonyms: ["deglaze", "deglazes", "deglazed", "deglazing", "lift the browned bits"],
utterances: [
"deglaze with white wine",
"pour the wine into the hot pan to lift the browned bits",
"scrape up the browned bits at the bottom of the pan with a splash of stock",
],
},
},
{
uid: "simmer",
fr: {
synonyms: [
"mijoter",
"mijotez",
"mijote",
"mijotant",
"mijoté",
"frémir",
"frémissant",
"frémissante",
"à petit feu",
],
utterances: [
"laisser mijoter à feu doux",
"faire mijoter pendant une heure",
"de petites bulles doivent remonter doucement à la surface",
"laisser cuire tout doucement à couvert pendant longtemps",
],
},
en: {
// NOT "simmering gently" — a word-prefix extension of "simmering"
// above (see `cook`'s comment for why that duplicates NER candidates).
synonyms: ["simmer", "simmers", "simmered", "simmering", "gentle simmer", "low simmer"],
utterances: [
"let it simmer over low heat",
"simmer for one hour",
"small bubbles should gently rise to the surface",
"let it cook very gently, covered, for a long time",
],
},
},
{
uid: "boil",
fr: {
synonyms: [
"bouillir",
"bouillant",
"bouillie",
"bouillies",
"ébullition",
"porter à ébullition",
"gros bouillons",
],
utterances: [
"porter à ébullition",
"faire bouillir l'eau",
"de grosses bulles doivent agiter la surface avec force",
"jusqu'à ce que ça bouillonne franchement",
],
},
en: {
// NOT "boiling point" — a word-prefix extension of "boiling" above
// (see `cook`'s comment for why that duplicates NER candidates).
synonyms: ["boil", "boils", "boiled", "boiling", "rolling boil"],
utterances: [
"bring to a boil",
"boil the water",
"large bubbles should be vigorously breaking the surface",
"until it's rolling vigorously",
],
},
},
{
uid: "roast",
fr: {
// NOT "rôti au four" — a word-prefix extension of "rôti" above (see
// `cook`'s comment for why that duplicates NER candidates).
synonyms: ["rôtir", "rôti", "rôtie", "rôties", "rôtis", "rôtissage"],
utterances: [
"faire rôtir la volaille entière",
"le rôti doit dorer uniformément de tous les côtés",
"cuire la pièce de viande entière au four à chaleur sèche",
],
},
en: {
synonyms: ["roast", "roasts", "roasted", "roasting", "oven-roast", "oven roasted"],
utterances: [
"roast the whole bird",
"it should brown evenly on every side",
"cook the whole piece of meat in dry oven heat",
],
},
},
{
uid: "grill",
fr: {
synonyms: [
"griller",
"grillez",
"grillé",
"grillée",
"grillées",
"grillade",
"grillades",
"barbecue",
"au barbecue",
],
utterances: [
"faire griller sur la grille du barbecue",
"marquer les steaks sur une plaque brûlante",
"des traces de quadrillage doivent apparaître à la cuisson",
],
},
en: {
synonyms: ["grill", "grills", "grilled", "grilling", "barbecue", "char-grill", "charbroiled"],
utterances: [
"grill on the barbecue rack",
"sear the steaks on a scorching-hot plate",
"char marks should appear as it cooks",
],
},
},
{
uid: "panFry",
fr: {
// Deliberately NOT "poêlé"/"poêlée"/"poêlés" here, despite reading
// like natural panFry vocabulary: node-nlp's French stemmer reduces
// them to the same root as the bare noun "poêle" (a pan), so
// registering them made every plain mention of "poêle" — e.g.
// `preheat`'s own "la poêle" — a false-positive panFry candidate too.
// Found via the "jusqu'à ce que le beurre ait disparu dans la poêle"
// regression test, which unexpectedly grew a spurious panFry match.
synonyms: ["sauter", "sautez", "sauté", "sautée", "sautées", "sautant", "à la poêle"],
utterances: [
"faire sauter les légumes à la poêle",
"saisir rapidement à feu vif en remuant sans cesse",
"faire revenir en remuant vivement dans une poêle très chaude",
],
},
en: {
synonyms: [
"sauté",
"sauteed",
"sautéed",
"sauteing",
"pan-fry",
"pan fried",
"pan-fried",
"stir-fry",
"pan searing",
"seared in a pan",
],
utterances: [
"sauté the vegetables in a pan",
"quickly sear over high heat, stirring constantly",
"cook briskly, stirring, in a very hot pan",
],
},
},
{
uid: "blanch",
fr: {
synonyms: ["blanchir", "blanchissez", "blanchi", "blanchie", "blanchies", "blanchiment"],
utterances: [
"faire blanchir les légumes deux minutes dans l'eau bouillante",
"plonger brièvement dans l'eau bouillante puis directement dans l'eau glacée",
"cuire très rapidement à l'eau bouillante avant de stopper la cuisson au froid",
],
},
en: {
// "parboil" is folded in here rather than kept a separate technique —
// in home-cooking usage (as opposed to professional usage, where they
// can differ) it names the same "briefly pre-cook in boiling water"
// move blanching does.
synonyms: [
"blanch",
"blanches",
"blanched",
"blanching",
"parboil",
"parboiled",
"parboiling",
],
utterances: [
"blanch the vegetables for two minutes in boiling water",
"briefly plunge into boiling water then straight into ice water",
"cook very quickly in boiling water before stopping it cold",
],
},
},
{
uid: "marinate",
fr: {
synonyms: [
"mariner",
"marinez",
"mariné",
"marinée",
"marinées",
"marinade",
"macérer",
"macérez",
"macération",
"faire mariner",
],
utterances: [
"laisser mariner la viande toute la nuit au réfrigérateur",
"faire tremper dans la sauce plusieurs heures avant cuisson pour parfumer",
"laisser reposer dans le mélange d'huile et d'épices avant de cuisiner",
],
},
en: {
// NOT "marinating for" — a word-prefix extension of "marinating"
// above (see `cook`'s comment for why that duplicates NER candidates
// — here it was even worse, misclassifying as `simmer`).
synonyms: [
"marinate",
"marinates",
"marinated",
"marinating",
"marinade",
"soak in the marinade",
],
utterances: [
"let the meat marinate overnight in the fridge",
"soak in the sauce for several hours before cooking to flavor it",
"let it sit in the oil and spice mixture before cooking",
],
},
},
{
uid: "chop",
fr: {
// NOT "hacher grossièrement" — a word-prefix extension of "hacher"
// above (see `cook`'s comment for why that duplicates NER candidates).
synonyms: [
"hacher",
"hachez",
"haché",
"hachée",
"hachées",
"hachis",
"couper en morceaux",
"tailler en morceaux",
],
utterances: [
"hacher finement les oignons",
"couper en tout petits morceaux irréguliers au couteau",
"réduire les herbes en petits fragments avant de les ajouter",
],
},
en: {
// NOT "chop coarsely" — a word-prefix extension of "chop" above (see
// `cook`'s comment for why that duplicates NER candidates).
synonyms: ["chop", "chops", "chopped", "chopping", "roughly chop", "coarsely chopped"],
utterances: [
"finely chop the onions",
"cut into small, uneven pieces with a knife",
"break the herbs down into small bits before adding them",
],
},
},
{
uid: "peel",
fr: {
synonyms: [
"éplucher",
"épluchez",
"épluché",
"épluchée",
"épluchées",
"épluchage",
"peler",
"pelez",
"pelé",
"pelée",
"pelées",
],
utterances: [
"éplucher les pommes de terre",
"retirer la peau des carottes avec un économe",
"ôter la pelure du fruit avant de le couper",
],
},
en: {
synonyms: ["peel", "peels", "peeled", "peeling", "pare", "pared", "paring"],
utterances: [
"peel the potatoes",
"remove the skin from the carrots with a peeler",
"take the skin off the fruit before cutting it",
],
},
},
{
uid: "mince",
fr: {
synonyms: [
"émincer",
"émincez",
"émincé",
"émincée",
"émincées",
"ciseler",
"ciselez",
"ciselé",
"ciselée",
"ciselées",
],
utterances: [
"émincer l'oignon en fines lamelles",
"couper en très fines tranches régulières",
"détailler en lamelles aussi fines que possible",
// Without this, a short clause naming a different vegetable —
// "Émincer les tomates" — scored just above `melt`'s confidence
// threshold instead (a training-set-composition side effect of
// adding utterances elsewhere in this same pass, found by the full
// regression suite). A second example anchored on a different noun
// widens `mince`'s own region enough to reclaim it.
"émincer les tomates en fines rondelles",
],
},
en: {
// NOT "mince finely" — a word-prefix extension of "mince" above (see
// `cook`'s comment for why that duplicates NER candidates).
synonyms: ["mince", "minces", "minced", "mincing", "thinly slice", "finely mince"],
utterances: [
"mince the onion into thin strips",
"cut into very thin, even slices",
"slice into strips as thin as possible",
],
},
},
{
uid: "mix",
fr: {
synonyms: [
"mélanger",
"mélangez",
"mélangé",
"mélangée",
"mélangées",
"mélange",
"brasser",
"brassez",
"amalgamer",
"amalgamez",
],
utterances: [
"mélanger tous les ingrédients dans un saladier",
"combiner le sucre et la farine ensemble",
"remuer jusqu'à obtenir une préparation homogène",
],
},
en: {
synonyms: [
"mix",
"mixes",
"mixed",
"mixing",
"combine",
"combined",
"blend",
"blended",
"blending",
"stir together",
],
utterances: [
"mix all the ingredients in a bowl",
"combine the sugar and flour together",
"stir until the mixture is smooth and even",
],
},
},
{
uid: "whisk",
fr: {
synonyms: [
"fouetter",
"fouettez",
"fouetté",
"fouettée",
"fouettées",
"au fouet",
"battre au fouet",
"monter au fouet",
],
utterances: [
"fouetter les œufs et le sucre",
"battre vigoureusement au fouet jusqu'à ce que ça blanchisse",
"travailler énergiquement pour incorporer de l'air au mélange",
// Without these, "Fouetter les blancs en neige" misclassified as
// `foldIn` — its own training utterance below also happens to say
// "les blancs en neige", and node-nlp's intent classifier leaned on
// that shared noun phrase over the actual verb. The exact phrase
// itself is needed (not just a paraphrase of it) — a longer,
// differently-worded utterance alone wasn't enough to outweigh
// `foldIn`'s own close phrasing.
"fouetter les blancs en neige",
"fouetter les blancs en neige jusqu'à ce qu'ils soient fermes",
],
},
en: {
synonyms: ["whisk", "whisks", "whisked", "whisking", "beat", "whip", "whipped", "whipping"],
utterances: [
"whisk the eggs and sugar",
"beat vigorously with a whisk until pale",
"work it briskly to whip air into the mixture",
"whisk the egg whites until stiff peaks form",
],
},
},
{
uid: "foldIn",
fr: {
synonyms: [
"incorporer",
"incorporez",
"incorporé",
"incorporée",
"incorporées",
// NOT "incorporer délicatement" — it's a superstring of "incorporer"
// above, so both would match the same text and hand
// `splitIntoClauses` two overlapping candidates for one mention
// (found via "Incorporer délicatement la farine" producing two
// duplicate matches instead of one).
"mélanger délicatement",
],
utterances: [
"incorporer délicatement les blancs en neige",
"ajouter en soulevant doucement la masse pour ne pas casser les bulles",
"mélanger tout doucement de bas en haut pour garder l'air emprisonné",
],
},
en: {
synonyms: ["fold in", "folds in", "folded in", "folding in", "gently fold", "fold gently"],
utterances: [
"gently fold in the beaten egg whites",
"add by gently lifting the batter so you don't knock the air out",
"very gently stir from the bottom up to keep the air trapped in",
],
},
},
{
uid: "setAside",
fr: {
synonyms: [
"réserver",
"réservez",
"réservé",
"réservée",
"réservées",
"mettre de côté",
"laisser de côté",
],
utterances: [
"réserver au frais en attendant",
"mettre de côté pour plus tard",
"laisser attendre sur le plan de travail pendant la préparation du reste",
],
},
en: {
synonyms: ["set aside", "sets aside", "setting aside", "set it aside", "reserve", "reserved"],
utterances: [
"set aside in the fridge for now",
"put it aside for later",
"let it wait on the counter while you prepare the rest",
],
},
},
{
uid: "season",
fr: {
synonyms: [
"assaisonner",
"assaisonnez",
"assaisonné",
"assaisonnée",
"assaisonnement",
"relever",
"relevez",
"épicer",
"épicez",
],
utterances: [
"assaisonner avec du sel et du poivre",
"rectifier le goût en ajoutant des épices",
"ajouter du sel selon votre goût avant de servir",
],
},
en: {
synonyms: ["season", "seasons", "seasoned", "seasoning", "spice it up", "add seasoning"],
utterances: [
"season with salt and pepper",
"adjust the taste by adding spices",
"add salt to taste before serving",
],
},
},
{
uid: "drain",
fr: {
synonyms: [
"égoutter",
"égouttez",
"égoutté",
"égouttée",
"égouttées",
"essorer",
"essorez",
"essoré",
"essorée",
],
utterances: [
"égoutter les pâtes dans une passoire",
"verser dans une passoire pour retirer l'eau de cuisson",
"laisser l'excédent d'eau s'écouler avant de servir",
],
},
en: {
synonyms: ["drain", "drains", "drained", "draining", "strain", "strained", "straining"],
utterances: [
"drain the pasta in a colander",
"pour into a colander to remove the cooking water",
"let the excess water run off before serving",
],
},
},
{
uid: "brown",
fr: {
synonyms: [
"faire revenir",
"faites revenir",
"faire dorer",
"faites dorer",
"colorer",
"colorez",
"faire colorer",
],
utterances: [
"faire revenir les oignons dans l'huile chaude",
"faire dorer la viande sur toutes les faces",
"saisir jusqu'à ce que la surface prenne une belle couleur caramel",
],
},
en: {
// Verb forms only (not bare "brown"), same reasoning the old regex
// doc comment gave — a bare "brown" false-positives on ingredient
// descriptions like "brown sugar"/"brown rice", which never get to
// the classifier since they're not step text, but keeping the
// synonym itself anchored costs nothing and stays consistent.
synonyms: ["browned", "browning"],
utterances: [
"brown the onions in hot oil",
"brown the meat on every side",
"sear until the surface turns a deep caramel color",
],
},
},
{
uid: "rest",
fr: {
synonyms: ["reposer", "laisser reposer", "laissez reposer", "temps de repos"],
utterances: [
"laisser reposer la pâte trente minutes",
"laisser la viande se détendre hors du four avant de la découper",
"attendre quelques minutes avant de servir pour que les jus se répartissent",
],
},
en: {
// Anchored to "let ... rest"/"rest for" rather than bare "rest",
// same false-positive reasoning as `brown` above ("the rest of the").
synonyms: ["let it rest", "let them rest", "resting for", "rested for", "resting time"],
utterances: [
"let the dough rest for thirty minutes",
"let the meat relax outside the oven before carving it",
"wait a few minutes before serving so the juices redistribute",
],
},
},
{
uid: "preheat",
fr: {
synonyms: [
"préchauffer",
"préchauffez",
"préchauffé",
"préchauffée",
// A pan already described as hot ("poêle chaude") implies it's
// been preheated, without the verb itself — the classic "Dans une
// poêle chaude, faire chauffer une noix de beurre" case (both
// `preheat` and `melt` in one instruction).
"poêle chaude",
"préchauffage",
],
utterances: [
"préchauffer le four à 180 degrés",
"mettre le four à chauffer avant d'y placer le plat",
"allumer le four à l'avance pour qu'il soit à température",
// A pan gets preheated too, not just an oven — without an example
// like this, "poêle" (which also appears throughout `panFry`'s own
// training utterances) biased the classifier toward `panFry` for
// any preheating clause that happens to mention a pan, found while
// testing against the classic "Préchauffer la poêle, puis faire
// fondre le beurre" case.
"préchauffer la poêle avant d'y verser l'huile",
"faire chauffer la poêle à vide quelques minutes",
// "poêle" + "feu vif" together still read as `panFry` (the act of
// actually cooking something in it) rather than `preheat` (getting
// it hot beforehand, nothing in it yet) without an example this
// close to that exact wording — found via "mettre la poêle sur feu
// vif" (no food mentioned at all) still classifying as panFry.
"mettre la poêle vide sur feu vif avant d'ajouter quoi que ce soit",
"mettre la poêle sur feu vif",
],
},
en: {
// NOT "preheating time" — a word-prefix extension of "preheating"
// above (see `cook`'s comment for why that duplicates NER candidates).
synonyms: ["preheat", "preheats", "preheated", "preheating", "hot pan"],
utterances: [
"preheat the oven to 180 degrees",
"turn the oven on to heat up before putting the dish in",
"switch the oven on ahead of time so it's up to temperature",
"preheat the pan before adding the oil",
"heat the empty pan for a few minutes first",
],
},
},
{
uid: "bake",
fr: {
synonyms: [
"cuire au four",
"cuisson au four",
"enfourner",
"enfournez",
"au four",
"enfourné",
"enfournée",
],
utterances: [
"enfourner pendant quarante-cinq minutes",
"mettre au four jusqu'à ce que ce soit doré",
"cuire dans le four préchauffé jusqu'à ce que la surface soit ferme",
],
},
en: {
// NOT "baked in the oven" — a word-prefix extension of "baked" above
// (see `cook`'s comment for why that duplicates NER candidates).
synonyms: ["bake", "bakes", "baked", "baking", "in the oven", "oven-baked"],
utterances: [
"bake for forty-five minutes",
"put it in the oven until golden",
"cook in the preheated oven until the surface is firm",
],
},
},
{
uid: "plate",
fr: {
// NOT "dressage de l'assiette" — a word-prefix extension of
// "dressage" above (see `cook`'s comment for why that duplicates NER
// candidates).
synonyms: ["dresser", "dressez", "dressage", "disposer dans l'assiette"],
utterances: [
"dresser harmonieusement dans les assiettes",
"disposer joliment sur l'assiette avant de servir",
"présenter avec soin au centre de l'assiette",
],
},
en: {
// NOT "plate up"/"plated nicely" — both are word-prefix extensions of
// "plate"/"plated" above (see `cook`'s comment for why that
// duplicates NER candidates).
synonyms: ["plate", "plates", "plated", "plating"],
utterances: [
"plate it up nicely",
"arrange it neatly on the plate before serving",
"present it carefully in the center of the plate",
],
},
},
{
uid: "coat",
fr: {
synonyms: [
"napper",
"nappez",
"nappé",
"nappée",
"nappées",
"nappage",
"enrober",
"enrobez",
"enrobé",
"enrobée",
"enrobées",
],
utterances: [
"napper le gâteau de chocolat fondu",
"recouvrir uniformément d'une fine couche de sauce",
"verser la sauce par-dessus pour bien enrober",
],
},
en: {
// NOT "coat evenly" — a word-prefix extension of "coat" above (see
// `cook`'s comment for why that duplicates NER candidates).
synonyms: ["coat", "coats", "coated", "coating", "dredge", "dredged", "dredging"],
utterances: [
"coat the cake with melted chocolate",
"cover evenly with a thin layer of sauce",
"pour the sauce over it so it's well covered",
],
},
},
];

4
apps/api/src/scripts/backfill-tech-steps.ts
View file

@ -6,8 +6,8 @@ import { renumberStepTechSteps } from "../modules/recipe/recipe-tech-step-correc
/**
* Recomputes every existing `Step`'s `"auto"`-sourced `StepTechStep`
* entries against the *current* classifier
* (`tech-step-matcher.ts`/`tech-step-training-data.ts`), the same way
* `updateRecipe` does when a user resaves a recipe through the UI —
* (`tech-step-matcher.ts`, delegating to `services/tech-step-intent-service`),
* the same way `updateRecipe` does when a user resaves a recipe through the UI —
* always `"fr"` (`DEFAULT_TECH_STEP_LOCALE` in `recipe.service.ts`; there's
* no persisted per-recipe locale to recover for a step that already
* exists, so this matches real resave behavior exactly rather than

9
apps/api/src/scripts/list-pending-training-suggestions.ts
View file

@ -6,14 +6,15 @@ import { prisma } from "../db/prisma.js";
* comment) — generated by `services/tech-step-llm-worker`'s scheduled
* jobs, from either a user correction or the worker's own low-confidence
* audit (`sourceType`). What a maintainer reads *before* hand-editing
* `tech-step-training-data.ts` and running `retrain-tech-steps.ts` — this
* script never writes anything, purely a read-only report to stdout:
* `services/tech-step-intent-service/intent_service/training_data.py` and
* running `retrain-tech-steps.ts` — this script never writes anything,
* purely a read-only report to stdout:
*
* pnpm --filter api exec tsx src/scripts/list-pending-training-suggestions.ts
*
* Grouped by technique key so every suggestion for the same entry in
* `TECH_STEP_TRAINING_DATA` is read together, matching how that file
* itself is organized (one block per technique).
* `training_data.py`'s `TECH_STEP_TRAINING_DATA` is read together, matching
* how that file itself is organized (one block per technique).
*/
async function listPendingTrainingSuggestions(): Promise<void> {
const suggestions = await prisma.techStepTrainingSuggestion.findMany({

11
apps/api/src/scripts/retrain-tech-steps.ts
View file

@ -28,10 +28,15 @@ function parseSuggestionIds(flag: "applied" | "rejected"): number[] {
* Maintainer workflow closing the loop on a training-corpus change (see
* this feature's plan document):
*
* 1. A maintainer has already hand-edited `tech-step-training-data.ts`
* (informed by `list-pending-training-suggestions.ts`'s report), and
* 1. A maintainer has already hand-edited
* `services/tech-step-intent-service/intent_service/training_data.py`
* (informed by `list-pending-training-suggestions.ts`'s report),
* decided which `TechStepTrainingSuggestion` ids they incorporated
* (`--applied=`) or explicitly discarded (`--rejected=`).
* (`--applied=`) or explicitly discarded (`--rejected=`), **and
* restarted `tech-step-intent-service`** so it retrains from the
* edited corpus — that service only ever trains once, at its own
* startup (see its README), so this script's eval gate below is
* meaningless against a service still running the old corpus.
* 2. This script re-runs the F1 regression gate
* ({@link runTechStepEvalSuite} against {@link MIN_OVERALL_F1}) —
* refuses to backfill at all if the edited corpus scores worse than

46
apps/api/test-support/mocha-root-hooks.ts
View file

@ -5,27 +5,23 @@ import { resetDatabase } from "./reset-db.js";
* Mocha root hook plugin (see `.mocharc.json`'s `require`) — runs once
* before every test file's own suites, regardless of load order.
*
* Warms up `techStepClassifier` here, with its own generous timeout,
* Warms up `techStepClassifier` here — resolving the `TechStep.key -> id`
* lookup from the DB (see `TechStepClassifierService._loadTechStepIds`) —
* instead of leaving it to happen lazily on whichever test file Mocha
* happens to load first. In production this one-time cost (a `POST
* /v1/train` round-trip per locale to `services/tech-step-intent-service`,
* training a real `textcat` on the full `TECH_STEP_TRAINING_DATA` corpus)
* is paid by `server.ts`'s own `techStepClassifier.warmUp()` before the
* server ever accepts traffic — but this test suite builds its `app`
* directly via `createApp()` (see e.g. `tech-step-worker.routes.test.ts`),
* never running `server.ts` at all. Without this hook, that cost instead
* landed inside whichever test's own call happened to trigger
* `_ensureTrained()` first — found the hard way in CI, where training the
* full corpus took longer than a single test's default 10s timeout
* (`.mocharc.json`) and failed an otherwise-unrelated test purely because
* Mocha loaded its file first alphabetically.
* happens to load first, same as `server.ts` does before the real server
* ever accepts traffic. Fast by itself (one DB query, one HTTP call to
* `services/tech-step-intent-service`): that service now trains itself
* entirely at its own process startup (see its own README), so unlike
* before this migration, nothing here waits on a slow training pass — CI's
* own "wait for `/health`" step (`.github/workflows/ci.yml`) is what
* ensures that service is already fully trained before `pnpm --filter api
* test` even starts.
*
* `resetDatabase()` runs first, deliberately: `_train()`
* (`tech-step-matcher.ts`) resolves `TechStep.key -> id` from the database
* alongside training, and a freshly-migrated (never-seeded) test database
* has no `TechStep` rows yet — every per-test `beforeEach` in this suite
* already calls `resetDatabase()` again before its own test, which is a
* no-op duplication of effort but not a correctness problem: `TRUNCATE ...
* `resetDatabase()` runs first, deliberately: id resolution needs
* `TechStep` rows, and a freshly-migrated (never-seeded) test database has
* none yet. Every per-test `beforeEach` in this suite already calls
* `resetDatabase()` again before its own test, which is a no-op
* duplication of effort but not a correctness problem: `TRUNCATE ...
* RESTART IDENTITY` plus deterministic re-seeding (`seedReferenceData`)
* assigns the exact same ids every time, so the `uid -> id` map memoized
* here from this first reset stays valid for every reset after it.
@ -33,13 +29,11 @@ import { resetDatabase } from "./reset-db.js";
export const mochaHooks = {
// biome-ignore lint/suspicious/noExplicitAny: Mocha's root hook `this` (a Context with `.timeout()`) isn't typed without @types/mocha (not a dependency here) — same untyped-`this` shape already used in tech-step-worker.routes.test.ts.
async beforeAll(this: any): Promise<void> {
// Generous on purpose: training both locales' `textcat` on the full
// corpus takes on the order of a couple of minutes combined (see
// `_TRAINING_ITERATIONS` in `services/tech-step-intent-service`'s
// `locale_pipeline.py`) — comfortably under 10 minutes even on a
// slower/contended CI runner, but nowhere near Mocha's normal 10s
// per-test default (`.mocharc.json`).
this.timeout(600000);
// A little more generous than Mocha's normal 10s per-test default
// (`.mocharc.json`) purely for a slower/contended CI runner's first
// network round-trip to `services/tech-step-intent-service` — not
// because anything here waits on training anymore.
this.timeout(30000);
await resetDatabase();
await techStepClassifier.warmUp();
},

5
apps/api/test/recipe-matching/recipe-translation.test.ts
View file

@ -38,8 +38,9 @@ describe("recipe-translation", () => {
// `translateRecipeSteps` now goes through `techStepClassifier` (a
// trained model, not a pure regex test against a caller-supplied
// mapping list — see `tech-step-matcher.ts`), so these tests exercise
// the real training corpus (`tech-step-training-data.ts`) against a real
// `TechStep` catalog rather than synthetic fixtures — same posture
// the real training corpus (`services/tech-step-intent-service`'s
// `training_data.py`) against a real `TechStep` catalog rather than
// synthetic fixtures — same posture
// `tech-step-matcher.test.ts`'s own `techStepClassifier` describe block
// takes, for the same reason.
describe("translateRecipeSteps", () => {

19
apps/api/test/recipe-matching/tech-step-matcher.test.ts
View file

@ -118,15 +118,16 @@ describe("tech-step-matcher", () => {
// `techStepClassifier` is the one shared singleton (see
// tech-step-matcher.ts's own doc comment on why) — these tests
// exercise it against the real training corpus
// (`tech-step-training-data.ts`) and the real seeded `TechStep`
// catalog, rather than synthetic injectable fixtures the old
// regex-based `matchTechStepSpans(description, mappings)` allowed.
// Training now round-trips over HTTP to a real, locally running
// `services/tech-step-intent-service` (see that service's own README
// and `apps/api/.env.test`) — the very first call in the whole suite
// pays for that plus the service's own spaCy pipeline setup (subsequent
// calls reuse the already-trained pipeline and are fast) — comfortably
// inside this suite's default 10s timeout (.mocharc.json).
// (`services/tech-step-intent-service`'s `training_data.py`) and the
// real seeded `TechStep` catalog, rather than synthetic injectable
// fixtures the old regex-based `matchTechStepSpans(description,
// mappings)` allowed. Every call round-trips over HTTP to a real,
// locally running `services/tech-step-intent-service` (see that
// service's own README and `apps/api/.env.test`) — that service trains
// itself once at its own startup (`test-support/mocha-root-hooks.ts`'s
// root hook doesn't wait on it, CI's own "wait for /health" step
// already does), so calls here are just a normal HTTP round-trip,
// comfortably inside this suite's default 10s timeout (.mocharc.json).
let simmerId: number;
let cookId: number;
let bakeId: number;

2
apps/api/test/recipe/recipe-tech-step-correction.test.ts
View file

@ -79,7 +79,7 @@ describe("Recipe tech-step corrections", () => {
const { agent, profileId } = await signup();
// "Faire mijoter la sauce." names no technique the classifier itself
// registers a bare-word anchor for at this exact span in isolation
// (see tech-step-training-data.ts) — irrelevant here either way,
// (see services/tech-step-intent-service's training_data.py) — irrelevant here either way,
// since this test's whole point is the *manual* addition, not
// whatever the classifier does or doesn't auto-detect for it.
const { recipeId, stepId } = await createPublicRecipeWithStep(profileId);

50
apps/web/src/locales/fr/translation.json
View file

@ -425,7 +425,55 @@
"preheat": "Préchauffer",
"bake": "Cuire au four",
"plate": "Dresser",
"coat": "Napper"
"coat": "Napper",
"baste": "Arroser",
"appertize": "Appertiser",
"whiskPale": "Blanchir (jaunes d'œufs)",
"goldenBrown": "Blondir",
"braise": "Braiser",
"truss": "Brider",
"caramelize": "Caraméliser",
"score": "Cerner",
"lineMold": "Chemiser",
"clarify": "Clarifier",
"compote": "Compoter",
"concasse": "Concasser",
"confit": "Confire",
"julienne": "Couper en julienne",
"brunoise": "Couper en brunoise",
"mirepoix": "Couper en mirepoix",
"paysanne": "Couper en paysanne",
"blindBake": "Cuire à blanc",
"bainMarie": "Cuire au bain-marie",
"smother": "Cuire à l'étouffée",
"decant": "Décanter",
"dilute": "Délayer",
"punchDown": "Dégazer",
"disgorge": "Dégorger",
"loosen": "Détendre",
"shellEgg": "Écaler",
"scald": "Échauder",
"pod": "Écosser",
"emulsify": "Émulsionner",
"hollowOut": "Évider",
"shock": "Frapper",
"setGel": "Gélifier",
"glaze": "Glacer",
"thicken": "Lier",
"filet": "Lever les filets",
"proof": "Laisser pousser",
"peelBlanch": "Monder",
"whipUp": "Monter",
"moisten": "Mouiller",
"pasteurize": "Pasteuriser",
"poach": "Pocher",
"reduce": "Réduire",
"rubIn": "Sabler",
"dustWithFlour": "Singer",
"sweat": "Suer",
"sift": "Tamiser",
"toast": "Torréfier",
"zest": "Zester"
},
"allergens": {
"gluten": "Gluten",

16
docker-compose.yml
View file

@ -88,10 +88,20 @@ services:
"-c",
"import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=2)",
]
interval: 10s
interval: 15s
timeout: 3s
retries: 10
start_period: 15s
retries: 5
# This service trains itself from scratch on every start (no model
# ever persisted to disk, see its own README) — `/health` only
# returns 200 once that's done, not just once the base spaCy models
# are loaded. Measured at ~200s per locale (~400s for fr+en combined)
# against the current ~74-technique corpus
# (`intent_service/training_data.py`) — `start_period` generous
# enough that failing checks during that whole window never count
# against `retries` (which would otherwise flip this container to
# "unhealthy" mid-training, blocking `app`'s own
# `depends_on: condition: service_healthy` indefinitely).
start_period: 600s
# Deliberately its own image, not built into `app`'s (see
# services/tech-step-llm-worker/Dockerfile's own doc comment) — a

105
services/tech-step-intent-service/README.md
View file

@ -19,18 +19,29 @@ ce service en l'état) de pouvoir aussi absorber ce que fait aujourd'hui
passer (les modèles `md`, avec vecteurs de mots, sont conservés dans ce but,
même si rien ici ne s'en sert encore).
## Pourquoi ce service ne possède aucune donnée d'entraînement
## Ce service est entièrement autonome
Contrairement à un service NLP habituel, **ce service ne connaît aucune
technique par lui-même** — `apps/api` reste l'unique source de vérité du
corpus (`TECH_STEP_TRAINING_DATA`,
`apps/api/src/lib/recipe-matching/tech-step-training-data.ts`, revu par PR
comme le reste du code). Il pousse l'intégralité du corpus ici via
`POST /v1/train` à chaque warm-up serveur (`TechStepClassifierService._train`)
— ce service (re)construit alors son pipeline en mémoire, sans jamais rien
persister sur disque. Le workflow mainteneur existant
(`apps/api/src/scripts/retrain-tech-steps.ts`, édition manuelle du corpus)
n'a pas changé.
Contrairement à sa toute première version, **ce service possède désormais
son propre corpus** — `intent_service/training_data.py`
(`TECH_STEP_TRAINING_DATA`), revu par PR comme le reste du code. Il
s'entraîne lui-même une seule fois, à son propre démarrage
(`PipelineRegistry.initialize()`, appelé par `main.py`'s `lifespan`), et ne
persiste jamais rien sur disque — un redémarrage du process réentraîne
toujours from scratch depuis ce fichier. `apps/api` ne connaît plus aucune
technique ni aucun synonyme : il n'appelle plus que `POST /v1/process` (plus
de `POST /v1/train`, supprimé).
Workflow mainteneur pour changer le corpus :
1. Éditer `intent_service/training_data.py` à la main (informé par le
rapport de `apps/api/src/scripts/list-pending-training-suggestions.ts`).
2. **Redémarrer ce service** (`docker compose restart tech-step-intent-service`,
ou simplement redéployer) — le nouveau corpus n'a d'effet qu'une fois
réentraîné au démarrage, contrairement à l'ancienne version qui pouvait
être réentraînée à chaud via `POST /v1/train`.
3. Depuis `apps/api`, lancer `pnpm --filter api exec tsx
src/scripts/retrain-tech-steps.ts` — vérifie le F1 contre
`TECH_STEP_EVAL_DATASET` avant de backfiller les recettes existantes.
## Pourquoi ce service vit hors du workspace pnpm
@ -39,31 +50,48 @@ n'a rien à faire dans `pnpm-workspace.yaml` (qui ne couvre que
`apps/*`/`packages/*`), et ses dépendances (spaCy, ses modèles) ne doivent
jamais se retrouver dans l'image `apps/api`. **Aucun accès direct à
Postgres** non plus — la résolution `TechStep.key -> id` reste entièrement
côté `apps/api` (`TechStepClassifierService._train`), ce service ne
manipule que des `uid` (chaînes opaques) tout du long.
côté `apps/api` (`TechStepClassifierService`), ce service ne manipule que
des `uid` (chaînes opaques) tout du long.
## Contrat HTTP
Voir `intent_service/schemas.py` pour le détail exact. En résumé :
- `GET /health` — sans authentification, `200` une fois les modèles spaCy
de base chargés (pas de lazy-load, voir `intent_service/main.py`).
- `POST /v1/train` — `{ locale, entries: [{ uid, synonyms, utterances }] }`
→ reconstruit le pipeline de `locale` à neuf.
- `GET /health` — sans authentification, `200` une fois ce service
entièrement prêt : modèles spaCy de base chargés **et** les deux locales
entraînées (pas de lazy-load, voir `intent_service/main.py`) — voir
"Temps de démarrage" plus bas pour ce que ça implique en pratique.
- `POST /v1/process` — `{ locale, text }` → `{ entities: [{ uid, start, end }], intent, score }`.
`/v1/train` et `/v1/process` exigent le header `X-Intent-Service-Secret`
(voir `intent_service/security.py`), qui doit matcher `INTENT_SERVICE_SECRET`
`/v1/process` exige le header `X-Intent-Service-Secret` (voir
`intent_service/security.py`), qui doit matcher `INTENT_SERVICE_SECRET`
côté `apps/api`.
## Temps de démarrage
**Ce service met plusieurs minutes à devenir `healthy`** — contrairement à
node-nlp (entraînement quasi instantané), entraîner le `textcat` sur le
corpus réel (~74 techniques) prend de l'ordre de 200 secondes par locale
(mesuré localement, sans GPU), donc environ 400 secondes (~7 minutes) pour
`fr`+`en` combinés à chaque démarrage du process. `docker-compose.yml` et
`.github/workflows/ci.yml` ont un `start_period`/timeout d'attente
généreux pour ça — voir leurs propres commentaires. C'est un compromis
assumé, pas un défaut de configuration à corriger : moins d'itérations
entraîne plus vite mais laisse des verdicts corrects sous
`CONFIDENCE_THRESHOLD` (voir le commentaire de cette constante,
`apps/api/src/lib/recipe-matching/tech-step-matcher.ts`, et celui de
`_TRAINING_ITERATIONS`/`_TRAINING_BATCH_SIZE` dans `locale_pipeline.py`
pour le détail du compromis).
## Logs
`intent_service/logging_config.py` branche un format JSON structuré (une
ligne par évènement — `timestamp`/`level`/`message` + champs métier fusionnés
— même convention que `LoggerService` côté `apps/api`) sur toute la
journalisation de ce service, niveau `LOG_LEVEL` (`INFO` par défaut, voir
`.env.example`). `routes/process.py` et `routes/train.py` journalisent
chaque appel avec son input et son output complets :
`.env.example`). `routes/process.py` journalise chaque appel avec son input
et son output complets, `pipeline_registry.py` journalise le déroulement de
l'entraînement au démarrage :
```json
{"timestamp": "...", "level": "info", "message": "tech-step NLP process", "locale": "fr", "text": "faire fondre le beurre", "entities": [{"uid": "melt", "start": 6, "end": 13}], "intent": "melt", "score": 0.93}
@ -95,7 +123,8 @@ uv run uvicorn intent_service.main:app --reload --port 8000
aux côtés de `postgres`/`app`/`tech-step-llm-worker` — **pas optionnel**,
contrairement au worker LLM : sans lui, `apps/api` ne peut plus détecter
aucune technique de cuisine. `app` attend qu'il soit `healthy`
(`depends_on: condition: service_healthy`) avant de démarrer.
(`depends_on: condition: service_healthy`) avant de démarrer — voir "Temps
de démarrage" ci-dessus pour combien de temps ça prend en pratique.
## Testing
@ -107,6 +136,10 @@ uv run pytest
exacts et d'insensibilité accents/casse de
`apps/api/test/recipe-matching/tech-step-matcher.test.ts` — le point de
fidélité le plus critique de ce service (voir le plan de migration).
`tests/conftest.py`'s fixture `client` (scope "session") ne s'entraîne
qu'une seule fois pour toute la suite — c'est *le vrai corpus complet*,
pas un jeu jouet, donc la première utilisation de cette fixture prend le
même temps qu'un vrai démarrage (voir "Temps de démarrage" ci-dessus).
Aucun test ici ne dépend d'une vraie base Postgres ni d'`apps/api` en
service — à l'inverse, la suite Mocha d'`apps/api`
@ -114,24 +147,24 @@ service — à l'inverse, la suite Mocha d'`apps/api`
vraie instance de ce service tournant (voir `apps/api/.env.test`), conforme
à la convention du repo de ne jamais mocker un service interne.
## Limitations connues (première version)
## Limitations connues
- **`/v1/train` prend de l'ordre de la minute par locale** (~110s mesuré en
CI avec `_TRAINING_ITERATIONS`/`_TRAINING_BATCH_SIZE` actuels, voir
`locale_pipeline.py`) — `apps/api` l'appelle deux fois au warm-up
(`fr`/`en`), donc un redémarrage prend quelques minutes avant qu'une
recette puisse voir ses techniques détectées. Contrairement à node-nlp
(entraînement quasi instantané), c'est un vrai compromis assumé : moins
d'itérations entraînait plus vite mais laissait des verdicts corrects
sous `CONFIDENCE_THRESHOLD` (voir le commentaire de cette constante,
`apps/api/src/lib/recipe-matching/tech-step-matcher.ts`).
- **Démarrage lent** (~7 minutes) — voir "Temps de démarrage" ci-dessus.
Une optimisation possible non explorée : parallélisation de
l'entraînement `fr`/`en` (actuellement séquentiel,
`PipelineRegistry.initialize`).
- **`CONFIDENCE_THRESHOLD` côté `apps/api` est un placeholder** depuis
l'élargissement du corpus à ~74 techniques (calibré à la main, pas via
une vraie repasse de `calibrate-tech-step-threshold.ts` contre
`TECH_STEP_EVAL_DATASET` — voir le commentaire de cette constante).
- **Textcat bag-of-words** (`spacy.TextCatBOW.v3`) — suffisant pour le
corpus actuel une fois correctement entraîné, mais n'exploite pas les
vecteurs de mots des modèles `md` chargés. Migrable vers une architecture
tok2vec/similarité sans changer le contrat HTTP, si le F1 mesuré par
`apps/api/src/scripts/calibrate-tech-step-threshold.ts` le justifie un
jour.
- **Reconstruit tout le pipeline à chaque `/v1/train`** (pas de fusion
incrémentale) — un choix délibéré (voir `LocalePipeline.train`), pas une
limitation à lever : `TECH_STEP_TRAINING_DATA` doit toujours rester
l'unique source de vérité, jamais un état local qui dérive.
- **Reconstruit tout le pipeline à chaque démarrage** (pas de persistance,
pas de fusion incrémentale) — un choix délibéré (voir
`LocalePipeline.train`), pas une limitation à lever : `training_data.py`
doit toujours rester l'unique source de vérité, jamais un état sur disque
qui pourrait dériver.

5
services/tech-step-intent-service/intent_service/config.py
View file

@ -42,8 +42,9 @@ class Settings(BaseSettings):
# Niveau du logging structuré (`logging_config.py`) — voir ce module pour
# le format. `INFO` par défaut : c'est à ce niveau que `routes/process.py`
# et `routes/train.py` journalisent chaque input/output du pipeline NLP,
# pour qu'un déploiement par défaut les voie sans configuration
# journalise chaque input/output du pipeline NLP, et que
# `pipeline_registry.py` journalise l'entraînement au démarrage, pour
# qu'un déploiement par défaut les voie sans configuration
# supplémentaire (`docker logs`/Portainer).
log_level: str = "INFO"

55
services/tech-step-intent-service/intent_service/locale_pipeline.py
View file

@ -2,8 +2,9 @@
`node-nlp`'s `NlpManager` faisait pour cette locale dans
`TechStepClassifierService` (`apps/api/src/lib/recipe-matching/tech-step-matcher.ts`) :
NER par entités enum (ici un `PhraseMatcher`) + classification d'intention
(ici un `textcat`), les deux entraînés à partir du même corpus
(`TECH_STEP_TRAINING_DATA`, poussé par `apps/api` via `POST /v1/train`).
(ici un `textcat`), les deux entraînés à partir du corpus possédé par ce
service lui-même (`training_data.TECH_STEP_TRAINING_DATA` — plus poussé par
`apps/api` via HTTP, voir `pipeline_registry.py`).
Le modèle de base spaCy (tokenizer + vecteurs + le composant
`diacritics_normalizer` défini plus bas) est chargé une seule fois
@ -12,9 +13,8 @@ paresseusement au premier `train()`, pour que `GET /health` ne devienne
`200` qu'une fois ce coût payé) puis réutilisé à chaque `train()` : seul le
`textcat` (retiré puis rajouté à neuf) et le `PhraseMatcher` (remplacé) sont
reconstruits à chaque appel, jamais le tokenizer/les vecteurs. Rien n'est
jamais persisté sur disque — même posture que `autoSave`/`autoLoad: false`
sur l'ancien `NlpManager` : `TECH_STEP_TRAINING_DATA` (côté `apps/api`) reste
l'unique source de vérité.
jamais persisté sur disque — `training_data.py` reste l'unique source de
vérité, reconstruite en mémoire depuis zéro à chaque démarrage du process.
"""
from __future__ import annotations
@ -50,21 +50,32 @@ _EXCLUDED_COMPONENTS = ["parser", "ner", "tagger", "morphologizer", "attribute_r
_TEXTCAT_PIPE_NAME = "textcat"
# Nombre d'itérations d'entraînement du textcat et taille de minibatch —
# calibrés empiriquement contre le corpus réel (`TECH_STEP_TRAINING_DATA`,
# ~26 techniques/locale), pas seulement contre les petits corpus jouets des
# tests de ce fichier. Une première valeur plus basse (30 itérations, lot de
# 8) convergeait mal sur le vrai corpus : des clauses correctement
# classifiées mais sans ancre NER (le cas motivant tout ce pipeline, voir
# `TechStepClassifierService`'s doc comment côté apps/api) scoraient à peine
# 0.5-0.7, et des clauses *avec* ancre à peine 0.2-0.3 — bien en dessous de
# `CONFIDENCE_THRESHOLD` (`tech-step-matcher.ts`), un CI réel l'a confirmé
# avant que ces valeurs ne soient relevées. `150`/`16` entraîne en ~110s par
# locale sur un runner GitHub Actions standard (donc ~220s pour fr+en
# combinés au warm-up — voir `server.ts`'s propre commentaire sur le retry
# côté apps/api) et pousse les mêmes scores nettement au-dessus du seuil
# (melt ~0.95, preheat ~0.90, bake ~0.51) sans dégrader le rejet du bruit
# (texte anglais via le classifieur français reste ~0.05, inchangé).
_TRAINING_ITERATIONS = 150
# calibrés empiriquement contre le corpus réel (`training_data.py`), pas
# seulement contre les petits corpus jouets des tests de ce fichier. Trop
# peu d'itérations laisse des clauses correctement classifiées (bon argmax)
# mais avec une confiance dérisoire (`0.02`-`0.08` observé à 5-15
# itérations) — bien en dessous de tout seuil raisonnable pour
# `CONFIDENCE_THRESHOLD` (`tech-step-matcher.ts`).
#
# `150` convenait au corpus original (~26 techniques) mais ne passe plus à
# l'échelle une fois le corpus élargi à ~74 : le temps d'entraînement croît
# avec le nombre de classes autant qu'avec les itérations (mesuré :
# ~150s pour seulement 30 itérations sur 74 classes, contre ~110s pour 150
# itérations sur 26 classes) — `150` sur 74 classes dépassait 17 minutes
# rien que pour une locale, constaté en CI. `40` est le meilleur compromis
# trouvé empiriquement sur ce corpus élargi : ~200s par locale (~400s pour
# fr+en combinés au démarrage), avec des scores exploitables sur tous les
# cas testés à la main (melt ~0.89, preheat ~0.76, compote ~0.76, zest
# ~0.64, julienne ~0.56, cook/bake ~0.33, simmer ~0.25 — le plus faible
# observé, toujours correct en argmax et de toute façon ancré par NER) et
# un bruit hors-vocabulaire qui reste négligeable (anglais via le
# classifieur français : `~0.02`). Une vraie repasse de
# `calibrate-tech-step-threshold.ts` contre `TECH_STEP_EVAL_DATASET` reste
# nécessaire pour confirmer/affiner ces deux valeurs (voir
# `CONFIDENCE_THRESHOLD`'s propre commentaire, `tech-step-matcher.ts`) — ce
# qui suit est une mesure manuelle ponctuelle, pas un remplacement de cette
# calibration.
_TRAINING_ITERATIONS = 40
_TRAINING_BATCH_SIZE = 16
# Abaissé de `0.2` avec le reste de cette recalibration — `0.1` régularise
# encore contre la petite taille du corpus par technique tout en laissant
@ -107,8 +118,8 @@ class _DiacriticsNormalizer:
@dataclass
class TrainEntry:
"""Une technique à entraîner pour une locale — miroir de
`TrainEntryPayload` (`schemas.py`)."""
"""Une technique à entraîner pour une locale — construit par
`PipelineRegistry.initialize()` depuis `training_data.entries_for_locale`."""
uid: str
synonyms: list[str] = field(default_factory=list)

5
services/tech-step-intent-service/intent_service/logging_config.py
View file

@ -6,7 +6,7 @@ Portainer ou un agrégateur de logs — cohérent avec le reste du repo plutôt
qu'un format propre à ce seul service.
Configuré une fois au démarrage (`main.py`) plutôt que par un `print()` ad
hoc dans chaque route — `routes/process.py`/`routes/train.py` appellent
hoc dans chaque route — `routes/process.py`/`pipeline_registry.py` appellent
`logging.getLogger(__name__)` normalement, ce module ne fait que brancher le
formateur JSON sur la racine du logging Python.
"""
@ -69,7 +69,8 @@ def configure_logging(level: str) -> None:
# spaCy/thinc journalisent leur propre chatter interne ("Created
# vocabulary", "Finished initializing nlp object"...) sur le logger
# `"spacy"`, qui propage jusqu'à la racine et se retrouverait donc
# mélangé aux lignes input/output de `routes/process.py`/`routes/train.py`
# mélangé aux lignes input/output de `routes/process.py`/l'entraînement
# journalisé par `pipeline_registry.py`
# — ce sont ces dernières que ce service existe pour rendre visibles, pas
# le détail interne de spaCy. `WARNING` laisse quand même remonter un
# vrai problème (dépréciation, échec partiel) sans le bruit `INFO`.

23
services/tech-step-intent-service/intent_service/main.py
View file

@ -1,11 +1,15 @@
"""Point d'entrée FastAPI — `uv run uvicorn intent_service.main:app` (voir
le Dockerfile et le README de ce service).
Le chargement des modèles spaCy de base (`PipelineRegistry.preload_all`) se
fait dans le handler `lifespan` ci-dessous, *avant* qu'uvicorn n'accepte de
requêtes — `GET /health` ne répond donc `200` qu'une fois ce coût payé,
jamais pendant un chargement encore en cours (uvicorn ne sert aucune requête
tant que le `lifespan` de démarrage n'est pas terminé).
Le chargement des modèles spaCy de base *et* l'entraînement de chaque
locale (`PipelineRegistry.initialize`) se font dans le handler `lifespan`
ci-dessous, *avant* qu'uvicorn n'accepte de requêtes — `GET /health` ne
répond donc `200` qu'une fois ce coût payé (chargement + entraînement),
jamais pendant qu'il est encore en cours (uvicorn ne sert aucune requête
tant que le `lifespan` de démarrage n'est pas terminé). Ce service est
autonome : `training_data.TECH_STEP_TRAINING_DATA` vit dans ce module,
`apps/api` ne pousse plus rien via HTTP (voir `pipeline_registry.py` pour
le détail de ce que ça change par rapport à la version précédente).
"""
from contextlib import asynccontextmanager
@ -15,22 +19,21 @@ from fastapi import FastAPI
from .config import settings
from .logging_config import configure_logging
from .pipeline_registry import registry
from .routes import health, process, train
from .routes import health, process
# Avant tout le reste : `routes/process.py`/`routes/train.py` journalisent
# dès la première requête, `preload_all()` ci-dessous journalise aussi (voir
# Avant tout le reste : `routes/process.py` journalise dès la première
# requête, `initialize()` ci-dessous journalise aussi (voir
# `pipeline_registry.py`) — le formateur JSON doit déjà être en place.
configure_logging(settings.log_level)
@asynccontextmanager
async def lifespan(app: FastAPI):
registry.preload_all()
registry.initialize()
yield
app = FastAPI(title="tech-step-intent-service", lifespan=lifespan)
app.include_router(health.router)
app.include_router(train.router)
app.include_router(process.router)

58
services/tech-step-intent-service/intent_service/pipeline_registry.py
View file

@ -3,16 +3,17 @@ partagé du process (une instance vit pour toute la durée de vie d'`uvicorn`,
montée sur `app.state`, voir `main.py`).
Volontairement une classe "registre" séparée de `LocalePipeline` lui-même :
`LocalePipeline` ne connaît qu'une seule locale, ce module route
`train`/`process` vers la bonne instance selon le `locale` reçu dans la
requête — même séparation de responsabilité que `TechStepClassifierService`
(une seule instance, un seul `NlpManager` multi-langues) avait implicitement
via node-nlp, explicitée ici puisque spaCy charge un modèle par langue.
`LocalePipeline` ne connaît qu'une seule locale, ce module route `process`
vers la bonne instance selon le `locale` reçu dans la requête — même
séparation de responsabilité que `TechStepClassifierService` (une seule
instance, un seul `NlpManager` multi-langues) avait implicitement via
node-nlp, explicitée ici puisque spaCy charge un modèle par langue.
"""
import logging
from .locale_pipeline import SUPPORTED_LOCALES, LocalePipeline, ProcessResult, TrainEntry, UnsupportedLocaleError
from .locale_pipeline import SUPPORTED_LOCALES, LocalePipeline, ProcessResult, TrainEntry
from .training_data import entries_for_locale
logger = logging.getLogger(__name__)
@ -23,21 +24,38 @@ class PipelineRegistry:
locale: LocalePipeline(locale) for locale in SUPPORTED_LOCALES
}
def preload_all(self) -> None:
"""Charge le modèle spaCy de base de chaque locale connue — appelé
une fois au démarrage du process (`main.py`), pas paresseusement au
premier appel, pour que `GET /health` ne réponde `200` qu'une fois
ce coût payé (voir `LocalePipeline.preload`)."""
logger.info("tech-step NLP preloading base pipelines", extra={"locales": list(self._pipelines)})
for pipeline in self._pipelines.values():
pipeline.preload()
logger.info("tech-step NLP base pipelines ready", extra={"locales": list(self._pipelines)})
def initialize(self) -> None:
"""Charge le modèle spaCy de base *et* entraîne chaque locale connue
depuis `training_data.TECH_STEP_TRAINING_DATA` — appelé une fois au
démarrage du process (`main.py`'s `lifespan`), avant que `uvicorn`
n'accepte de requêtes.
def train(self, locale: str, entries: list[TrainEntry]) -> tuple[int, int, int]:
pipeline = self._pipelines.get(locale)
if pipeline is None:
raise UnsupportedLocaleError(f"Unsupported locale: {locale!r}")
return pipeline.train(entries)
Contrairement à la version précédente de ce service (où `apps/api`
poussait le corpus via `POST /v1/train` à son propre warm-up), ce
service est maintenant entièrement autonome : `apps/api` ne connaît
plus aucune technique, seulement le résultat de
`POST /v1/process`. `GET /health` ne répond `200` qu'une fois cette
méthode terminée (chargement *et* entraînement) — pas seulement le
chargement — pour que `docker-compose.yml`'s `depends_on: ...
condition: service_healthy` (et la boucle d'attente équivalente en
CI) ne laisse jamais `apps/api` démarrer face à un service qui
répondrait mais ne saurait encore rien détecter.
"""
logger.info("tech-step NLP initializing pipelines", extra={"locales": list(self._pipelines)})
for locale, pipeline in self._pipelines.items():
pipeline.preload()
entries = [TrainEntry(**entry) for entry in entries_for_locale(locale)]
label_count, utterance_count, synonym_count = pipeline.train(entries)
logger.info(
"tech-step NLP pipeline trained",
extra={
"locale": locale,
"labelCount": label_count,
"utteranceCount": utterance_count,
"synonymCount": synonym_count,
},
)
logger.info("tech-step NLP pipelines ready", extra={"locales": list(self._pipelines)})
def process(self, locale: str, text: str) -> ProcessResult:
pipeline = self._pipelines.get(locale)

58
services/tech-step-intent-service/intent_service/routes/train.py
View file

@ -1,58 +0,0 @@
"""`POST /v1/train` — appelé par `apps/api` (`IntentServiceClient.train`,
`TechStepClassifierService._train`) une fois par locale à chaque warm-up
serveur, avec l'intégralité de `TECH_STEP_TRAINING_DATA` filtrée pour cette
locale. Voir `LocalePipeline.train` pour ce que "reconstruit à neuf" signifie
concrètement.
"""
import logging
from fastapi import APIRouter, Depends, HTTPException, status
from ..locale_pipeline import TrainEntry, UnsupportedLocaleError
from ..pipeline_registry import registry
from ..schemas import TrainRequest, TrainResponse
from ..security import require_valid_secret
logger = logging.getLogger(__name__)
router = APIRouter(dependencies=[Depends(require_valid_secret)])
@router.post("/v1/train", response_model=TrainResponse)
def train(request: TrainRequest) -> TrainResponse:
entries = [
TrainEntry(uid=entry.uid, synonyms=entry.synonyms, utterances=entry.utterances)
for entry in request.entries
]
logger.info(
"tech-step NLP train starting",
extra={"locale": request.locale, "uids": [entry.uid for entry in entries]},
)
try:
label_count, utterance_count, synonym_count = registry.train(request.locale, entries)
except UnsupportedLocaleError as err:
logger.warning("tech-step NLP train rejected", extra={"locale": request.locale, "error": str(err)})
# 422, pas 500 : une locale non supportée dans une requête de
# `apps/api` est une erreur de configuration/version-skew entre les
# deux services (voir le contrat documenté dans le plan de
# migration), pas un échec inattendu du service lui-même.
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(err)) from err
logger.info(
"tech-step NLP train done",
extra={
"locale": request.locale,
"labelCount": label_count,
"utteranceCount": utterance_count,
"synonymCount": synonym_count,
},
)
return TrainResponse(
locale=request.locale,
label_count=label_count,
utterance_count=utterance_count,
synonym_count=synonym_count,
)

40
services/tech-step-intent-service/intent_service/schemas.py
View file

@ -1,43 +1,13 @@
"""Modèles Pydantic du contrat HTTP — voir le plan de migration pour le
contrat exact attendu côté `apps/api` (`IntentServiceClient`,
`apps/api/src/lib/recipe-matching/intent-service-client.ts`).
Pas de `POST /v1/train` ici — ce service s'entraîne lui-même au démarrage
depuis `training_data.py` (voir `pipeline_registry.py`/`main.py`), plus
besoin d'un contrat HTTP pour ça.
"""
from pydantic import BaseModel, ConfigDict, Field
from pydantic.alias_generators import to_camel
# ---------------------------------------------------------------------------
# POST /v1/train
# ---------------------------------------------------------------------------
class TrainEntryPayload(BaseModel):
"""Une technique — mêmes champs qu'une entrée de `TECH_STEP_TRAINING_DATA`
(`apps/api/src/lib/recipe-matching/tech-step-training-data.ts`) pour une
locale donnée."""
uid: str
synonyms: list[str] = Field(default_factory=list)
utterances: list[str] = Field(default_factory=list)
class TrainRequest(BaseModel):
locale: str
entries: list[TrainEntryPayload]
class TrainResponse(BaseModel):
# camelCase en sortie (`labelCount`, pas `label_count`) — cohérent avec
# la convention JSON déjà en place côté `apps/api`/`packages/shared`
# (voir `TechStepAuditClauseView` etc.), même si le code Python interne
# reste en snake_case (convention PEP 8).
model_config = ConfigDict(alias_generator=to_camel, populate_by_name=True)
locale: str
label_count: int
utterance_count: int
synonym_count: int
from pydantic import BaseModel
# ---------------------------------------------------------------------------
# POST /v1/process

1692
services/tech-step-intent-service/intent_service/training_data.py Normal file
View file

File diff suppressed because it is too large Load diff

25
services/tech-step-intent-service/tests/conftest.py
View file

@ -2,9 +2,30 @@
`INTENT_SERVICE_SECRET` est absent — cette variable doit donc être définie
avant le tout premier `import intent_service...` de la session pytest.
`conftest.py` est chargé par pytest avant la collecte des modules de test,
donc avant que `test_routes_*.py`/`test_security.py` n'importent
`intent_service.main`."""
donc avant que `test_routes_process.py`/`test_security.py` n'importent
`intent_service.main`.
"""
import os
os.environ.setdefault("INTENT_SERVICE_SECRET", "pytest-only-secret-not-used-anywhere-else-32ch")
import pytest # noqa: E402 — après le `setdefault` ci-dessus, voir le docstring.
from fastapi.testclient import TestClient # noqa: E402
from intent_service.main import app # noqa: E402
@pytest.fixture(scope="session")
def client():
"""`TestClient(app)` utilisé comme gestionnaire de contexte déclenche le
vrai `lifespan` — puisque `main.py`'s `lifespan` entraîne maintenant
l'intégralité du vrai corpus `training_data.TECH_STEP_TRAINING_DATA`
(pas un jeu jouet, voir `PipelineRegistry.initialize`), refaire ça une
fois par fichier de test (ou pire, une fois par test) multiplierait un
entraînement non négligeable sur toute la suite pour rien — scope
"session" pour que chaque test ayant besoin d'une vraie app en cours
d'exécution partage la même instance déjà entraînée.
"""
with TestClient(app) as test_client:
yield test_client

5
services/tech-step-intent-service/tests/test_logging_config.py
View file

@ -1,6 +1,7 @@
"""Vérifie le format des lignes de log produites par
`logging_config._JsonFormatter` — ce que `routes/process.py`/`routes/train.py`
utilisent pour journaliser l'input/l'output de chaque appel NLP."""
`logging_config._JsonFormatter` — ce que `routes/process.py` et
`pipeline_registry.py` utilisent pour journaliser l'input/l'output de
chaque appel NLP et le déroulement de l'entraînement au démarrage."""
import json
import logging

68
services/tech-step-intent-service/tests/test_routes_process.py
View file

@ -1,52 +1,31 @@
"""Contrat JSON de `POST /v1/process` — voir `schemas.py`/`routes/process.py`."""
"""Contrat JSON de `POST /v1/process` — voir `schemas.py`/`routes/process.py`.
Le service s'entraîne désormais lui-même au démarrage sur le vrai corpus
(`training_data.TECH_STEP_TRAINING_DATA`, voir `conftest.py`'s fixture
`client` partagée) — ces tests vérifient donc le contrat HTTP contre des
phrases réelles du corpus, plus besoin d'un `POST /v1/train` préalable avec
des données jouets.
"""
import pytest
from fastapi.testclient import TestClient
from intent_service.config import settings
from intent_service.main import app
_HEADERS = {"X-Intent-Service-Secret": settings.intent_service_secret}
@pytest.fixture
def client():
with TestClient(app) as test_client:
yield test_client
def test_process_against_an_untrained_locale_returns_empty_result(client: TestClient):
def test_process_against_an_unsupported_locale_returns_empty_result(client: TestClient):
# "de" n'a aucun modèle spaCy connu (`SUPPORTED_LOCALES`) — se comporte
# comme "jamais entraîné" côté `/v1/process`, jamais une erreur (voir
# `PipelineRegistry.process`).
response = client.post(
"/v1/process", headers=_HEADERS, json={"locale": "fr", "text": "faire mijoter à feu doux"}
"/v1/process", headers=_HEADERS, json={"locale": "de", "text": "faire mijoter à feu doux"}
)
assert response.status_code == 200
assert response.json() == {"entities": [], "intent": None, "score": 0.0}
def test_process_after_train_returns_entities_and_intent(client: TestClient):
client.post(
"/v1/train",
headers=_HEADERS,
json={
"locale": "fr",
"entries": [
# `textcat` (exclusive_classes) exige >= 2 labels (voir
# LocalePipeline.train) — un second label est nécessaire
# même si ce test ne vérifie que celui de "simmer".
{
"uid": "simmer",
"synonyms": ["mijoter"],
"utterances": ["faire mijoter à feu doux", "laisser mijoter à couvert"],
},
{
"uid": "boil",
"synonyms": ["bouillir"],
"utterances": ["faire bouillir l'eau", "porter à ébullition"],
},
],
},
)
def test_process_returns_entities_and_intent_for_a_real_corpus_sentence(client: TestClient):
response = client.post(
"/v1/process", headers=_HEADERS, json={"locale": "fr", "text": "Faire mijoter à feu doux"}
)
@ -57,15 +36,16 @@ def test_process_after_train_returns_entities_and_intent(client: TestClient):
assert [entity["uid"] for entity in body["entities"]] == ["simmer"]
def test_process_with_blank_text_returns_empty_result(client: TestClient):
client.post(
"/v1/train",
headers=_HEADERS,
json={
"locale": "en",
"entries": [{"uid": "boil", "synonyms": ["boil"], "utterances": ["bring to the boil"]}],
},
def test_process_matches_english_text_against_the_english_trained_vocabulary(client: TestClient):
response = client.post(
"/v1/process", headers=_HEADERS, json={"locale": "en", "text": "Chop the onions finely"}
)
response = client.post("/v1/process", headers=_HEADERS, json={"locale": "en", "text": " "})
assert response.status_code == 200
body = response.json()
assert [entity["uid"] for entity in body["entities"]] == ["chop"]
def test_process_with_blank_text_returns_empty_result(client: TestClient):
response = client.post("/v1/process", headers=_HEADERS, json={"locale": "fr", "text": " "})
assert response.status_code == 200
assert response.json() == {"entities": [], "intent": None, "score": 0.0}

47
services/tech-step-intent-service/tests/test_routes_train.py
View file

@ -1,47 +0,0 @@
"""Contrat JSON de `POST /v1/train` — voir `schemas.py`/`routes/train.py`."""
import pytest
from fastapi.testclient import TestClient
from intent_service.config import settings
from intent_service.main import app
_HEADERS = {"X-Intent-Service-Secret": settings.intent_service_secret}
@pytest.fixture
def client():
with TestClient(app) as test_client:
yield test_client
def test_train_returns_counts(client: TestClient):
response = client.post(
"/v1/train",
headers=_HEADERS,
json={
"locale": "fr",
"entries": [
{"uid": "melt", "synonyms": ["faire fondre"], "utterances": ["faire fondre le beurre"]},
{"uid": "boil", "synonyms": ["bouillir"], "utterances": ["faire bouillir l'eau"]},
],
},
)
assert response.status_code == 200
body = response.json()
assert body == {"locale": "fr", "labelCount": 2, "utteranceCount": 2, "synonymCount": 2}
def test_train_with_unsupported_locale_returns_422(client: TestClient):
response = client.post(
"/v1/train",
headers=_HEADERS,
json={"locale": "de", "entries": [{"uid": "melt", "synonyms": [], "utterances": []}]},
)
assert response.status_code == 422
def test_train_with_empty_entries_returns_zero_counts(client: TestClient):
response = client.post("/v1/train", headers=_HEADERS, json={"locale": "en", "entries": []})
assert response.status_code == 200
assert response.json() == {"locale": "en", "labelCount": 0, "utteranceCount": 0, "synonymCount": 0}

13
services/tech-step-intent-service/tests/test_security.py
View file

@ -1,17 +1,12 @@
"""`require_valid_secret` — miroir inversé de
`require-internal-worker.test.ts` côté `apps/api`."""
`require-internal-worker.test.ts` côté `apps/api`. Utilise la fixture
`client` partagée (`conftest.py`) — pas besoin d'une app entraînée
séparément juste pour tester l'authentification.
"""
import pytest
from fastapi.testclient import TestClient
from intent_service.config import settings
from intent_service.main import app
@pytest.fixture
def client():
with TestClient(app) as test_client:
yield test_client
def test_rejects_a_missing_secret(client: TestClient):

34
specs/backend-architecture.md
View file

@ -479,10 +479,13 @@ appelé en HTTP par `TechStepClassifierService` via `IntentServiceClient`
(`intent-service-client.ts`) — `node-nlp` était peu maintenu et tournait
in-process dans l'event loop Node ; spaCy offre un écosystème NLP plus
robuste, dans un processus séparé, avec l'ambition à terme de pouvoir aussi
absorber ce que fait `services/tech-step-llm-worker`. `TECH_STEP_TRAINING_DATA`
reste possédé par `apps/api` (revu par PR comme le reste du code) et poussé
intégralement à ce service via `POST /v1/train` à chaque warm-up — ce service
ne touche jamais Postgres lui-même (voir son propre README).
absorber ce que fait `services/tech-step-llm-worker`. Ce service est
entièrement autonome : `TECH_STEP_TRAINING_DATA` (~74 techniques) vit
désormais dans son propre `training_data.py`, revu par PR comme le reste du
code mais plus poussé par `apps/api` via HTTP — le service s'entraîne
lui-même une seule fois, à son propre démarrage, et ne touche jamais
Postgres (voir son propre README, y compris pour le temps de démarrage —
plusieurs minutes, l'entraînement n'étant jamais persisté sur disque).
`normalizeText` (décomposition NFD + suppression des diacritiques + minuscule)
reste utilisée par `ingredient-matcher.ts`, mais n'intervient plus dans la
@ -516,12 +519,15 @@ normalisation du pipeline spaCy côté service.
un match clairement ancré sur un mot-clé juste parce que le modèle n'est
pas assez confiant.
Entraînement (`_train`) et résolution `TechStep.key -> id` sont mémoïsés une
seule fois sur le singleton partagé `techStepClassifier` (jamais par requête).
`server.ts` appelle `techStepClassifier.warmUp()` avant d'accepter du trafic,
avec retry/backoff si `services/tech-step-intent-service` n'est pas encore
prêt (le cas normal en Docker Compose, où `app` attend qu'il soit `healthy`
avant même de démarrer — voir `docker-compose.yml`).
Résolution `TechStep.key -> id` mémoïsée une seule fois sur le singleton
partagé `techStepClassifier` (jamais par requête) — c'est tout ce
qu'`apps/api` a encore à mémoïser, l'entraînement du modèle lui-même vivant
entièrement côté `services/tech-step-intent-service`. `server.ts` appelle
`techStepClassifier.warmUp()` avant d'accepter du trafic, avec retry/backoff
si `services/tech-step-intent-service` n'est pas encore joignable (le cas
normal en Docker Compose, où `app` attend qu'il soit `healthy` avant même de
démarrer — voir `docker-compose.yml`, et le README de ce service pour
combien de temps ça prend).
**Pièges rencontrés en construisant ce pipeline**, tous corrigés dans le code
(pas juste contournés) :
@ -538,10 +544,10 @@ avant même de démarrer — voir `docker-compose.yml`).
défaut — persistait le modèle entraîné dans un fichier `model.nlp` (cwd du
process) et le rechargeait *au lieu de* ré-entraîner au prochain démarrage
s'il existait déjà. Un modèle obsolète sur disque aurait masqué
silencieusement toute mise à jour de `TECH_STEP_TRAINING_DATA`. Non
applicable au service Python actuel : `POST /v1/train` reconstruit tout en
mémoire à chaque appel, sans jamais rien persister sur disque (voir ce
service's own README).
silencieusement toute mise à jour du corpus. Non applicable au service
Python actuel : il réentraîne tout en mémoire à chaque démarrage du
process, sans jamais rien persister sur disque (voir ce service's own
README).
- `nlp.make_doc()` (spaCy) ne fait tourner que le tokenizer, pas les
composants du pipeline — un piège trouvé en construisant le `PhraseMatcher`
du nouveau service : les patterns de synonymes doivent explicitement

6
specs/batch-cooking-modele.md
View file

@ -353,9 +353,9 @@ fiable.
`tech_step` (`TechStep`, `key` unique, ex. `"simmer"`) est le catalogue des
techniques (mijoter, préchauffer…) — juste un id/clé stable référencé par
`step_tech_step`. Les données de détection elles-mêmes (synonymes + phrases
d'exemple par langue, entraînant le microservice spaCy
`services/tech-step-intent-service`) vivent en code
(`tech-step-training-data.ts`), pas dans une table — l'ancienne
d'exemple par langue) vivent en code dans le microservice spaCy lui-même
(`services/tech-step-intent-service/intent_service/training_data.py`), pas
dans une table ni côté `apps/api` — l'ancienne
`tech_step_mapping` (`TechStepMapping`, une regex par technique/locale) a
été supprimée une fois constaté que les regex ne généralisaient jamais
au-delà de leur propre vocabulaire — voir