batchCooking/apps/api/src/lib/recipe-matching/recipe-translation.ts
kyuno053 520e539fe6
fix(recipes): active le matching ingredients/unites pour les sources en francais (#72)
Cause racine du signalement "beaucoup d'ingredients ne sont pas linkes,
de meme pour les unites et les quantites" sur Marmiton/750g/Manger
Bouger : translateRecipe (recipe-translation.ts) ET previewSourceItem
(sources.service.ts) sautaient integralement loadIngredientCatalog/
loadUnitCatalog/translateRecipeIngredients des que locale !== "en" —
aucune tentative de matching n'etait jamais faite pour une source
francaise, pas un probleme de qualite de matching. Les trois sources
ajoutees dans cette session sont toutes locale: "fr".

Corrige en trois temps :

- packages/shared/src/data/catalog-labels-fr.ts (nouveau) :
  INGREDIENT_LABELS_FR (554 entrees, copiees depuis
  apps/web/src/locales/fr/translation.json qui les avait deja pour
  l'UI — pas une nouvelle redaction), INGREDIENT_LABEL_SYNONYMS_FR
  (mecanisme existant, pour patcher au cas par cas les libelles dont le
  phrasage "affichage" ne correspond pas a l'ordre naturel d'un texte
  de recette — ex. vanillaBean), UNIT_LABELS_FR (17 entrees,
  redigees a la main comme UNIT_LABELS_EN — abreviations/variantes
  reellement utilisees en francais : cuillere a soupe/cas/c.a.s...).

- ingredient-matcher.ts : stemWord se scinde en stemWordEn/stemWordFr
  (locale parametrable, defaut "en" pour ne rien casser) — le stemmer
  anglais appliquait sa regle "es" -> "" a des pluriels francais
  reguliers ("carottes" -> "carott" au lieu de "carotte"), cassant
  silencieusement le matching pour la quasi-totalite des ingredients
  francais dont le singulier se termine par une voyelle. matchUnit est
  reecrit pour chercher une sous-sequence ordonnee (comme
  matchIngredientName) plutot qu'une egalite du seul premier mot : un
  synonyme francais peut etre multi-mots ("cuillere a soupe"), une
  phrase entiere ne pouvant jamais egaler un seul mot extrait.
  loadIngredientCatalog/loadUnitCatalog prennent un parametre locale.

- recipe-translation.ts/sources.service.ts : suppression du
  if (locale !== "en") qui court-circuitait tout — les catalogues sont
  desormais toujours charges avec la locale de la source ; une locale
  sans table de libelles recoit simplement des catalogues vides (degrade
  gracieusement, ne plante pas).

Tests : 14 nouveaux tests purs (matchIngredientName/matchUnit fr,
stemmer, regression), 3 nouveaux tests DB (loadIngredientCatalog/
loadUnitCatalog fr + locale inconnue), 3 nouveaux tests
recipe-translation remplacant un test qui figeait l'ancien comportement
cassé, 1 nouveau test d'integration HTTP (sources.test.ts) avec un
adaptateur factice francais bout en bout. Les tests DB n'ont pas pu
etre executes localement (pas de Postgres/Docker dans cet environnement
sandbox) — a verifier en CI.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-22 20:34:32 +02:00

331 lines
15 KiB
TypeScript

import type { UnitType } from "@batch-cooking/shared";
import type {
ParsedRecipe,
ParsedRecipeIngredient,
ParsedRecipeStep,
} from "../recipe-sources/recipe-source-adapter.js";
import {
extractQuantity,
type IngredientMatchEntry,
loadIngredientCatalog,
loadUnitCatalog,
matchIngredientName,
matchUnit,
type UnitMatchEntry,
} from "./ingredient-matcher.js";
import { techStepClassifier } from "./tech-step-matcher.js";
/**
* The "Traduction en étapes" stage of the import pipeline described in
* specs/batch-cooking-architecture.md (Import depuis source → **Traduction
* en étapes** → Sauvegarde) — takes a source-agnostic {@link ParsedRecipe}
* (recipe-source-adapter.ts's `parse()` output) and declares each step's
* technique sequence, the same `techStepIds: number[]` shape
* `Step.techSteps`/`StepTechStep` (schema.prisma) will eventually persist.
*
* Also resolves ingredients — matching each free-text `ParsedRecipeIngredient`
* line against our `Ingredient`/`Unit` catalogs (`ingredient-matcher.ts`),
* the same "free source text -> our catalog id" idea as tech-step
* detection, just for ingredients/units/quantities instead of technique
* verbs. Like tech-step matching, this doesn't turn the result into a
* saveable `Recipe` either (no `dietIds`/`visibility`/author, a source
* can't know those) — this is one step of the pipeline, not the whole
* thing.
*
* `translateRecipeIngredients` stays pure (takes its matching data as plain
* arguments, same convention `matchIngredientName` itself has) so it's
* unit-testable without a database. `translateRecipeSteps` no longer is —
* technique detection now goes through `techStepClassifier`'s trained
* model (`tech-step-matcher.ts`), which needs an async call — but is still
* exported separately from `translateRecipe` for callers/tests that only
* care about step translation, not ingredients too.
*/
/** A {@link ParsedRecipeStep}, after tech-step detection — declares its technique sequence alongside the description/picture it already had. */
export interface TranslatedRecipeStep extends ParsedRecipeStep {
/** Ordered sequence of detected `TechStep` ids (see `TechStepClassifierService.matchTechSteps`) — empty if this step doesn't mention any known technique. */
techStepIds: number[];
}
/** A {@link ParsedRecipeIngredient}, after ingredient/unit matching — `quantity` is filled in from `rawText` when the source itself left it `null` (see `extractQuantity`); `ingredientId`/`unitId` are `null` when nothing in the catalog matched. */
export interface TranslatedRecipeIngredient extends ParsedRecipeIngredient {
ingredientId: number | null;
unitId: number | null;
}
/** A {@link ParsedRecipe} whose `steps`/`ingredients` have been translated — everything else (name, portions, …) passes through unchanged. */
export interface TranslatedRecipe extends Omit<ParsedRecipe, "steps" | "ingredients"> {
steps: TranslatedRecipeStep[];
ingredients: TranslatedRecipeIngredient[];
}
/**
* Declares each of `recipe`'s steps' technique sequence for `locale`,
* leaving everything else about the recipe untouched — including
* ingredients, which are only stubbed to `TranslatedRecipeIngredient`'s
* shape here (`ingredientId`/`unitId: null`, `quantity`/`unit` untouched);
* actually resolving them is {@link translateRecipeIngredients}'s job, kept
* separate the same way tech-step and ingredient matching are two
* independent concerns everywhere else in this module. Async — technique
* detection now runs against `techStepClassifier`'s trained model rather
* than a caller-supplied mapping list (see `tech-step-matcher.ts`), so this
* can no longer stay a plain synchronous function the way it used to.
*/
export async function translateRecipeSteps(
recipe: ParsedRecipe,
locale: string,
): Promise<TranslatedRecipe> {
try {
const steps = await Promise.all(
recipe.steps.map(async (step) => ({
...step,
techStepIds: await techStepClassifier.matchTechSteps(step.description, locale),
})),
);
return {
...recipe,
ingredients: recipe.ingredients.map((ingredient) => ({
...ingredient,
ingredientId: null,
unitId: null,
})),
steps,
};
} catch (err) {
// Rethrown as-is — the caller (`sources.service.ts`) already
// handles/logs failures centrally; this function just isn't allowed a
// bare `await` per the repo's async/try-catch convention.
throw err;
}
}
/**
* Locale-specific label {@link matchUnit} is fed when a quantity was found
* but no unit word was — see the `unitId` fallback below. Each is the
* catalog's generic "counted, no further unit" entry (`Unit.key` `"piece"`)
* in that locale's own label table (`UNIT_LABELS_EN`/`UNIT_LABELS_FR`,
* `packages/shared`). A locale with neither entry (anything but
* `"en"`/`"fr"`) falls back to the English spelling — harmless, since
* `unitCatalog` itself is already empty for an unsupported locale (see
* `loadUnitCatalog`), so this fallback lookup finds nothing either way.
*/
const FALLBACK_COUNT_UNIT_LABEL_BY_LOCALE: Record<string, string> = {
en: "piece",
fr: "unité",
};
/**
* Resolves each of `ingredients`' free-text `name`/`unit`/`quantity`
* against `ingredientCatalog`/`unitCatalog` (see `ingredient-matcher.ts`).
* Pure — testable with hand-built catalogs, no database involved (see
* `translateRecipe` for the DB-backed loader). `quantity` falls back to
* `extractQuantity(rawText)` only when the source itself left it `null`;
* same for `unit` falling back to `extractQuantity`'s `remainder` before
* being matched against `unitCatalog` — a source that already states a
* clean unit/quantity is trusted over re-deriving it from `rawText`.
*
* When a quantity was found but nothing in the remaining text matched a
* unit (e.g. `"4 Egg Yolks"` — quantity `4`, remainder `"Egg Yolks"`, no
* unit word anywhere in it), `unitId` falls back to the catalog's generic
* `piece` unit rather than staying `null`: a bare count with no explicit
* unit word is overwhelmingly "N of them" (eggs, onions, cloves not
* spelled out as "clove") in practice, not a genuinely missing unit — see
* issue #53, where this previously left the import review form's submit
* button disabled with no indication why on almost any recipe with a
* whole-item ingredient. No fallback when `quantity` itself is `null`
* (e.g. `"To taste"`) — there's nothing to count, so nothing to default.
*
* `locale` (default `"en"`, matching every pre-existing caller) must agree
* with whichever locale `ingredientCatalog`/`unitCatalog` were loaded in
* (see `loadIngredientCatalog`/`loadUnitCatalog`) — it's threaded through to
* `matchIngredientName`/`matchUnit` for stemming, and picks the right
* spelling of the "piece" fallback below.
*/
export function translateRecipeIngredients(
ingredients: ParsedRecipeIngredient[],
ingredientCatalog: IngredientMatchEntry[],
unitCatalog: UnitMatchEntry[],
locale = "en",
): TranslatedRecipeIngredient[] {
const fallbackCountUnitLabel = FALLBACK_COUNT_UNIT_LABEL_BY_LOCALE[locale] ?? "piece";
return ingredients.map((ingredient) => {
const ingredientId = matchIngredientName(ingredient.name, ingredientCatalog, locale);
const extracted = extractQuantity(ingredient.rawText);
const quantity = ingredient.quantity ?? extracted.quantity;
const unitText = ingredient.unit ?? extracted.remainder;
const unitId =
matchUnit(unitText, unitCatalog, locale) ??
(quantity !== null ? matchUnit(fallbackCountUnitLabel, unitCatalog, locale) : null);
return { ...ingredient, quantity, ingredientId, unitId };
});
}
/** What {@link mergeDuplicateIngredients} needs to know about a `Unit` to combine two of them — the `type`/`toBaseFactor` slice of `UnitView` (`packages/shared`). */
export interface UnitConversionEntry {
id: number;
type: UnitType;
toBaseFactor: number;
}
/**
* Combines `a`/`b` — two lines already confirmed to resolve to the same
* ingredient — into one, summing their quantities, or returns `null` when
* that can't be done safely. `null` (quantity or unit missing on either
* side, unit not found in `unitById`, mismatched `UnitType`, or either side
* a `COUNT` unit) means "don't merge", not "error" — see
* {@link mergeDuplicateIngredients}.
*
* Same unit on both sides sums directly. Different units of the same
* measurable *type* (`MASS`/`VOLUME`) convert `b`'s quantity into `a`'s
* unit via `toBaseFactor` first — the groundwork that field's own doc
* comment (`UnitView`, `packages/shared`) already anticipated ("a future
* conversion feature ... summing '500g' + '0.5kg'"). `COUNT` units are
* never converted against each other even when `toBaseFactor` matches — a
* "pincée" isn't a fixed fraction of a "gousse" (same doc comment) — so two
* different `COUNT` units for the same ingredient are left unmerged.
* Rounded to 2 decimal places (`RecipeIngredient.quantity` is
* `Decimal(10, 2)`, schema.prisma) to avoid floating-point noise from the
* conversion.
*/
function combineIngredientLines(
a: TranslatedRecipeIngredient,
b: TranslatedRecipeIngredient,
unitById: Map<number, UnitConversionEntry>,
): TranslatedRecipeIngredient | null {
if (a.quantity === null || b.quantity === null || a.unitId === null || b.unitId === null) {
return null;
}
if (a.unitId === b.unitId) {
return {
...a,
quantity: a.quantity + b.quantity,
rawText: `${a.rawText} + ${b.rawText}`,
};
}
const unitA = unitById.get(a.unitId);
const unitB = unitById.get(b.unitId);
if (!unitA || !unitB) return null;
if (unitA.type !== unitB.type || unitA.type === "COUNT") return null;
const combinedInBaseUnit = a.quantity * unitA.toBaseFactor + b.quantity * unitB.toBaseFactor;
const quantity = Math.round((combinedInBaseUnit / unitA.toBaseFactor) * 100) / 100;
return { ...a, quantity, rawText: `${a.rawText} + ${b.rawText}` };
}
/**
* Folds `ingredients` down to one line per resolved `ingredientId`,
* concatenating (summing the quantity of) every duplicate into the first
* line it matches — see issue #53's follow-up: two raw source lines (e.g.
* TheMealDB's "Egg Yolks"/"Eggs", or "100g Sugar" used in two different
* steps) can independently resolve to the same catalog `Ingredient`, and
* `RecipeIngredient`'s primary key (`recipeId`, `ingredientId`) only
* allows one row per ingredient per recipe — the review form used to
* either crash on submit (before `createRecipeSchema` rejected it) or
* require the person to manually delete every extra line by hand.
*
* Unresolved lines (`ingredientId: null`) are never merged with one
* another or with anything else — nothing reliable to key them on. Two
* lines that resolve to the same ingredient but can't be combined safely
* (see {@link combineIngredientLines} — mismatched quantity/unit, or
* genuinely incompatible units) are left as separate, still-duplicate
* lines: `createRecipeSchema` still rejects the result, and
* `RecipeImportForm` still highlights them, same safety net as before this
* merge step existed — merging never *invents* a number it isn't confident
* in.
*
* Pure — testable with a hand-built `unitCatalog`, no database involved.
* Order-preserving: a merged line keeps its first occurrence's position.
*/
export function mergeDuplicateIngredients(
ingredients: TranslatedRecipeIngredient[],
unitCatalog: UnitConversionEntry[],
): TranslatedRecipeIngredient[] {
const unitById = new Map(unitCatalog.map((unit) => [unit.id, unit]));
const merged: TranslatedRecipeIngredient[] = [];
const mergedIndexByIngredientId = new Map<number, number>();
for (const line of ingredients) {
const existingIndex =
line.ingredientId !== null ? mergedIndexByIngredientId.get(line.ingredientId) : undefined;
const existingLine = existingIndex !== undefined ? merged[existingIndex] : undefined;
if (existingIndex === undefined || existingLine === undefined) {
if (line.ingredientId !== null) {
mergedIndexByIngredientId.set(line.ingredientId, merged.length);
}
merged.push(line);
continue;
}
const combined = combineIngredientLines(existingLine, line, unitById);
if (combined === null) {
merged.push(line);
} else {
merged[existingIndex] = combined;
}
}
return merged;
}
/**
* Convenience wrapper around {@link translateRecipeSteps}/
* {@link translateRecipeIngredients} that loads every catalog itself —
* what a caller reaches for when translating a single recipe on its own
* (e.g. the eventual "import this one recipe" endpoint). A caller
* translating many recipes at once should load the catalogs once and reuse
* them across calls instead, the same "don't requery per item" reasoning
* `recipe.service.ts`'s `createRecipe`/`updateRecipe` already follow for
* manually-authored recipes.
*
* No user- or recipe-level language preference exists anywhere in the app
* yet (see `tech-step-matcher.ts`'s `TechStepClassifierService`) — callers
* pass a locale explicitly rather than this module guessing one. Note that
* an English-language source (e.g. TheMealDB) translated against `"fr"`
* will currently get an empty (or nonsensical) `techStepIds` sequence on
* every step — the classifier is trained per-locale, so calling it with a
* locale that doesn't match the actual text's language doesn't degrade
* gracefully, it just gets things wrong.
*
* Ingredient/unit matching has data for `"en"` and `"fr"` today
* (`INGREDIENT_LABELS_EN`/`_FR`, `UNIT_LABELS_EN`/`_FR`, `packages/shared`)
* — `loadIngredientCatalog(locale)`/`loadUnitCatalog(locale)` are always
* called, never specially skipped for a particular locale: a locale with no
* label table of its own (anything but `"en"`/`"fr"`) just gets back empty
* catalogs from those two loaders, and `translateRecipeIngredients` over an
* empty catalog naturally leaves every ingredient's `ingredientId`/`unitId`
* at the neutral `null` `translateRecipeSteps` already stubs in — the same
* "no matching-language data" degradation tech-step matching already has
* for a locale with no mappings, just arrived at by *not* special-casing
* which locales are "supported" here at all (that's `INGREDIENT_LABELS_BY_LOCALE`/
* `UNIT_LABELS_BY_LOCALE`'s job, in `ingredient-matcher.ts` — this function
* doesn't need its own copy of that list to stay in sync with).
*/
export async function translateRecipe(
recipe: ParsedRecipe,
locale: string,
): Promise<TranslatedRecipe> {
try {
const translated = await translateRecipeSteps(recipe, locale);
const [ingredientCatalog, unitCatalog] = await Promise.all([
loadIngredientCatalog(locale),
loadUnitCatalog(locale),
]);
return {
...translated,
ingredients: translateRecipeIngredients(
recipe.ingredients,
ingredientCatalog,
unitCatalog,
locale,
),
};
} catch (err) {
// Rethrown as-is — the caller (`sources.service.ts`) already
// handles/logs failures centrally; this function just isn't allowed a
// bare `await` per the repo's async/try-catch convention.
throw err;
}
}