Cause racine du signalement "beaucoup d'ingredients ne sont pas linkes,
de meme pour les unites et les quantites" sur Marmiton/750g/Manger
Bouger : translateRecipe (recipe-translation.ts) ET previewSourceItem
(sources.service.ts) sautaient integralement loadIngredientCatalog/
loadUnitCatalog/translateRecipeIngredients des que locale !== "en" —
aucune tentative de matching n'etait jamais faite pour une source
francaise, pas un probleme de qualite de matching. Les trois sources
ajoutees dans cette session sont toutes locale: "fr".
Corrige en trois temps :
- packages/shared/src/data/catalog-labels-fr.ts (nouveau) :
INGREDIENT_LABELS_FR (554 entrees, copiees depuis
apps/web/src/locales/fr/translation.json qui les avait deja pour
l'UI — pas une nouvelle redaction), INGREDIENT_LABEL_SYNONYMS_FR
(mecanisme existant, pour patcher au cas par cas les libelles dont le
phrasage "affichage" ne correspond pas a l'ordre naturel d'un texte
de recette — ex. vanillaBean), UNIT_LABELS_FR (17 entrees,
redigees a la main comme UNIT_LABELS_EN — abreviations/variantes
reellement utilisees en francais : cuillere a soupe/cas/c.a.s...).
- ingredient-matcher.ts : stemWord se scinde en stemWordEn/stemWordFr
(locale parametrable, defaut "en" pour ne rien casser) — le stemmer
anglais appliquait sa regle "es" -> "" a des pluriels francais
reguliers ("carottes" -> "carott" au lieu de "carotte"), cassant
silencieusement le matching pour la quasi-totalite des ingredients
francais dont le singulier se termine par une voyelle. matchUnit est
reecrit pour chercher une sous-sequence ordonnee (comme
matchIngredientName) plutot qu'une egalite du seul premier mot : un
synonyme francais peut etre multi-mots ("cuillere a soupe"), une
phrase entiere ne pouvant jamais egaler un seul mot extrait.
loadIngredientCatalog/loadUnitCatalog prennent un parametre locale.
- recipe-translation.ts/sources.service.ts : suppression du
if (locale !== "en") qui court-circuitait tout — les catalogues sont
desormais toujours charges avec la locale de la source ; une locale
sans table de libelles recoit simplement des catalogues vides (degrade
gracieusement, ne plante pas).
Tests : 14 nouveaux tests purs (matchIngredientName/matchUnit fr,
stemmer, regression), 3 nouveaux tests DB (loadIngredientCatalog/
loadUnitCatalog fr + locale inconnue), 3 nouveaux tests
recipe-translation remplacant un test qui figeait l'ancien comportement
cassé, 1 nouveau test d'integration HTTP (sources.test.ts) avec un
adaptateur factice francais bout en bout. Les tests DB n'ont pas pu
etre executes localement (pas de Postgres/Docker dans cet environnement
sandbox) — a verifier en CI.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
331 lines
15 KiB
TypeScript
331 lines
15 KiB
TypeScript
import type { UnitType } from "@batch-cooking/shared";
|
|
import type {
|
|
ParsedRecipe,
|
|
ParsedRecipeIngredient,
|
|
ParsedRecipeStep,
|
|
} from "../recipe-sources/recipe-source-adapter.js";
|
|
import {
|
|
extractQuantity,
|
|
type IngredientMatchEntry,
|
|
loadIngredientCatalog,
|
|
loadUnitCatalog,
|
|
matchIngredientName,
|
|
matchUnit,
|
|
type UnitMatchEntry,
|
|
} from "./ingredient-matcher.js";
|
|
import { techStepClassifier } from "./tech-step-matcher.js";
|
|
|
|
/**
|
|
* The "Traduction en étapes" stage of the import pipeline described in
|
|
* specs/batch-cooking-architecture.md (Import depuis source → **Traduction
|
|
* en étapes** → Sauvegarde) — takes a source-agnostic {@link ParsedRecipe}
|
|
* (recipe-source-adapter.ts's `parse()` output) and declares each step's
|
|
* technique sequence, the same `techStepIds: number[]` shape
|
|
* `Step.techSteps`/`StepTechStep` (schema.prisma) will eventually persist.
|
|
*
|
|
* Also resolves ingredients — matching each free-text `ParsedRecipeIngredient`
|
|
* line against our `Ingredient`/`Unit` catalogs (`ingredient-matcher.ts`),
|
|
* the same "free source text -> our catalog id" idea as tech-step
|
|
* detection, just for ingredients/units/quantities instead of technique
|
|
* verbs. Like tech-step matching, this doesn't turn the result into a
|
|
* saveable `Recipe` either (no `dietIds`/`visibility`/author, a source
|
|
* can't know those) — this is one step of the pipeline, not the whole
|
|
* thing.
|
|
*
|
|
* `translateRecipeIngredients` stays pure (takes its matching data as plain
|
|
* arguments, same convention `matchIngredientName` itself has) so it's
|
|
* unit-testable without a database. `translateRecipeSteps` no longer is —
|
|
* technique detection now goes through `techStepClassifier`'s trained
|
|
* model (`tech-step-matcher.ts`), which needs an async call — but is still
|
|
* exported separately from `translateRecipe` for callers/tests that only
|
|
* care about step translation, not ingredients too.
|
|
*/
|
|
|
|
/** A {@link ParsedRecipeStep}, after tech-step detection — declares its technique sequence alongside the description/picture it already had. */
|
|
export interface TranslatedRecipeStep extends ParsedRecipeStep {
|
|
/** Ordered sequence of detected `TechStep` ids (see `TechStepClassifierService.matchTechSteps`) — empty if this step doesn't mention any known technique. */
|
|
techStepIds: number[];
|
|
}
|
|
|
|
/** A {@link ParsedRecipeIngredient}, after ingredient/unit matching — `quantity` is filled in from `rawText` when the source itself left it `null` (see `extractQuantity`); `ingredientId`/`unitId` are `null` when nothing in the catalog matched. */
|
|
export interface TranslatedRecipeIngredient extends ParsedRecipeIngredient {
|
|
ingredientId: number | null;
|
|
unitId: number | null;
|
|
}
|
|
|
|
/** A {@link ParsedRecipe} whose `steps`/`ingredients` have been translated — everything else (name, portions, …) passes through unchanged. */
|
|
export interface TranslatedRecipe extends Omit<ParsedRecipe, "steps" | "ingredients"> {
|
|
steps: TranslatedRecipeStep[];
|
|
ingredients: TranslatedRecipeIngredient[];
|
|
}
|
|
|
|
/**
|
|
* Declares each of `recipe`'s steps' technique sequence for `locale`,
|
|
* leaving everything else about the recipe untouched — including
|
|
* ingredients, which are only stubbed to `TranslatedRecipeIngredient`'s
|
|
* shape here (`ingredientId`/`unitId: null`, `quantity`/`unit` untouched);
|
|
* actually resolving them is {@link translateRecipeIngredients}'s job, kept
|
|
* separate the same way tech-step and ingredient matching are two
|
|
* independent concerns everywhere else in this module. Async — technique
|
|
* detection now runs against `techStepClassifier`'s trained model rather
|
|
* than a caller-supplied mapping list (see `tech-step-matcher.ts`), so this
|
|
* can no longer stay a plain synchronous function the way it used to.
|
|
*/
|
|
export async function translateRecipeSteps(
|
|
recipe: ParsedRecipe,
|
|
locale: string,
|
|
): Promise<TranslatedRecipe> {
|
|
try {
|
|
const steps = await Promise.all(
|
|
recipe.steps.map(async (step) => ({
|
|
...step,
|
|
techStepIds: await techStepClassifier.matchTechSteps(step.description, locale),
|
|
})),
|
|
);
|
|
return {
|
|
...recipe,
|
|
ingredients: recipe.ingredients.map((ingredient) => ({
|
|
...ingredient,
|
|
ingredientId: null,
|
|
unitId: null,
|
|
})),
|
|
steps,
|
|
};
|
|
} catch (err) {
|
|
// Rethrown as-is — the caller (`sources.service.ts`) already
|
|
// handles/logs failures centrally; this function just isn't allowed a
|
|
// bare `await` per the repo's async/try-catch convention.
|
|
throw err;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Locale-specific label {@link matchUnit} is fed when a quantity was found
|
|
* but no unit word was — see the `unitId` fallback below. Each is the
|
|
* catalog's generic "counted, no further unit" entry (`Unit.key` `"piece"`)
|
|
* in that locale's own label table (`UNIT_LABELS_EN`/`UNIT_LABELS_FR`,
|
|
* `packages/shared`). A locale with neither entry (anything but
|
|
* `"en"`/`"fr"`) falls back to the English spelling — harmless, since
|
|
* `unitCatalog` itself is already empty for an unsupported locale (see
|
|
* `loadUnitCatalog`), so this fallback lookup finds nothing either way.
|
|
*/
|
|
const FALLBACK_COUNT_UNIT_LABEL_BY_LOCALE: Record<string, string> = {
|
|
en: "piece",
|
|
fr: "unité",
|
|
};
|
|
|
|
/**
|
|
* Resolves each of `ingredients`' free-text `name`/`unit`/`quantity`
|
|
* against `ingredientCatalog`/`unitCatalog` (see `ingredient-matcher.ts`).
|
|
* Pure — testable with hand-built catalogs, no database involved (see
|
|
* `translateRecipe` for the DB-backed loader). `quantity` falls back to
|
|
* `extractQuantity(rawText)` only when the source itself left it `null`;
|
|
* same for `unit` falling back to `extractQuantity`'s `remainder` before
|
|
* being matched against `unitCatalog` — a source that already states a
|
|
* clean unit/quantity is trusted over re-deriving it from `rawText`.
|
|
*
|
|
* When a quantity was found but nothing in the remaining text matched a
|
|
* unit (e.g. `"4 Egg Yolks"` — quantity `4`, remainder `"Egg Yolks"`, no
|
|
* unit word anywhere in it), `unitId` falls back to the catalog's generic
|
|
* `piece` unit rather than staying `null`: a bare count with no explicit
|
|
* unit word is overwhelmingly "N of them" (eggs, onions, cloves not
|
|
* spelled out as "clove") in practice, not a genuinely missing unit — see
|
|
* issue #53, where this previously left the import review form's submit
|
|
* button disabled with no indication why on almost any recipe with a
|
|
* whole-item ingredient. No fallback when `quantity` itself is `null`
|
|
* (e.g. `"To taste"`) — there's nothing to count, so nothing to default.
|
|
*
|
|
* `locale` (default `"en"`, matching every pre-existing caller) must agree
|
|
* with whichever locale `ingredientCatalog`/`unitCatalog` were loaded in
|
|
* (see `loadIngredientCatalog`/`loadUnitCatalog`) — it's threaded through to
|
|
* `matchIngredientName`/`matchUnit` for stemming, and picks the right
|
|
* spelling of the "piece" fallback below.
|
|
*/
|
|
export function translateRecipeIngredients(
|
|
ingredients: ParsedRecipeIngredient[],
|
|
ingredientCatalog: IngredientMatchEntry[],
|
|
unitCatalog: UnitMatchEntry[],
|
|
locale = "en",
|
|
): TranslatedRecipeIngredient[] {
|
|
const fallbackCountUnitLabel = FALLBACK_COUNT_UNIT_LABEL_BY_LOCALE[locale] ?? "piece";
|
|
return ingredients.map((ingredient) => {
|
|
const ingredientId = matchIngredientName(ingredient.name, ingredientCatalog, locale);
|
|
const extracted = extractQuantity(ingredient.rawText);
|
|
const quantity = ingredient.quantity ?? extracted.quantity;
|
|
const unitText = ingredient.unit ?? extracted.remainder;
|
|
const unitId =
|
|
matchUnit(unitText, unitCatalog, locale) ??
|
|
(quantity !== null ? matchUnit(fallbackCountUnitLabel, unitCatalog, locale) : null);
|
|
return { ...ingredient, quantity, ingredientId, unitId };
|
|
});
|
|
}
|
|
|
|
/** What {@link mergeDuplicateIngredients} needs to know about a `Unit` to combine two of them — the `type`/`toBaseFactor` slice of `UnitView` (`packages/shared`). */
|
|
export interface UnitConversionEntry {
|
|
id: number;
|
|
type: UnitType;
|
|
toBaseFactor: number;
|
|
}
|
|
|
|
/**
|
|
* Combines `a`/`b` — two lines already confirmed to resolve to the same
|
|
* ingredient — into one, summing their quantities, or returns `null` when
|
|
* that can't be done safely. `null` (quantity or unit missing on either
|
|
* side, unit not found in `unitById`, mismatched `UnitType`, or either side
|
|
* a `COUNT` unit) means "don't merge", not "error" — see
|
|
* {@link mergeDuplicateIngredients}.
|
|
*
|
|
* Same unit on both sides sums directly. Different units of the same
|
|
* measurable *type* (`MASS`/`VOLUME`) convert `b`'s quantity into `a`'s
|
|
* unit via `toBaseFactor` first — the groundwork that field's own doc
|
|
* comment (`UnitView`, `packages/shared`) already anticipated ("a future
|
|
* conversion feature ... summing '500g' + '0.5kg'"). `COUNT` units are
|
|
* never converted against each other even when `toBaseFactor` matches — a
|
|
* "pincée" isn't a fixed fraction of a "gousse" (same doc comment) — so two
|
|
* different `COUNT` units for the same ingredient are left unmerged.
|
|
* Rounded to 2 decimal places (`RecipeIngredient.quantity` is
|
|
* `Decimal(10, 2)`, schema.prisma) to avoid floating-point noise from the
|
|
* conversion.
|
|
*/
|
|
function combineIngredientLines(
|
|
a: TranslatedRecipeIngredient,
|
|
b: TranslatedRecipeIngredient,
|
|
unitById: Map<number, UnitConversionEntry>,
|
|
): TranslatedRecipeIngredient | null {
|
|
if (a.quantity === null || b.quantity === null || a.unitId === null || b.unitId === null) {
|
|
return null;
|
|
}
|
|
if (a.unitId === b.unitId) {
|
|
return {
|
|
...a,
|
|
quantity: a.quantity + b.quantity,
|
|
rawText: `${a.rawText} + ${b.rawText}`,
|
|
};
|
|
}
|
|
|
|
const unitA = unitById.get(a.unitId);
|
|
const unitB = unitById.get(b.unitId);
|
|
if (!unitA || !unitB) return null;
|
|
if (unitA.type !== unitB.type || unitA.type === "COUNT") return null;
|
|
|
|
const combinedInBaseUnit = a.quantity * unitA.toBaseFactor + b.quantity * unitB.toBaseFactor;
|
|
const quantity = Math.round((combinedInBaseUnit / unitA.toBaseFactor) * 100) / 100;
|
|
return { ...a, quantity, rawText: `${a.rawText} + ${b.rawText}` };
|
|
}
|
|
|
|
/**
|
|
* Folds `ingredients` down to one line per resolved `ingredientId`,
|
|
* concatenating (summing the quantity of) every duplicate into the first
|
|
* line it matches — see issue #53's follow-up: two raw source lines (e.g.
|
|
* TheMealDB's "Egg Yolks"/"Eggs", or "100g Sugar" used in two different
|
|
* steps) can independently resolve to the same catalog `Ingredient`, and
|
|
* `RecipeIngredient`'s primary key (`recipeId`, `ingredientId`) only
|
|
* allows one row per ingredient per recipe — the review form used to
|
|
* either crash on submit (before `createRecipeSchema` rejected it) or
|
|
* require the person to manually delete every extra line by hand.
|
|
*
|
|
* Unresolved lines (`ingredientId: null`) are never merged with one
|
|
* another or with anything else — nothing reliable to key them on. Two
|
|
* lines that resolve to the same ingredient but can't be combined safely
|
|
* (see {@link combineIngredientLines} — mismatched quantity/unit, or
|
|
* genuinely incompatible units) are left as separate, still-duplicate
|
|
* lines: `createRecipeSchema` still rejects the result, and
|
|
* `RecipeImportForm` still highlights them, same safety net as before this
|
|
* merge step existed — merging never *invents* a number it isn't confident
|
|
* in.
|
|
*
|
|
* Pure — testable with a hand-built `unitCatalog`, no database involved.
|
|
* Order-preserving: a merged line keeps its first occurrence's position.
|
|
*/
|
|
export function mergeDuplicateIngredients(
|
|
ingredients: TranslatedRecipeIngredient[],
|
|
unitCatalog: UnitConversionEntry[],
|
|
): TranslatedRecipeIngredient[] {
|
|
const unitById = new Map(unitCatalog.map((unit) => [unit.id, unit]));
|
|
const merged: TranslatedRecipeIngredient[] = [];
|
|
const mergedIndexByIngredientId = new Map<number, number>();
|
|
|
|
for (const line of ingredients) {
|
|
const existingIndex =
|
|
line.ingredientId !== null ? mergedIndexByIngredientId.get(line.ingredientId) : undefined;
|
|
|
|
const existingLine = existingIndex !== undefined ? merged[existingIndex] : undefined;
|
|
if (existingIndex === undefined || existingLine === undefined) {
|
|
if (line.ingredientId !== null) {
|
|
mergedIndexByIngredientId.set(line.ingredientId, merged.length);
|
|
}
|
|
merged.push(line);
|
|
continue;
|
|
}
|
|
|
|
const combined = combineIngredientLines(existingLine, line, unitById);
|
|
if (combined === null) {
|
|
merged.push(line);
|
|
} else {
|
|
merged[existingIndex] = combined;
|
|
}
|
|
}
|
|
|
|
return merged;
|
|
}
|
|
|
|
/**
|
|
* Convenience wrapper around {@link translateRecipeSteps}/
|
|
* {@link translateRecipeIngredients} that loads every catalog itself —
|
|
* what a caller reaches for when translating a single recipe on its own
|
|
* (e.g. the eventual "import this one recipe" endpoint). A caller
|
|
* translating many recipes at once should load the catalogs once and reuse
|
|
* them across calls instead, the same "don't requery per item" reasoning
|
|
* `recipe.service.ts`'s `createRecipe`/`updateRecipe` already follow for
|
|
* manually-authored recipes.
|
|
*
|
|
* No user- or recipe-level language preference exists anywhere in the app
|
|
* yet (see `tech-step-matcher.ts`'s `TechStepClassifierService`) — callers
|
|
* pass a locale explicitly rather than this module guessing one. Note that
|
|
* an English-language source (e.g. TheMealDB) translated against `"fr"`
|
|
* will currently get an empty (or nonsensical) `techStepIds` sequence on
|
|
* every step — the classifier is trained per-locale, so calling it with a
|
|
* locale that doesn't match the actual text's language doesn't degrade
|
|
* gracefully, it just gets things wrong.
|
|
*
|
|
* Ingredient/unit matching has data for `"en"` and `"fr"` today
|
|
* (`INGREDIENT_LABELS_EN`/`_FR`, `UNIT_LABELS_EN`/`_FR`, `packages/shared`)
|
|
* — `loadIngredientCatalog(locale)`/`loadUnitCatalog(locale)` are always
|
|
* called, never specially skipped for a particular locale: a locale with no
|
|
* label table of its own (anything but `"en"`/`"fr"`) just gets back empty
|
|
* catalogs from those two loaders, and `translateRecipeIngredients` over an
|
|
* empty catalog naturally leaves every ingredient's `ingredientId`/`unitId`
|
|
* at the neutral `null` `translateRecipeSteps` already stubs in — the same
|
|
* "no matching-language data" degradation tech-step matching already has
|
|
* for a locale with no mappings, just arrived at by *not* special-casing
|
|
* which locales are "supported" here at all (that's `INGREDIENT_LABELS_BY_LOCALE`/
|
|
* `UNIT_LABELS_BY_LOCALE`'s job, in `ingredient-matcher.ts` — this function
|
|
* doesn't need its own copy of that list to stay in sync with).
|
|
*/
|
|
export async function translateRecipe(
|
|
recipe: ParsedRecipe,
|
|
locale: string,
|
|
): Promise<TranslatedRecipe> {
|
|
try {
|
|
const translated = await translateRecipeSteps(recipe, locale);
|
|
|
|
const [ingredientCatalog, unitCatalog] = await Promise.all([
|
|
loadIngredientCatalog(locale),
|
|
loadUnitCatalog(locale),
|
|
]);
|
|
return {
|
|
...translated,
|
|
ingredients: translateRecipeIngredients(
|
|
recipe.ingredients,
|
|
ingredientCatalog,
|
|
unitCatalog,
|
|
locale,
|
|
),
|
|
};
|
|
} catch (err) {
|
|
// Rethrown as-is — the caller (`sources.service.ts`) already
|
|
// handles/logs failures centrally; this function just isn't allowed a
|
|
// bare `await` per the repo's async/try-catch convention.
|
|
throw err;
|
|
}
|
|
}
|