feat(recipes): ajoute un adaptateur RecipeSourceAdapter pour 750g
Suit le même schéma que marmitonAdapter (construit sur jsonLdRecipeAdapter), avec deux différences propres à 750g.com : - list() n'a pas d'ItemList JSON-LD à lire sur ses résultats de recherche (la recherche du site est un widget client-side) — appelle donc directement le endpoint GET que ce widget interroge lui-même en interne (un « moteur de réponse IA » qui renvoie un lot de recettes pour une requête en texte libre), et scrape les cartes de résultat par regex en associant à chaque lien de recette sa dernière image précédente plutôt qu'un zip naïf par index (des images décoratives sans carte associée existent réellement dans ce fragment). Vérifié en direct : demander une « page 2 » revient toujours vide, donc nextCursor vaut toujours null, comme theMealDbAdapter. - parse() ne délègue pas aussi directement à jsonLdRecipeAdapter.parse que marmitonAdapter — le générateur JSON-LD de 750g.com a deux bugs réels : des caractères de contrôle bruts non échappés dans certaines chaînes JSON (~1 recette sur 3 dans un échantillon vérifié en direct, sinon JSON.parse échoue et jsonLdRecipeAdapter rapporte à tort « aucun Recipe trouvé »), et un texte parfois doublement encodé en entités HTML (ex. un vrai « é » devient &eacute; au lieu de é). Les deux sont corrigés en pré/post-traitement autour de la même délégation, pas une réimplémentation. Enregistre sevenFiftyGAdapter dans registerAllRecipeSources (sources/index.ts), au même titre que marmitonAdapter. Complète aussi test/sources/sources-index.test.ts, qui ne couvrait encore que TheMealDB malgré l'ajout de Marmiton dans une PR précédente. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
bc456973b7
commit
be01730a98
5 changed files with 713 additions and 6 deletions
389
apps/api/src/sources/750g.ts
Normal file
389
apps/api/src/sources/750g.ts
Normal file
|
|
@ -0,0 +1,389 @@
|
|||
import type {
|
||||
ParsedRecipe,
|
||||
RecipeSourceAdapter,
|
||||
RecipeSourceListItem,
|
||||
RecipeSourceListParams,
|
||||
RecipeSourceListResult,
|
||||
} from "../lib/recipe-sources/recipe-source-adapter.js";
|
||||
import {
|
||||
RecipeSourceFetchError,
|
||||
RecipeSourceParseError,
|
||||
} from "../lib/recipe-sources/recipe-source-errors.js";
|
||||
import { jsonLdRecipeAdapter } from "./json-ld-recipe.js";
|
||||
|
||||
const SOURCE_KEY = "750g";
|
||||
|
||||
// 750g.com's own site search is a client-side widget (results are fetched
|
||||
// by the page's own JS after load, nothing server-rendered to scrape) — but
|
||||
// that JS itself calls this plain GET endpoint, an "AI answer engine" that
|
||||
// returns an HTML fragment of recipe cards for a free-text query. Verified
|
||||
// live: works with a bare `fetch`, no special headers/cookies/session
|
||||
// needed, same as every other adapter in this family.
|
||||
const SEARCH_URL = "https://www.750g.com/genius/query/";
|
||||
|
||||
/**
|
||||
* Matches every `<script type="application/ld+json">…</script>` block —
|
||||
* same shape as `JSON_LD_SCRIPT_PATTERN` in json-ld-recipe.ts, kept as its
|
||||
* own private copy here rather than sharing that module's export: this one
|
||||
* does textual surgery on the *raw HTML* before `jsonLdRecipeAdapter` ever
|
||||
* sees it (see {@link sanitizeJsonLdBlocks} below), a different concern
|
||||
* from extracting-and-parsing blocks into objects.
|
||||
*/
|
||||
const JSON_LD_SCRIPT_PATTERN =
|
||||
/(<script[^>]*type\s*=\s*["']application\/ld\+json["'][^>]*>)([\s\S]*?)(<\/script>)/gi;
|
||||
|
||||
/**
|
||||
* Escapes any raw (unescaped) JSON control character — U+0000–U+001F —
|
||||
* found *inside* a string literal of `json`, leaving everything outside
|
||||
* string literals (structural whitespace, brackets, …) untouched. Fixes a
|
||||
* real bug in 750g.com's own JSON-LD generator: some `HowToStep.text`
|
||||
* values contain a literal, un-escaped `\r\n` where valid JSON requires
|
||||
* `\\r\\n` (verified live, e.g.
|
||||
* https://www.750g.com/poulet-au-vin-jaune-et-aux-morilles-r3844.htm — and
|
||||
* roughly a third of a random sample of recipe pages hit this) —
|
||||
* `JSON.parse` throws "Bad control character in string literal" on these
|
||||
* pages as-is, which would make `jsonLdRecipeAdapter.parse` wrongly report
|
||||
* "no JSON-LD Recipe found" on a page that has a perfectly good one.
|
||||
*
|
||||
* A blind find/replace across the whole block would be wrong: JSON also
|
||||
* uses real newlines as *structural* whitespace between tokens
|
||||
* (pretty-printing), where they're perfectly legal and must be left alone —
|
||||
* only walking the text with string-literal awareness (tracking `"…"`
|
||||
* boundaries and `\`-escapes) can tell the two apart.
|
||||
*/
|
||||
function escapeRawControlCharactersInStrings(json: string): string {
|
||||
const SHORT_ESCAPES: Record<string, string> = {
|
||||
"\b": "\\b",
|
||||
"\f": "\\f",
|
||||
"\n": "\\n",
|
||||
"\r": "\\r",
|
||||
"\t": "\\t",
|
||||
};
|
||||
|
||||
let result = "";
|
||||
let inString = false;
|
||||
let escapedNext = false;
|
||||
for (const ch of json) {
|
||||
if (!inString) {
|
||||
if (ch === '"') inString = true;
|
||||
result += ch;
|
||||
continue;
|
||||
}
|
||||
if (escapedNext) {
|
||||
result += ch;
|
||||
escapedNext = false;
|
||||
continue;
|
||||
}
|
||||
if (ch === "\\") {
|
||||
result += ch;
|
||||
escapedNext = true;
|
||||
continue;
|
||||
}
|
||||
if (ch === '"') {
|
||||
inString = false;
|
||||
result += ch;
|
||||
continue;
|
||||
}
|
||||
if (ch < " ") {
|
||||
result += SHORT_ESCAPES[ch] ?? `\\u${ch.charCodeAt(0).toString(16).padStart(4, "0")}`;
|
||||
continue;
|
||||
}
|
||||
result += ch;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs {@link escapeRawControlCharactersInStrings} over every JSON-LD
|
||||
* `<script>` block's content in `html`, leaving the rest of the page
|
||||
* untouched — the repair step `fetchDetail`'s raw HTML needs before
|
||||
* `jsonLdRecipeAdapter.parse` (which does its own extraction/`JSON.parse`
|
||||
* internally) ever sees it.
|
||||
*/
|
||||
function sanitizeJsonLdBlocks(html: string): string {
|
||||
return html.replace(
|
||||
JSON_LD_SCRIPT_PATTERN,
|
||||
(_match, openTag: string, json: string, closeTag: string) =>
|
||||
`${openTag}${escapeRawControlCharactersInStrings(json)}${closeTag}`,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Numeric entities plus a hand-picked table of named ones — not a general
|
||||
* HTML5 entity decoder (400+ named entities exist in the spec), just what's
|
||||
* actually been observed necessary to clean up 750g.com's French recipe
|
||||
* text: the five basic XML entities, Latin-1 accented letters, and a
|
||||
* handful of common punctuation entities.
|
||||
*/
|
||||
const NAMED_ENTITIES: Record<string, string> = {
|
||||
amp: "&",
|
||||
lt: "<",
|
||||
gt: ">",
|
||||
quot: '"',
|
||||
apos: "'",
|
||||
nbsp: " ",
|
||||
eacute: "é",
|
||||
egrave: "è",
|
||||
ecirc: "ê",
|
||||
euml: "ë",
|
||||
agrave: "à",
|
||||
acirc: "â",
|
||||
auml: "ä",
|
||||
icirc: "î",
|
||||
iuml: "ï",
|
||||
ocirc: "ô",
|
||||
ouml: "ö",
|
||||
ucirc: "û",
|
||||
ugrave: "ù",
|
||||
uuml: "ü",
|
||||
ccedil: "ç",
|
||||
oelig: "œ",
|
||||
aelig: "æ",
|
||||
laquo: "«",
|
||||
raquo: "»",
|
||||
lsquo: "‘",
|
||||
rsquo: "’",
|
||||
ldquo: "“",
|
||||
rdquo: "”",
|
||||
hellip: "…",
|
||||
ndash: "–",
|
||||
mdash: "—",
|
||||
deg: "°",
|
||||
};
|
||||
|
||||
/** One pass of numeric (`'`/`'`) and {@link NAMED_ENTITIES} decoding — see {@link decodeHtmlEntities}, which is what actually runs against parsed text; this is split out only so that function can run it twice. */
|
||||
function decodeHtmlEntitiesOnce(text: string): string {
|
||||
return text.replace(/&(#x?[0-9a-fA-F]+|[a-zA-Z]+);/g, (entity, body: string) => {
|
||||
if (body[0] === "#") {
|
||||
const isHex = body[1] === "x" || body[1] === "X";
|
||||
const codePoint = isHex
|
||||
? Number.parseInt(body.slice(2), 16)
|
||||
: Number.parseInt(body.slice(1), 10);
|
||||
return Number.isFinite(codePoint) ? String.fromCodePoint(codePoint) : entity;
|
||||
}
|
||||
return NAMED_ENTITIES[body] ?? entity;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Decodes HTML entities in free-text pulled from 750g.com, run **twice**:
|
||||
* its JSON-LD sometimes double-escapes text that already went through its
|
||||
* own HTML-entity encoder once — e.g. a real "é" ends up as `&eacute;`
|
||||
* (the `&` of an already-produced `é` got re-escaped to `&`)
|
||||
* rather than a plain `é` or a raw "é" (verified live, e.g.
|
||||
* "Pr&eacute;parez" on
|
||||
* https://www.750g.com/poulet-au-vin-jaune-et-aux-morilles-r3844.htm). One
|
||||
* pass turns that into `é` — a *newly formed*, valid-looking entity
|
||||
* — so a second pass is needed to resolve it the rest of the way to "é". A
|
||||
* string with no entities at all (the common case) is unaffected by either
|
||||
* pass.
|
||||
*/
|
||||
function decodeHtmlEntities(text: string): string {
|
||||
return decodeHtmlEntitiesOnce(decodeHtmlEntitiesOnce(text));
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs {@link decodeHtmlEntities} over every free-text field of a
|
||||
* `ParsedRecipe` produced by `jsonLdRecipeAdapter.parse`. `picture`/
|
||||
* `sourceUrl` are deliberately left untouched — they're URLs, not prose,
|
||||
* and the entity-encoding bug this fixes has only ever been observed in
|
||||
* name/description/instruction/ingredient text, never in a URL field.
|
||||
*/
|
||||
function decodeParsedRecipeText(recipe: ParsedRecipe): ParsedRecipe {
|
||||
return {
|
||||
...recipe,
|
||||
name: decodeHtmlEntities(recipe.name),
|
||||
description: recipe.description === null ? null : decodeHtmlEntities(recipe.description),
|
||||
ingredients: recipe.ingredients.map((ingredient) => ({
|
||||
...ingredient,
|
||||
rawText: decodeHtmlEntities(ingredient.rawText),
|
||||
name: decodeHtmlEntities(ingredient.name),
|
||||
})),
|
||||
steps: recipe.steps.map((step) => ({
|
||||
...step,
|
||||
description: decodeHtmlEntities(step.description),
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
/** One recipe card as scraped off a 750g.com results fragment (search or listing page) — see {@link extractRecipeCards}. */
|
||||
interface SevenFiftyGCard {
|
||||
url: string;
|
||||
title: string;
|
||||
image: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scrapes every recipe card out of a 750g.com results HTML fragment —
|
||||
* there's no JSON-LD `ItemList` on this endpoint to lean on (unlike
|
||||
* marmiton.ts's search page), just the same server-rendered `card-recipe`
|
||||
* markup 750g.com uses everywhere. Each card's title/url comes from its
|
||||
* `<a class="card-link">`; its image is whichever `<img>` most recently
|
||||
* preceded that link, rather than a naive same-index zip of "every image on
|
||||
* the page" against "every link on the page" — a plain fragment like this
|
||||
* one carries a few extra decorative images with no card of their own
|
||||
* (verified live: 28 `<img>` tags against 23 real cards for one sample
|
||||
* query), which would silently shift every image after the first stray one
|
||||
* onto the wrong title. Each card's own `<img>` always sits immediately
|
||||
* before its title link in the markup, so "nearest preceding image" is
|
||||
* unambiguous and doesn't depend on the two counts matching.
|
||||
*/
|
||||
function extractRecipeCards(html: string): SevenFiftyGCard[] {
|
||||
const linkPattern =
|
||||
/<a\s+href="(https:\/\/www\.750g\.com\/[^"]+)"\s+class="card-link[^"]*">([^<]+)<\/a>/g;
|
||||
const imagePattern = /<img[^>]*\ssrc="(https:\/\/static\.750g\.com\/images\/[^"]+)"[^>]*>/g;
|
||||
const images = [...html.matchAll(imagePattern)];
|
||||
|
||||
const cards: SevenFiftyGCard[] = [];
|
||||
let searchFrom = 0;
|
||||
for (const linkMatch of html.matchAll(linkPattern)) {
|
||||
let image: string | null = null;
|
||||
for (const imgMatch of images) {
|
||||
if (imgMatch.index === undefined || imgMatch.index >= linkMatch.index) break;
|
||||
if (imgMatch.index >= searchFrom) image = imgMatch[1] ?? null;
|
||||
}
|
||||
cards.push({
|
||||
url: linkMatch[1] ?? "",
|
||||
title: decodeHtmlEntities(linkMatch[2] ?? ""),
|
||||
image,
|
||||
});
|
||||
searchFrom = linkMatch.index;
|
||||
}
|
||||
return cards;
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-labels a `RecipeSourceFetchError`/`RecipeSourceParseError` thrown by
|
||||
* the generic `jsonLdRecipeAdapter` (`sourceKey` `"jsonLdRecipe"`) as having
|
||||
* come from this adapter instead (`sourceKey` `"750g"`) — same reasoning as
|
||||
* marmiton.ts's identically-named helper: `fetchDetail`/`parse` below are
|
||||
* thin wrappers around the generic adapter's own methods, but a caller
|
||||
* catching `RecipeSourceError` and reading `.sourceKey` should see "750g",
|
||||
* the source it actually asked about.
|
||||
*/
|
||||
function rekeySourceError(err: unknown): unknown {
|
||||
if (err instanceof RecipeSourceFetchError) {
|
||||
return new RecipeSourceFetchError(SOURCE_KEY, err.message, { cause: err.cause });
|
||||
}
|
||||
if (err instanceof RecipeSourceParseError) {
|
||||
return new RecipeSourceParseError(SOURCE_KEY, err.message, { cause: err.cause });
|
||||
}
|
||||
return err;
|
||||
}
|
||||
|
||||
/**
|
||||
* 750g.com — one of France's largest recipe sites. Unofficial (`official:
|
||||
* false`): no published API, this adapter fetches ordinary pages and reads
|
||||
* the schema.org structured data 750g.com embeds for search engines, built
|
||||
* on {@link jsonLdRecipeAdapter} the same way marmiton.ts is. Two real
|
||||
* 750g-specific problems separate this adapter from a pure thin wrapper
|
||||
* like marmiton.ts, though:
|
||||
*
|
||||
* - `list()` has no `ItemList` JSON-LD to read off its search results (see
|
||||
* {@link extractRecipeCards}) — its site search is a client-side widget,
|
||||
* so this instead calls the plain GET endpoint that widget's own JS calls
|
||||
* internally (`SEARCH_URL`), an "AI answer engine" that returns a curated
|
||||
* batch of cards for a free-text query rather than an exhaustive,
|
||||
* paginated catalog — verified live, requesting `page=2` of the same
|
||||
* query always comes back empty, so `nextCursor` is always `null` here,
|
||||
* same as `theMealDbAdapter`'s "one response holds every match".
|
||||
* - `parse()` doesn't delegate to `jsonLdRecipeAdapter.parse` as directly as
|
||||
* marmiton.ts's does — 750g.com's own JSON-LD generator has two real bugs
|
||||
* this adapter works around: some pages embed literal, unescaped control
|
||||
* characters inside a JSON string (see {@link sanitizeJsonLdBlocks}), and
|
||||
* its free text is sometimes double HTML-entity-encoded (see
|
||||
* {@link decodeParsedRecipeText}). Both are pre/post-processing around the
|
||||
* same underlying delegation, not a reimplementation of it.
|
||||
*/
|
||||
export const sevenFiftyGAdapter: RecipeSourceAdapter<{ html: string; url: string }> = {
|
||||
key: SOURCE_KEY,
|
||||
name: "750g",
|
||||
official: false,
|
||||
// Un chemin stable, sans le paramètre `?v=…` de cache-busting que 750g.com
|
||||
// ajoute à ses balises <link> (susceptible de changer à chaque
|
||||
// déploiement) — cette adresse répond correctement sans lui.
|
||||
iconUrl: "https://www.750g.com/img/750g/favicons/favicon.svg",
|
||||
// Le contenu de 750g.com (noms, ingrédients, instructions) est en
|
||||
// français — détermine contre quel modèle/locale d'étiquettes
|
||||
// d'ingrédients translateRecipe (recipe-translation.ts) résout les
|
||||
// recettes de cette source lors d'une prévisualisation/d'un import.
|
||||
locale: "fr",
|
||||
|
||||
async list(params: RecipeSourceListParams): Promise<RecipeSourceListResult> {
|
||||
try {
|
||||
const query = params.query ?? "";
|
||||
// `params.cursor` est ignoré : voir le commentaire du module — cette
|
||||
// recherche ne pagine pas réellement, il n'existe donc jamais de
|
||||
// curseur légitime à faire transiter (`nextCursor` vaut toujours
|
||||
// `null` ci-dessous).
|
||||
const searchUrl = `${SEARCH_URL}?query=${encodeURIComponent(query)}&query_type=written_query&page=1`;
|
||||
|
||||
let response: Response;
|
||||
try {
|
||||
response = await fetch(searchUrl);
|
||||
} catch (cause) {
|
||||
throw new RecipeSourceFetchError(
|
||||
SOURCE_KEY,
|
||||
`Network error searching 750g (${searchUrl})`,
|
||||
{
|
||||
cause,
|
||||
},
|
||||
);
|
||||
}
|
||||
if (!response.ok) {
|
||||
throw new RecipeSourceFetchError(
|
||||
SOURCE_KEY,
|
||||
`750g search responded ${response.status} (${searchUrl})`,
|
||||
);
|
||||
}
|
||||
const html = await response.text();
|
||||
|
||||
// No filter for a missing title/url here (unlike marmiton.ts/
|
||||
// the-meal-db.ts, which drop entries with a null field from a
|
||||
// structured API response) — `extractRecipeCards`' own regex requires
|
||||
// at least one character for both, so there's no "absent field" shape
|
||||
// to guard against.
|
||||
const items: RecipeSourceListItem[] = extractRecipeCards(html).map((card) => ({
|
||||
externalId: card.url,
|
||||
title: card.title,
|
||||
picture: card.image,
|
||||
url: card.url,
|
||||
}));
|
||||
|
||||
return { items, nextCursor: null };
|
||||
} catch (err) {
|
||||
// Rethrown as-is (already keyed "750g" by whichever branch above
|
||||
// threw it) — this adapter's only caller (`sources.service.ts`)
|
||||
// already handles/logs failures centrally; this method just isn't
|
||||
// allowed a bare `async` body without a try/catch per the repo's
|
||||
// convention. Same reasoning as `marmiton.ts`/`json-ld-recipe.ts`.
|
||||
throw err;
|
||||
}
|
||||
},
|
||||
|
||||
// `externalId` est directement l'URL canonique de la recette sur
|
||||
// 750g.com (renvoyée telle quelle par `list()` ci-dessus) — même
|
||||
// convention que `jsonLdRecipeAdapter.fetchDetail`, à qui cette méthode
|
||||
// délègue entièrement : la réparation du JSON-LD (voir
|
||||
// `sanitizeJsonLdBlocks`) n'a lieu qu'à l'étape `parse()`, pas ici.
|
||||
async fetchDetail(externalId: string): Promise<{ html: string; url: string }> {
|
||||
try {
|
||||
return await jsonLdRecipeAdapter.fetchDetail(externalId);
|
||||
} catch (err) {
|
||||
// Pas un simple re-throw : `rekeySourceError` est le traitement utile
|
||||
// que ce point d'appel doit faire de l'erreur (relabelliser sa
|
||||
// `sourceKey`), conformément à la convention await/try-catch du repo.
|
||||
throw rekeySourceError(err);
|
||||
}
|
||||
},
|
||||
|
||||
parse(raw: { html: string; url: string }): ParsedRecipe {
|
||||
try {
|
||||
const sanitizedHtml = sanitizeJsonLdBlocks(raw.html);
|
||||
const parsed = jsonLdRecipeAdapter.parse({ html: sanitizedHtml, url: raw.url });
|
||||
return decodeParsedRecipeText(parsed);
|
||||
} catch (err) {
|
||||
throw rekeySourceError(err);
|
||||
}
|
||||
},
|
||||
};
|
||||
|
|
@ -1,12 +1,14 @@
|
|||
import { registerRecipeSource } from "../lib/recipe-sources/recipe-source-registry.js";
|
||||
import { sevenFiftyGAdapter } from "./750g.js";
|
||||
import { marmitonAdapter } from "./marmiton.js";
|
||||
import { theMealDbAdapter } from "./the-meal-db.js";
|
||||
|
||||
/**
|
||||
* Registers every concrete, *browsable* `RecipeSourceAdapter` this app
|
||||
* ships with into the shared in-memory registry (`recipe-source-registry.ts`)
|
||||
* — `theMealDbAdapter` and `marmitonAdapter`. Called once, explicitly, by
|
||||
* the two real entry points that need the registry populated:
|
||||
* — `theMealDbAdapter`, `marmitonAdapter` and `sevenFiftyGAdapter`. Called
|
||||
* once, explicitly, by the two real entry points that need the registry
|
||||
* populated:
|
||||
*
|
||||
* - `server.ts` — the running API process, before it starts listening.
|
||||
* - `prisma/seed.ts` — so `syncRecipeSources` has something to mirror into
|
||||
|
|
@ -26,12 +28,13 @@ import { theMealDbAdapter } from "./the-meal-db.js";
|
|||
* specialized per scraped website, not a household-toggleable `Source` in
|
||||
* its own right: nobody can meaningfully "trust" or "enable" a generic
|
||||
* parsing mechanism the way they can a named website. `marmitonAdapter`
|
||||
* (marmiton.ts) is exactly that specialization for marmiton.org — the
|
||||
* first concrete adapter built on top of it, per its own doc comment's
|
||||
* anticipation ("a concrete adapter for a specific site would use it
|
||||
* (marmiton.ts) and `sevenFiftyGAdapter` (750g.ts) are exactly that
|
||||
* specialization, one per site — the concrete adapters its own doc comment
|
||||
* anticipated ("a concrete adapter for a specific site would use it
|
||||
* internally").
|
||||
*/
|
||||
export function registerAllRecipeSources(): void {
|
||||
registerRecipeSource(theMealDbAdapter);
|
||||
registerRecipeSource(marmitonAdapter);
|
||||
registerRecipeSource(sevenFiftyGAdapter);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -182,7 +182,10 @@ function flattenInstructions(instructions: SchemaOrgRecipe["recipeInstructions"]
|
|||
* marmiton.org recipe page's JSON-LD is a plain schema.org `Recipe`, nothing
|
||||
* site-specific to handle), and it only adds the `list()` this adapter
|
||||
* itself can't offer, by reading the separate `ItemList` marmiton.org embeds
|
||||
* on its search-results pages.
|
||||
* on its search-results pages. `750g.ts`'s `sevenFiftyGAdapter` follows the
|
||||
* same shape for 750g.com, but its `parse()` wraps this adapter's own
|
||||
* (rather than delegating untouched) to work around two real bugs in that
|
||||
* site's JSON-LD — see that module's doc comment.
|
||||
*/
|
||||
export const jsonLdRecipeAdapter: RecipeSourceAdapter<{
|
||||
html: string;
|
||||
|
|
|
|||
294
apps/api/test/recipe-sources/750g.test.ts
Normal file
294
apps/api/test/recipe-sources/750g.test.ts
Normal file
|
|
@ -0,0 +1,294 @@
|
|||
import { expect } from "chai";
|
||||
import {
|
||||
RecipeSourceFetchError,
|
||||
RecipeSourceParseError,
|
||||
} from "../../src/lib/recipe-sources/recipe-source-errors.js";
|
||||
import { sevenFiftyGAdapter } from "../../src/sources/750g.js";
|
||||
|
||||
/** Stubs `globalThis.fetch` to return `html` as the response body — same pattern as `json-ld-recipe.test.ts`'s `stubFetchHtml`. */
|
||||
function stubFetchHtml(html: string, status = 200) {
|
||||
globalThis.fetch = (async () => new Response(html, { status })) as typeof fetch;
|
||||
}
|
||||
|
||||
const RECIPE_URL = "https://www.750g.com/poulet-au-vin-jaune-et-aux-morilles-r3844.htm";
|
||||
|
||||
/**
|
||||
* A raw (not `JSON.stringify`-escaped) JSON-LD `Recipe` payload, deliberately
|
||||
* reproducing two real 750g.com bugs verified live on the recipe this test's
|
||||
* URL/content is modeled after:
|
||||
* - a literal, unescaped `\r\n` inside `recipeInstructions[0].text` (invalid
|
||||
* JSON as-is — this is exactly what {@link sanitizeJsonLdBlocks} in the
|
||||
* adapter under test has to repair before `JSON.parse` can succeed);
|
||||
* - `Pr&eacute;parez` — a real "é" that went through 750g's own
|
||||
* HTML-entity encoder twice (`decodeHtmlEntities` has to run twice to
|
||||
* fully resolve it back to "é").
|
||||
* Plus a plain `'` apostrophe entity in an ingredient line, the more
|
||||
* common single-encoding case.
|
||||
*/
|
||||
const RAW_RECIPE_JSON_LD = `{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Recipe",
|
||||
"name": "Poulet au vin jaune et aux morilles",
|
||||
"description": "Une recette de f\\u00eate.",
|
||||
"image": {"@type": "ImageObject", "url": "https://static.750g.com/images/poulet-vin-jaune.jpg"},
|
||||
"recipeYield": "6 personnes",
|
||||
"recipeIngredient": ["1 poulet fermier", "Sel 'fin'"],
|
||||
"recipeInstructions": [
|
||||
{"@type": "HowToStep", "text": "Pr&eacute;parez les morilles :\r\nFendez-les en deux."}
|
||||
],
|
||||
"url": "${RECIPE_URL}"
|
||||
}`;
|
||||
|
||||
function htmlWithRawJsonLd(rawJson: string): string {
|
||||
return `<!doctype html><html><head><script type="application/ld+json">${rawJson}</script></head><body></body></html>`;
|
||||
}
|
||||
|
||||
describe("sevenFiftyGAdapter", () => {
|
||||
let originalFetch: typeof fetch;
|
||||
|
||||
beforeEach(() => {
|
||||
originalFetch = globalThis.fetch;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
globalThis.fetch = originalFetch;
|
||||
});
|
||||
|
||||
it("declares itself as an unofficial, French-locale source with an icon", () => {
|
||||
expect(sevenFiftyGAdapter.key).to.equal("750g");
|
||||
expect(sevenFiftyGAdapter.name).to.equal("750g");
|
||||
expect(sevenFiftyGAdapter.official).to.equal(false);
|
||||
expect(sevenFiftyGAdapter.iconUrl).to.be.a("string");
|
||||
expect(sevenFiftyGAdapter.locale).to.equal("fr");
|
||||
});
|
||||
|
||||
describe("list", () => {
|
||||
/**
|
||||
* Models the real shape found live: a card's own `<img>` sits
|
||||
* immediately before its `<a class="card-link">`, but the fragment also
|
||||
* carries decorative images that belong to no card at all (verified
|
||||
* live: 28 `<img>` tags against 23 real cards for one sample query) —
|
||||
* an image search that isn't "nearest preceding, not naive same-index
|
||||
* zip" would misattribute every card after the first stray image.
|
||||
*/
|
||||
const CARDS_HTML = `
|
||||
<div class="grid">
|
||||
<img src="https://static.750g.com/images/x/orphan-lead.jpg" class="decorative" />
|
||||
<div class="card">
|
||||
<img src="https://static.750g.com/images/x/tarte.jpg" alt="Tarte" />
|
||||
<a href="https://www.750g.com/tarte-aux-pommes-r1.htm" class="card-link ">Tarte aux pommes</a>
|
||||
</div>
|
||||
<img src="https://static.750g.com/images/x/orphan-mid-1.jpg" class="decorative" />
|
||||
<img src="https://static.750g.com/images/x/orphan-mid-2.jpg" class="decorative" />
|
||||
<div class="card">
|
||||
<img src="https://static.750g.com/images/x/gratin.jpg" alt="Gratin" />
|
||||
<a href="https://www.750g.com/gratin-dauphinois-r2.htm" class="card-link ">Gratin dauphinois</a>
|
||||
</div>
|
||||
<div class="card">
|
||||
<a href="https://www.750g.com/pain-perdu-r3.htm" class="card-link ">Pain perdu</a>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
it("scrapes each card's title/url/image, matching each image to its nearest preceding link and ignoring orphan images", async () => {
|
||||
stubFetchHtml(CARDS_HTML);
|
||||
|
||||
const result = await sevenFiftyGAdapter.list({ query: "tarte" });
|
||||
|
||||
expect(result.items).to.deep.equal([
|
||||
{
|
||||
externalId: "https://www.750g.com/tarte-aux-pommes-r1.htm",
|
||||
title: "Tarte aux pommes",
|
||||
picture: "https://static.750g.com/images/x/tarte.jpg",
|
||||
url: "https://www.750g.com/tarte-aux-pommes-r1.htm",
|
||||
},
|
||||
{
|
||||
externalId: "https://www.750g.com/gratin-dauphinois-r2.htm",
|
||||
title: "Gratin dauphinois",
|
||||
picture: "https://static.750g.com/images/x/gratin.jpg",
|
||||
url: "https://www.750g.com/gratin-dauphinois-r2.htm",
|
||||
},
|
||||
{
|
||||
externalId: "https://www.750g.com/pain-perdu-r3.htm",
|
||||
title: "Pain perdu",
|
||||
picture: null,
|
||||
url: "https://www.750g.com/pain-perdu-r3.htm",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("decodes HTML entities in a card's title", async () => {
|
||||
stubFetchHtml(
|
||||
`<a href="https://www.750g.com/tarte-r1.htm" class="card-link ">Tarte aux pommes 'reinettes'</a>`,
|
||||
);
|
||||
|
||||
const result = await sevenFiftyGAdapter.list({ query: "tarte" });
|
||||
|
||||
expect(result.items[0]?.title).to.equal("Tarte aux pommes 'reinettes'");
|
||||
});
|
||||
|
||||
it("always returns nextCursor: null — this search isn't really paginated (requesting a further page comes back empty)", async () => {
|
||||
stubFetchHtml(CARDS_HTML);
|
||||
|
||||
const result = await sevenFiftyGAdapter.list({ query: "tarte" });
|
||||
|
||||
expect(result.nextCursor).to.be.null;
|
||||
});
|
||||
|
||||
it("ignores params.cursor and always requests page=1 — there's never a legitimate cursor to pass back", async () => {
|
||||
let requestedUrl: string | undefined;
|
||||
globalThis.fetch = (async (url: string) => {
|
||||
requestedUrl = url;
|
||||
return new Response("", { status: 200 });
|
||||
}) as typeof fetch;
|
||||
|
||||
await sevenFiftyGAdapter.list({ query: "tarte", cursor: "7" });
|
||||
|
||||
expect(requestedUrl).to.include("page=1");
|
||||
expect(requestedUrl).not.to.include("page=7");
|
||||
});
|
||||
|
||||
it("URL-encodes the query", async () => {
|
||||
let requestedUrl: string | undefined;
|
||||
globalThis.fetch = (async (url: string) => {
|
||||
requestedUrl = url;
|
||||
return new Response("", { status: 200 });
|
||||
}) as typeof fetch;
|
||||
|
||||
await sevenFiftyGAdapter.list({ query: "tarte aux pommes" });
|
||||
|
||||
expect(requestedUrl).to.include("query=tarte%20aux%20pommes");
|
||||
});
|
||||
|
||||
it("throws RecipeSourceFetchError on a non-2xx response", async () => {
|
||||
stubFetchHtml("", 500);
|
||||
|
||||
try {
|
||||
await sevenFiftyGAdapter.list({ query: "x" });
|
||||
expect.fail("expected list to throw");
|
||||
} catch (err) {
|
||||
expect(err).to.be.instanceOf(RecipeSourceFetchError);
|
||||
expect((err as RecipeSourceFetchError).sourceKey).to.equal("750g");
|
||||
}
|
||||
});
|
||||
|
||||
it("throws RecipeSourceFetchError when the network request itself fails", async () => {
|
||||
globalThis.fetch = (async () => {
|
||||
throw new Error("network down");
|
||||
}) as typeof fetch;
|
||||
|
||||
try {
|
||||
await sevenFiftyGAdapter.list({ query: "x" });
|
||||
expect.fail("expected list to throw");
|
||||
} catch (err) {
|
||||
expect(err).to.be.instanceOf(RecipeSourceFetchError);
|
||||
expect((err as RecipeSourceFetchError).cause).to.be.instanceOf(Error);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("fetchDetail", () => {
|
||||
it("fetches the given recipe URL and returns its html alongside the url", async () => {
|
||||
stubFetchHtml(htmlWithRawJsonLd(RAW_RECIPE_JSON_LD));
|
||||
|
||||
const result = await sevenFiftyGAdapter.fetchDetail(RECIPE_URL);
|
||||
|
||||
expect(result.url).to.equal(RECIPE_URL);
|
||||
expect(result.html).to.include("Poulet au vin jaune");
|
||||
});
|
||||
|
||||
it("throws a RecipeSourceFetchError keyed to 750g, not the underlying generic adapter", async () => {
|
||||
stubFetchHtml("", 404);
|
||||
|
||||
try {
|
||||
await sevenFiftyGAdapter.fetchDetail(RECIPE_URL);
|
||||
expect.fail("expected fetchDetail to throw");
|
||||
} catch (err) {
|
||||
expect(err).to.be.instanceOf(RecipeSourceFetchError);
|
||||
expect((err as RecipeSourceFetchError).sourceKey).to.equal("750g");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("parse", () => {
|
||||
it("repairs a raw unescaped \\r\\n inside a JSON-LD string that would otherwise fail JSON.parse", () => {
|
||||
const parsed = sevenFiftyGAdapter.parse({
|
||||
html: htmlWithRawJsonLd(RAW_RECIPE_JSON_LD),
|
||||
url: RECIPE_URL,
|
||||
});
|
||||
|
||||
expect(parsed.name).to.equal("Poulet au vin jaune et aux morilles");
|
||||
expect(parsed.steps).to.deep.equal([
|
||||
{ description: "Préparez les morilles :\r\nFendez-les en deux.", picture: null },
|
||||
]);
|
||||
});
|
||||
|
||||
it("decodes a double HTML-entity-encoded accented character (é -> é -> &eacute;)", () => {
|
||||
const parsed = sevenFiftyGAdapter.parse({
|
||||
html: htmlWithRawJsonLd(RAW_RECIPE_JSON_LD),
|
||||
url: RECIPE_URL,
|
||||
});
|
||||
|
||||
expect(parsed.steps[0]?.description).to.include("Préparez");
|
||||
});
|
||||
|
||||
it("decodes a plain numeric apostrophe entity in ingredient text", () => {
|
||||
const parsed = sevenFiftyGAdapter.parse({
|
||||
html: htmlWithRawJsonLd(RAW_RECIPE_JSON_LD),
|
||||
url: RECIPE_URL,
|
||||
});
|
||||
|
||||
expect(parsed.ingredients).to.deep.equal([
|
||||
{ rawText: "1 poulet fermier", quantity: null, unit: null, name: "1 poulet fermier" },
|
||||
{ rawText: "Sel 'fin'", quantity: null, unit: null, name: "Sel 'fin'" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("leaves picture/sourceUrl untouched by entity decoding", () => {
|
||||
const parsed = sevenFiftyGAdapter.parse({
|
||||
html: htmlWithRawJsonLd(RAW_RECIPE_JSON_LD),
|
||||
url: RECIPE_URL,
|
||||
});
|
||||
|
||||
expect(parsed.picture).to.equal("https://static.750g.com/images/poulet-vin-jaune.jpg");
|
||||
expect(parsed.sourceUrl).to.equal(RECIPE_URL);
|
||||
});
|
||||
|
||||
it("maps a recipe with no quirks end to end, same as the generic adapter would", () => {
|
||||
const clean = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Recipe",
|
||||
name: "Tarte aux pommes",
|
||||
description: "Une tarte classique.",
|
||||
image: "https://static.750g.com/images/tarte.jpg",
|
||||
recipeYield: 6,
|
||||
recipeIngredient: ["3 pommes", "1 pâte brisée"],
|
||||
recipeInstructions: ["Éplucher les pommes.", "Enfourner 30 minutes."],
|
||||
};
|
||||
const html = `<!doctype html><html><head><script type="application/ld+json">${JSON.stringify(
|
||||
clean,
|
||||
)}</script></head><body></body></html>`;
|
||||
|
||||
const parsed = sevenFiftyGAdapter.parse({ html, url: RECIPE_URL });
|
||||
|
||||
expect(parsed.name).to.equal("Tarte aux pommes");
|
||||
expect(parsed.portions).to.equal(6);
|
||||
expect(parsed.steps).to.deep.equal([
|
||||
{ description: "Éplucher les pommes.", picture: null },
|
||||
{ description: "Enfourner 30 minutes.", picture: null },
|
||||
]);
|
||||
});
|
||||
|
||||
it("throws a RecipeSourceParseError keyed to 750g, not the underlying generic adapter", () => {
|
||||
const html = "<!doctype html><html><body>Pas de JSON-LD ici</body></html>";
|
||||
|
||||
try {
|
||||
sevenFiftyGAdapter.parse({ html, url: RECIPE_URL });
|
||||
expect.fail("expected parse to throw");
|
||||
} catch (err) {
|
||||
expect(err).to.be.instanceOf(RecipeSourceParseError);
|
||||
expect((err as RecipeSourceParseError).sourceKey).to.equal("750g");
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
@ -25,6 +25,24 @@ describe("registerAllRecipeSources", () => {
|
|||
expect(listRecipeSources().map((adapter) => adapter.key)).to.include("theMealDb");
|
||||
});
|
||||
|
||||
it("registers Marmiton into the shared registry", () => {
|
||||
registerAllRecipeSources();
|
||||
|
||||
const marmiton = getRecipeSource("marmiton");
|
||||
expect(marmiton).to.not.be.undefined;
|
||||
expect(marmiton?.name).to.equal("Marmiton");
|
||||
expect(listRecipeSources().map((adapter) => adapter.key)).to.include("marmiton");
|
||||
});
|
||||
|
||||
it("registers 750g into the shared registry", () => {
|
||||
registerAllRecipeSources();
|
||||
|
||||
const sevenFiftyG = getRecipeSource("750g");
|
||||
expect(sevenFiftyG).to.not.be.undefined;
|
||||
expect(sevenFiftyG?.name).to.equal("750g");
|
||||
expect(listRecipeSources().map((adapter) => adapter.key)).to.include("750g");
|
||||
});
|
||||
|
||||
it("does not register the generic JSON-LD adapter — it's not a household-toggleable source in its own right", () => {
|
||||
registerAllRecipeSources();
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue