feat(recipes): migre la detection des tech steps de node-nlp vers un microservice Python spaCy
Remplace TechStepClassifierService's node-nlp (NlpManager) par services/tech-step-intent-service, un microservice FastAPI/spaCy dedie (PhraseMatcher pour le NER par synonymes, textcat pour la classification d'intention). Corpus (TECH_STEP_TRAINING_DATA) toujours possede par apps/api, pousse au service via POST /v1/train a chaque warm-up ; le service ne touche jamais Postgres (meme posture que services/tech-step-llm-worker). Cote apps/api : - intent-service-client.ts : client HTTP vers le nouveau service - tech-step-matcher.ts : delegue NER + intent classification au client, logique pure (splitIntoClauses, seuil/fallback) inchangee - env.ts : INTENT_SERVICE_BASE_URL/INTENT_SERVICE_SECRET (secret requis, service coeur non optionnel) - server.ts : warm-up avec retry/backoff (service Python demarre a part) - scripts/calibrate-tech-step-threshold.ts : recalibration empirique de CONFIDENCE_THRESHOLD contre le jeu d'eval existant - node-nlp retire (package.json, node-nlp.d.ts, model.nlp du .gitignore) docker-compose.yml : nouveau service tech-step-intent-service (pas de port expose, healthcheck, app en depend). CI : job intent-service-test (pytest) + le job test demarre le service en arriere-plan avant la suite Mocha (jamais de mock d'un service interne, cf specs/dev-conventions.md). Verifie : 26/26 tests pytest du service (dont les offsets caracteres exacts de tech-step-matcher.test.ts), lint + build complets du monorepo, smoke test HTTP reel bout en bout. La suite Mocha et docker compose build/up n'ont pas pu etre executes dans cet environnement (pas de Postgres/Docker disponibles ici) — a confirmer via la CI et en local. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
109dde9c7b
commit
18abae7b6a
44 changed files with 3368 additions and 936 deletions
|
|
@ -22,6 +22,14 @@ JWT_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
||||||
# browser, so login "succeeds" but every subsequent request 401s.
|
# browser, so login "succeeds" but every subsequent request 401s.
|
||||||
# COOKIE_SECURE=false
|
# COOKIE_SECURE=false
|
||||||
|
|
||||||
|
# Required — secret shared between "app" and "tech-step-intent-service"
|
||||||
|
# (docker-compose.yml, apps/api/src/config/env.ts). Unlike
|
||||||
|
# INTERNAL_WORKER_SECRET below, there's no "leave it unset" escape hatch:
|
||||||
|
# tech-step-intent-service is a core dependency, not an optional background
|
||||||
|
# job — without it, no recipe step can have its techniques detected at all.
|
||||||
|
# Generate your own the same way as JWT_SECRET above.
|
||||||
|
INTENT_SERVICE_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
||||||
|
|
||||||
# Only needed to run the optional `tech-step-llm-worker` service — shared
|
# Only needed to run the optional `tech-step-llm-worker` service — shared
|
||||||
# between it and "app" (docker-compose.yml). Generate your own the same
|
# between it and "app" (docker-compose.yml). Generate your own the same
|
||||||
# way as JWT_SECRET above; leave both this and the service commented
|
# way as JWT_SECRET above; leave both this and the service commented
|
||||||
|
|
|
||||||
52
.github/workflows/ci.yml
vendored
52
.github/workflows/ci.yml
vendored
|
|
@ -19,9 +19,15 @@ env:
|
||||||
# exercise the success path (matching secret), not just the "unset"
|
# exercise the success path (matching secret), not just the "unset"
|
||||||
# rejection every environment that doesn't set this gets by default.
|
# rejection every environment that doesn't set this gets by default.
|
||||||
INTERNAL_WORKER_SECRET: "ci-only-worker-secret-not-used-anywhere-else-32chars+"
|
INTERNAL_WORKER_SECRET: "ci-only-worker-secret-not-used-anywhere-else-32chars+"
|
||||||
|
# Shared between the `test` job's own uvicorn step (below) and apps/api's
|
||||||
|
# IntentServiceClient — see the `test` job for why this can't be a
|
||||||
|
# `services:` container like postgres above (GitHub Actions can only pull
|
||||||
|
# a published image, not build services/tech-step-intent-service/Dockerfile).
|
||||||
|
INTENT_SERVICE_BASE_URL: "http://localhost:8000"
|
||||||
|
INTENT_SERVICE_SECRET: "ci-only-intent-secret-not-used-anywhere-else-32chars+"
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
# Four independent jobs, no needs: between them — each starts in parallel
|
# Five independent jobs, no needs: between them — each starts in parallel
|
||||||
# and reports as its own check, instead of the previous single chained
|
# and reports as its own check, instead of the previous single chained
|
||||||
# "lint-and-test then e2e" pipeline.
|
# "lint-and-test then e2e" pipeline.
|
||||||
lint:
|
lint:
|
||||||
|
|
@ -65,10 +71,54 @@ jobs:
|
||||||
node-version: 22
|
node-version: 22
|
||||||
cache: pnpm
|
cache: pnpm
|
||||||
|
|
||||||
|
- uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.12"
|
||||||
|
- uses: astral-sh/setup-uv@v3
|
||||||
|
with:
|
||||||
|
enable-cache: true
|
||||||
|
|
||||||
|
# `services:` (like the `postgres` container above) can only pull an
|
||||||
|
# already-published image — it can't build
|
||||||
|
# services/tech-step-intent-service/Dockerfile from this checkout.
|
||||||
|
# Running `uvicorn` as a plain background step instead: it keeps
|
||||||
|
# running for the rest of this job (GitHub Actions steps in one job
|
||||||
|
# share the same runner process tree), and `pnpm --filter api test`
|
||||||
|
# below needs a real instance to talk to per this repo's "never mock
|
||||||
|
# an internal service" test convention — same reasoning as the real
|
||||||
|
# `postgres` container just above, not a mock HTTP server.
|
||||||
|
- name: Install services/tech-step-intent-service
|
||||||
|
working-directory: services/tech-step-intent-service
|
||||||
|
run: uv sync --frozen
|
||||||
|
- name: Start services/tech-step-intent-service in the background
|
||||||
|
working-directory: services/tech-step-intent-service
|
||||||
|
run: |
|
||||||
|
uv run uvicorn intent_service.main:app --host 0.0.0.0 --port 8000 &
|
||||||
|
timeout 60 bash -c 'until curl -sf http://localhost:8000/health > /dev/null; do sleep 1; done'
|
||||||
|
|
||||||
- run: pnpm install --frozen-lockfile
|
- run: pnpm install --frozen-lockfile
|
||||||
- run: pnpm --filter api exec prisma migrate deploy
|
- run: pnpm --filter api exec prisma migrate deploy
|
||||||
- run: pnpm --filter api test
|
- run: pnpm --filter api test
|
||||||
|
|
||||||
|
intent-service-test:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.12"
|
||||||
|
- uses: astral-sh/setup-uv@v3
|
||||||
|
with:
|
||||||
|
enable-cache: true
|
||||||
|
|
||||||
|
- name: Install services/tech-step-intent-service
|
||||||
|
working-directory: services/tech-step-intent-service
|
||||||
|
run: uv sync --frozen
|
||||||
|
- name: Run pytest
|
||||||
|
working-directory: services/tech-step-intent-service
|
||||||
|
run: uv run pytest -q
|
||||||
|
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
|
|
|
||||||
11
.gitignore
vendored
11
.gitignore
vendored
|
|
@ -71,11 +71,12 @@ web_modules/
|
||||||
!.env.example
|
!.env.example
|
||||||
!.env.test.example
|
!.env.test.example
|
||||||
|
|
||||||
# node-nlp's default auto-save file (apps/api/src/lib/recipe-matching/
|
# Python virtualenvs/caches for services/tech-step-intent-service (this repo
|
||||||
# tech-step-matcher.ts explicitly disables autoSave/autoLoad, but this is a
|
# is otherwise all-Node — see that service's own .gitignore for the rest;
|
||||||
# belt-and-suspenders guard against it ever reappearing — a stale trained
|
# duplicated here too since some tooling only honors the repo-root file).
|
||||||
# model on disk must never silently shadow TECH_STEP_TRAINING_DATA).
|
services/tech-step-intent-service/.venv/
|
||||||
model.nlp
|
services/tech-step-intent-service/__pycache__/
|
||||||
|
services/tech-step-intent-service/.pytest_cache/
|
||||||
|
|
||||||
# parcel-bundler cache (https://parceljs.org/)
|
# parcel-bundler cache (https://parceljs.org/)
|
||||||
.cache
|
.cache
|
||||||
|
|
|
||||||
30
README.md
30
README.md
|
|
@ -44,6 +44,8 @@ runtime Node pur (Docker, pas de transpilation à la volée), voir la note dans
|
||||||
- Node.js 22 (voir `.nvmrc`)
|
- Node.js 22 (voir `.nvmrc`)
|
||||||
- pnpm 10 (`corepack enable` puis `corepack use pnpm@10.12.4`, ou installation manuelle)
|
- pnpm 10 (`corepack enable` puis `corepack use pnpm@10.12.4`, ou installation manuelle)
|
||||||
- Docker (pour Postgres en local)
|
- Docker (pour Postgres en local)
|
||||||
|
- Python 3.12+ et [`uv`](https://docs.astral.sh/uv/) (pour
|
||||||
|
`services/tech-step-intent-service` en dev natif — requis, voir plus bas)
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
|
|
@ -100,6 +102,15 @@ pnpm --filter api exec prisma migrate dev
|
||||||
# techniques...) — automatique après `prisma migrate reset`, sinon à la main :
|
# techniques...) — automatique après `prisma migrate reset`, sinon à la main :
|
||||||
pnpm --filter api prisma:seed
|
pnpm --filter api prisma:seed
|
||||||
|
|
||||||
|
# Microservice de détection des techniques (spaCy) — requis, `pnpm dev:api`
|
||||||
|
# ne peut plus détecter aucune technique de cuisine sans lui (voir son
|
||||||
|
# propre README pour le détail)
|
||||||
|
cd services/tech-step-intent-service
|
||||||
|
uv sync
|
||||||
|
cp .env.example .env # édite-le : même INTENT_SERVICE_SECRET que apps/api/.env
|
||||||
|
uv run uvicorn intent_service.main:app --reload --port 8000
|
||||||
|
cd ../..
|
||||||
|
|
||||||
# Backend (http://localhost:3000)
|
# Backend (http://localhost:3000)
|
||||||
pnpm dev:api
|
pnpm dev:api
|
||||||
|
|
||||||
|
|
@ -137,7 +148,7 @@ pnpm --filter web cy:run:component # tests de composant UI isolés (Cypress com
|
||||||
pnpm build # build de tous les workspaces
|
pnpm build # build de tous les workspaces
|
||||||
```
|
```
|
||||||
|
|
||||||
La CI GitHub Actions (`.github/workflows/ci.yml`) exécute quatre jobs indépendants (`lint`, `test`, `build`, `e2e` — ce dernier lance aussi `cy:run:component`) en parallèle, sur chaque push (toutes branches) et sur chaque PR vers `main` — pas de chaînage entre eux, chacun apparaît comme son propre check. Voir aussi [Déploiement](#déploiement) pour le pipeline de release (`.github/workflows/release.yml`).
|
La CI GitHub Actions (`.github/workflows/ci.yml`) exécute cinq jobs indépendants (`lint`, `test`, `intent-service-test`, `build`, `e2e` — ce dernier lance aussi `cy:run:component`) en parallèle, sur chaque push (toutes branches) et sur chaque PR vers `main` — pas de chaînage entre eux, chacun apparaît comme son propre check. `test` démarre `services/tech-step-intent-service` en arrière-plan (voir ce fichier) puisque la suite Mocha ne mocke jamais un service interne. Voir aussi [Déploiement](#déploiement) pour le pipeline de release (`.github/workflows/release.yml`).
|
||||||
|
|
||||||
### Base de test isolée de la base de dev (`apps/api`)
|
### Base de test isolée de la base de dev (`apps/api`)
|
||||||
|
|
||||||
|
|
@ -158,6 +169,13 @@ Un garde-fou (`assertRunningAgainstTestDatabase()`) refuse d'exécuter
|
||||||
`resetDatabase()` si `DATABASE_URL` ne contient ni `"test"` ni `"ci"` — la
|
`resetDatabase()` si `DATABASE_URL` ne contient ni `"test"` ni `"ci"` — la
|
||||||
seule base qu'il doit rejeter est ta vraie base de dev.
|
seule base qu'il doit rejeter est ta vraie base de dev.
|
||||||
|
|
||||||
|
`services/tech-step-intent-service` doit aussi tourner en local avant
|
||||||
|
`pnpm --filter api test` — les tests touchant `tech-step-matcher.ts` passent
|
||||||
|
par le vrai service (jamais un mock, voir
|
||||||
|
[specs/dev-conventions.md](specs/dev-conventions.md)) et échouent avec une
|
||||||
|
erreur de connexion, pas une assertion utile, s'il n'est pas démarré. Voir la
|
||||||
|
section [Développement](#développement) ci-dessus.
|
||||||
|
|
||||||
## Déploiement
|
## Déploiement
|
||||||
|
|
||||||
Une seule image Docker (`apps/api/Dockerfile`) sert à la fois l'API et le frontend
|
Une seule image Docker (`apps/api/Dockerfile`) sert à la fois l'API et le frontend
|
||||||
|
|
@ -180,10 +198,12 @@ synchronisation de la table `sources` depuis le registre d'adaptateurs de code
|
||||||
puis `node dist/server.js`. Les trois étapes sont sûres/idempotentes à
|
puis `node dist/server.js`. Les trois étapes sont sûres/idempotentes à
|
||||||
répéter à chaque redémarrage du conteneur.
|
répéter à chaque redémarrage du conteneur.
|
||||||
|
|
||||||
`docker-compose.yml` ne définit donc que deux services : `postgres` et `app` (un
|
Le duo `postgres`/`app` de `docker-compose.yml` n'expose donc qu'un seul port
|
||||||
seul port, `APP_PORT`, défaut `3000` — plus de `WEB_PORT`/`CORS_ORIGIN` à
|
applicatif, `APP_PORT` (défaut `3000`) — plus de `WEB_PORT`/`CORS_ORIGIN` à
|
||||||
coordonner entre deux origines, le frontend et l'API sont désormais servis depuis
|
coordonner entre deux origines, le frontend et l'API sont désormais servis
|
||||||
la même origine).
|
depuis la même origine. Les deux autres services du fichier
|
||||||
|
(`tech-step-intent-service`, `tech-step-llm-worker`) n'exposent eux aucun port
|
||||||
|
au host — voir leurs propres README pour leur rôle.
|
||||||
|
|
||||||
**Pas de registre d'image** dans cette configuration : l'instance **Portainer** de
|
**Pas de registre d'image** dans cette configuration : l'instance **Portainer** de
|
||||||
production est reliée directement au dépôt Git et reconstruit elle-même
|
production est reliée directement au dépôt Git et reconstruit elle-même
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,13 @@ JWT_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
||||||
# JWT_EXPIRES_IN=7d
|
# JWT_EXPIRES_IN=7d
|
||||||
# AUTH_COOKIE_NAME=session
|
# AUTH_COOKIE_NAME=session
|
||||||
# CORS_ORIGIN=http://localhost:5173
|
# CORS_ORIGIN=http://localhost:5173
|
||||||
|
# INTENT_SERVICE_BASE_URL=http://localhost:8000
|
||||||
|
|
||||||
|
# Required — services/tech-step-intent-service must be running locally (see
|
||||||
|
# that service's own README) for any recipe save/preview to detect
|
||||||
|
# techniques at all. Must match that service's own INTENT_SERVICE_SECRET.
|
||||||
|
# Generate your own the same way as JWT_SECRET above.
|
||||||
|
INTENT_SERVICE_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
||||||
|
|
||||||
# Only needed if you're running services/tech-step-llm-worker locally —
|
# Only needed if you're running services/tech-step-llm-worker locally —
|
||||||
# every /internal/tech-steps/* request is rejected outright while unset.
|
# every /internal/tech-steps/* request is rejected outright while unset.
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,15 @@ DATABASE_URL="postgresql://changeme:changeme@localhost:5432/batchcooking_test?sc
|
||||||
# node -e "console.log(require('crypto').randomBytes(48).toString('hex'))"
|
# node -e "console.log(require('crypto').randomBytes(48).toString('hex'))"
|
||||||
JWT_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
JWT_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
||||||
|
|
||||||
|
# Required — the Mocha suite exercises the real techStepClassifier, which
|
||||||
|
# now round-trips over HTTP to services/tech-step-intent-service (no mocks
|
||||||
|
# of internal services, per this repo's test conventions). Start that
|
||||||
|
# service locally first (see its own README) with a matching
|
||||||
|
# INTENT_SERVICE_SECRET, or every test touching tech-step-matcher.ts fails
|
||||||
|
# with a connection error rather than a useful assertion failure.
|
||||||
|
INTENT_SERVICE_BASE_URL=http://localhost:8000
|
||||||
|
INTENT_SERVICE_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
||||||
|
|
||||||
# Optional — only needed to exercise tech-step-worker.routes.test.ts's
|
# Optional — only needed to exercise tech-step-worker.routes.test.ts's
|
||||||
# success path (a request with a matching secret); every other test runs
|
# success path (a request with a matching secret); every other test runs
|
||||||
# fine without it. Any value at least 32 chars works locally.
|
# fine without it. Any value at least 32 chars works locally.
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,6 @@
|
||||||
"dotenv": "^16.4.5",
|
"dotenv": "^16.4.5",
|
||||||
"express": "^4.21.1",
|
"express": "^4.21.1",
|
||||||
"jsonwebtoken": "^9.0.3",
|
"jsonwebtoken": "^9.0.3",
|
||||||
"node-nlp": "4.27.0",
|
|
||||||
"prisma": "^5.22.0",
|
"prisma": "^5.22.0",
|
||||||
"zod": "^3.23.8"
|
"zod": "^3.23.8"
|
||||||
},
|
},
|
||||||
|
|
|
||||||
|
|
@ -627,13 +627,13 @@ model RecipeIngredient {
|
||||||
/// Matching a step's free text against these (`tech-step-matcher.ts`'s
|
/// Matching a step's free text against these (`tech-step-matcher.ts`'s
|
||||||
/// `TechStepClassifierService`) used to go through a DB-backed
|
/// `TechStepClassifierService`) used to go through a DB-backed
|
||||||
/// `TechStepMapping` table of per-locale regex expressions — replaced with
|
/// `TechStepMapping` table of per-locale regex expressions — replaced with
|
||||||
/// a node-nlp model trained from in-code data
|
/// a spaCy-based model (`services/tech-step-intent-service`) trained from
|
||||||
/// (`tech-step-training-data.ts`) once regexes turned out unable to
|
/// in-code data (`tech-step-training-data.ts`) once regexes turned out
|
||||||
/// generalize past their own literal vocabulary. Nothing queries/edits
|
/// unable to generalize past their own literal vocabulary. Nothing
|
||||||
/// that matching data at runtime anymore (it only ever feeds the
|
/// queries/edits that matching data at runtime anymore (it only ever feeds
|
||||||
/// classifier's one-time training pass), so it no longer needs a table of
|
/// that service's one-time training pass), so it no longer needs a table
|
||||||
/// its own — this row now only exists to be a stable id/key other tables
|
/// of its own — this row now only exists to be a stable id/key other
|
||||||
/// (`StepTechStep`) reference.
|
/// tables (`StepTechStep`) reference.
|
||||||
model TechStep {
|
model TechStep {
|
||||||
id Int @id @default(autoincrement())
|
id Int @id @default(autoincrement())
|
||||||
key String @unique
|
key String @unique
|
||||||
|
|
|
||||||
|
|
@ -71,6 +71,25 @@ const envSchema = z.object({
|
||||||
* fails closed rather than open if a real deployment forgets to set it.
|
* fails closed rather than open if a real deployment forgets to set it.
|
||||||
*/
|
*/
|
||||||
INTERNAL_WORKER_SECRET: z.string().min(32).optional(),
|
INTERNAL_WORKER_SECRET: z.string().min(32).optional(),
|
||||||
|
/**
|
||||||
|
* Base URL of `services/tech-step-intent-service` (the spaCy-based
|
||||||
|
* microservice `TechStepClassifierService` delegates NER + intent
|
||||||
|
* classification to, see `lib/recipe-matching/intent-service-client.ts`).
|
||||||
|
* Has a default (unlike `DATABASE_URL`/secrets below) since it isn't
|
||||||
|
* secret and dev natively runs it on a fixed local port — Docker Compose
|
||||||
|
* overrides it to the compose network's service name.
|
||||||
|
*/
|
||||||
|
INTENT_SERVICE_BASE_URL: z.string().url().default("http://localhost:8000"),
|
||||||
|
/**
|
||||||
|
* Shared secret sent as an `X-Intent-Service-Secret` header on every call
|
||||||
|
* to `services/tech-step-intent-service`. Unlike `INTERNAL_WORKER_SECRET`
|
||||||
|
* above, **required, no `.optional()`** — that service is a core
|
||||||
|
* dependency (recipe save/preview can no longer detect any technique
|
||||||
|
* without it), not an optional background job; an environment that
|
||||||
|
* forgets to set this must fail loudly at startup, not silently run with
|
||||||
|
* every technique detection request failing one at a time.
|
||||||
|
*/
|
||||||
|
INTENT_SERVICE_SECRET: z.string().min(32, "INTENT_SERVICE_SECRET must be at least 32 characters"),
|
||||||
});
|
});
|
||||||
|
|
||||||
/** Parsed, validated environment — import this instead of reading `process.env` directly anywhere else. */
|
/** Parsed, validated environment — import this instead of reading `process.env` directly anywhere else. */
|
||||||
|
|
|
||||||
124
apps/api/src/lib/recipe-matching/intent-service-client.ts
Normal file
124
apps/api/src/lib/recipe-matching/intent-service-client.ts
Normal file
|
|
@ -0,0 +1,124 @@
|
||||||
|
import { env } from "../../config/env.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Thin fetch wrapper around `services/tech-step-intent-service`'s HTTP
|
||||||
|
* contract (`/v1/train`, `/v1/process`) — the microservice
|
||||||
|
* {@link TechStepClassifierService} (`tech-step-matcher.ts`) delegates NER +
|
||||||
|
* intent classification to, in place of the `node-nlp` `NlpManager` it used
|
||||||
|
* to own directly. See that service's own README for the full contract and
|
||||||
|
* why it never touches Postgres itself.
|
||||||
|
*
|
||||||
|
* Authenticated with `INTENT_SERVICE_SECRET` — the inverse direction of
|
||||||
|
* `requireInternalWorker`'s `INTERNAL_WORKER_SECRET` (this time `apps/api`
|
||||||
|
* is the caller, not the callee), but the same "one flat shared secret"
|
||||||
|
* shape.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** One candidate technique mention the service's `PhraseMatcher` found — offsets `[start, end)`, same convention as `String.prototype.slice`. Mirrors `EntityPayload` (Python `schemas.py`). */
|
||||||
|
export interface IntentServiceEntity {
|
||||||
|
uid: string;
|
||||||
|
start: number;
|
||||||
|
end: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The full result of a `POST /v1/process` call — mirrors `ProcessResponse` (Python `schemas.py`). `intent` is `null` only when `locale` was never trained or `text` is blank; otherwise always a real `uid` (the Python service's `textcat` has no "None" sentinel, unlike node-nlp — see that service's README). */
|
||||||
|
export interface IntentServiceProcessResult {
|
||||||
|
entities: IntentServiceEntity[];
|
||||||
|
intent: string | null;
|
||||||
|
score: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One technique's training data for one locale, as sent to `POST /v1/train` — mirrors `TrainEntryPayload` (Python `schemas.py`), itself shaped after `TechStepLocaleTrainingData` (`tech-step-training-data.ts`). */
|
||||||
|
export interface IntentServiceTrainEntry {
|
||||||
|
uid: string;
|
||||||
|
synonyms: string[];
|
||||||
|
utterances: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Client for `services/tech-step-intent-service` — a real class (not a
|
||||||
|
* plain object of functions) per this repo's service-style-logic
|
||||||
|
* convention, even though it holds no state of its own: it's used as the
|
||||||
|
* one shared {@link intentServiceClient} singleton below, same reasoning as
|
||||||
|
* `TechStepClassifierService` itself.
|
||||||
|
*/
|
||||||
|
export class IntentServiceClient {
|
||||||
|
/**
|
||||||
|
* Performs a JSON request against the intent service and returns the
|
||||||
|
* parsed body.
|
||||||
|
*
|
||||||
|
* @throws {Error} if the response status is not in the 2xx range, or the
|
||||||
|
* request itself fails (network error, service down) — left as a plain
|
||||||
|
* `Error` rather than a typed `HttpError`: this is an internal
|
||||||
|
* service-to-service call, not a request `apps/api`'s own HTTP layer
|
||||||
|
* needs to map to a client-facing status code (see
|
||||||
|
* `TechStepClassifierService.warmUp`'s retry in `server.ts` for how a
|
||||||
|
* failure here is actually handled).
|
||||||
|
*/
|
||||||
|
private async _request<TResponseBody>(
|
||||||
|
path: string,
|
||||||
|
init: RequestInit = {},
|
||||||
|
): Promise<TResponseBody> {
|
||||||
|
try {
|
||||||
|
const response = await fetch(`${env.INTENT_SERVICE_BASE_URL}${path}`, {
|
||||||
|
...init,
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"X-Intent-Service-Secret": env.INTENT_SERVICE_SECRET,
|
||||||
|
...init.headers,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
const body = await response.text().catch(() => "");
|
||||||
|
throw new Error(`${init.method ?? "GET"} ${path} failed: ${response.status} ${body}`);
|
||||||
|
}
|
||||||
|
return (await response.json()) as TResponseBody;
|
||||||
|
} catch (err) {
|
||||||
|
// Rethrown as-is — every caller (`TechStepClassifierService`) already
|
||||||
|
// wraps its own `await`s per the repo's try/catch convention; this is
|
||||||
|
// just where the `await` itself has to sit inside one.
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* (Re)trains the intent service's pipeline for `locale` from `entries` —
|
||||||
|
* called once per locale by `TechStepClassifierService._train`, itself
|
||||||
|
* memoized so this only ever runs once per server process (see that
|
||||||
|
* method's own doc comment). Reconstructs the whole pipeline server-side,
|
||||||
|
* never a partial/incremental update — same "always retrains fresh from
|
||||||
|
* the one source of truth" posture the old in-process `NlpManager` had.
|
||||||
|
*/
|
||||||
|
public async train(locale: string, entries: IntentServiceTrainEntry[]): Promise<void> {
|
||||||
|
try {
|
||||||
|
await this._request("/v1/train", {
|
||||||
|
method: "POST",
|
||||||
|
body: JSON.stringify({ locale, entries }),
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Equivalent to the old `NlpManager.process(locale, text)` — returns every
|
||||||
|
* candidate technique mention (NER) plus the intent classifier's verdict
|
||||||
|
* for `text` as a whole, whether `text` is a full step description or a
|
||||||
|
* single clause `TechStepClassifierService` already cut out of one (this
|
||||||
|
* service doesn't know or care which, exactly like `NlpManager` before
|
||||||
|
* it).
|
||||||
|
*/
|
||||||
|
public async process(locale: string, text: string): Promise<IntentServiceProcessResult> {
|
||||||
|
try {
|
||||||
|
return await this._request("/v1/process", {
|
||||||
|
method: "POST",
|
||||||
|
body: JSON.stringify({ locale, text }),
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Single shared instance — stateless, no reason for more than one (same reasoning as `techStepClassifier`/`prisma`). */
|
||||||
|
export const intentServiceClient = new IntentServiceClient();
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
import { NlpManager } from "node-nlp";
|
|
||||||
import { prisma } from "../../db/prisma.js";
|
import { prisma } from "../../db/prisma.js";
|
||||||
|
import { intentServiceClient } from "./intent-service-client.js";
|
||||||
import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
|
import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
@ -15,25 +15,26 @@ import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
|
||||||
* generalize past its own vocabulary — a step describing melting butter as
|
* generalize past its own vocabulary — a step describing melting butter as
|
||||||
* "jusqu'à ce que le beurre ait disparu dans la poêle" mentions no verb any
|
* "jusqu'à ce que le beurre ait disparu dans la poêle" mentions no verb any
|
||||||
* regex could anchor on, yet unmistakably *means* `melt`. Replaced with a
|
* regex could anchor on, yet unmistakably *means* `melt`. Replaced with a
|
||||||
* small hybrid pipeline built on `node-nlp` ({@link TechStepClassifierService}):
|
* small hybrid pipeline (originally built on `node-nlp`, now delegated to
|
||||||
|
* `services/tech-step-intent-service` — a spaCy-based microservice, see
|
||||||
|
* {@link IntentServiceClient} and that service's own README):
|
||||||
*
|
*
|
||||||
* 1. **NER** (node-nlp enum entities, `synonyms` in `TECH_STEP_TRAINING_DATA`)
|
* 1. **NER** (the intent service's `PhraseMatcher`, built from `synonyms` in
|
||||||
* finds every *candidate* technique mention in the whole
|
* `TECH_STEP_TRAINING_DATA`) finds every *candidate* technique mention in
|
||||||
* description, each with its exact character span — mechanically the
|
* the whole description, each with its exact character span —
|
||||||
* same job the old regexes did, just as flat synonym lists instead of
|
* mechanically the same job the old regexes did, just as flat synonym
|
||||||
* hand-written patterns (node-nlp's own stemmer/fuzzy matching already
|
* lists instead of hand-written patterns. This step alone is *not* the
|
||||||
* covers minor conjugation/typo variance the regexes had to enumerate
|
* final answer — see step 3.
|
||||||
* by hand). This step alone is *not* the final answer — see step 3.
|
|
||||||
* 2. The description is cut into clauses around those candidate spans
|
* 2. The description is cut into clauses around those candidate spans
|
||||||
* ({@link splitIntoClauses}) — a step naming two techniques ("Dans une
|
* ({@link splitIntoClauses}) — a step naming two techniques ("Dans une
|
||||||
* poêle chaude, faire chauffer une noix de beurre" is both `preheat`
|
* poêle chaude, faire chauffer une noix de beurre" is both `preheat`
|
||||||
* and `melt`) needs each judged on its own surrounding context, not the
|
* and `melt`) needs each judged on its own surrounding context, not the
|
||||||
* whole step lumped into one classification.
|
* whole step lumped into one classification.
|
||||||
* 3. **NLP intent classification** (node-nlp's `NlpManager`, trained on
|
* 3. **NLP intent classification** (the intent service's `textcat`, trained
|
||||||
* `TECH_STEP_TRAINING_DATA`'s `utterances`) then classifies each clause
|
* on `TECH_STEP_TRAINING_DATA`'s `utterances`) then classifies each
|
||||||
* on its own — this is what actually delivers "meaning, not keywords":
|
* clause on its own — this is what actually delivers "meaning, not
|
||||||
* the classifier was deliberately trained on paraphrases that never use
|
* keywords": the classifier was deliberately trained on paraphrases that
|
||||||
* the technique's own verb (e.g. "jusqu'à ce que le beurre ait
|
* never use the technique's own verb (e.g. "jusqu'à ce que le beurre ait
|
||||||
* disparu" for `melt`), so a clause reaching it gets labeled by what it
|
* disparu" for `melt`), so a clause reaching it gets labeled by what it
|
||||||
* was trained to recognize as *meaning* a technique, not by which
|
* was trained to recognize as *meaning* a technique, not by which
|
||||||
* literal word the NER step happened to anchor on. The NER-implied
|
* literal word the NER step happened to anchor on. The NER-implied
|
||||||
|
|
@ -50,11 +51,11 @@ import { TECH_STEP_TRAINING_DATA } from "./tech-step-training-data.js";
|
||||||
* `normalizeText` and {@link splitIntoClauses} are pure (no DB/model
|
* `normalizeText` and {@link splitIntoClauses} are pure (no DB/model
|
||||||
* access) so they stay unit-testable in isolation (see
|
* access) so they stay unit-testable in isolation (see
|
||||||
* `test/tech-step-matcher.test.ts`); the classifier itself needs a one-time
|
* `test/tech-step-matcher.test.ts`); the classifier itself needs a one-time
|
||||||
* training pass (`_ensureTrained`, node-nlp's `NlpManager.train()`) plus a
|
* training pass (`_ensureTrained`, a `POST /v1/train` call per locale to the
|
||||||
* `TechStep.key -> id` lookup from the DB, both memoized on the shared
|
* intent service) plus a `TechStep.key -> id` lookup from the DB, both
|
||||||
* {@link techStepClassifier} singleton rather than repeated per call —
|
* memoized on the shared {@link techStepClassifier} singleton rather than
|
||||||
* training is the expensive part (a few hundred ms for this corpus), never
|
* repeated per call — training is the expensive part, never worth redoing
|
||||||
* worth redoing per request let alone per step.
|
* per request let alone per step.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
@ -273,70 +274,36 @@ export interface TechStepClauseClassification {
|
||||||
end: number;
|
end: number;
|
||||||
/** The clause's NER anchor's own implied technique `uid`, if it had one — same as `TechStepClause.anchor.uid`. */
|
/** The clause's NER anchor's own implied technique `uid`, if it had one — same as `TechStepClause.anchor.uid`. */
|
||||||
anchorUid: string | null;
|
anchorUid: string | null;
|
||||||
/** The intent classifier's own top guess for this clause, whatever its score — `null` only when it returned node-nlp's `"None"` sentinel. Unlike {@link TechStepMatch}, never silently replaced by the anchor's uid — the whole point of this type is to expose the classifier's raw opinion, confident or not. */
|
/** The intent classifier's own top guess for this clause, whatever its score — `null` only when the intent service had nothing trained for `locale`, or the clause text was blank. Unlike {@link TechStepMatch}, never silently replaced by the anchor's uid — the whole point of this type is to expose the classifier's raw opinion, confident or not. */
|
||||||
intentUid: string | null;
|
intentUid: string | null;
|
||||||
/** The intent classifier's own confidence for `intentUid` — `0` when `intentUid` is `null` (nothing to have a score about). */
|
/** The intent classifier's own confidence for `intentUid` — `0` when `intentUid` is `null` (nothing to have a score about). */
|
||||||
score: number;
|
score: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Trains and owns the `node-nlp` model behind {@link matchTechStepSpans} —
|
* Owns the trained state behind {@link matchTechStepSpans} — a real class
|
||||||
* a real class (not a plain object of functions) per this repo's
|
* (not a plain object of functions) per this repo's service-style-logic
|
||||||
* service-style-logic convention, even though it's only ever used as the
|
* convention, even though it's only ever used as the one shared
|
||||||
* one shared {@link techStepClassifier} singleton below: it holds real
|
* {@link techStepClassifier} singleton below: it holds real state (the
|
||||||
* state (the trained model, the memoized training/lookup promises), not
|
* memoized training/lookup promises), not just grouped stateless helpers.
|
||||||
* just grouped stateless helpers.
|
* The actual NER/intent-classification model lives in
|
||||||
|
* `services/tech-step-intent-service` (a separate process) — this class's
|
||||||
|
* own state is just what it needs to talk to that service correctly
|
||||||
|
* (whether training has been kicked off yet, and the `TechStep.key -> id`
|
||||||
|
* lookup that service's `uid`s must still be resolved through).
|
||||||
*/
|
*/
|
||||||
export class TechStepClassifierService {
|
export class TechStepClassifierService {
|
||||||
/** node-nlp's manager — both NER (enum entities) and NLP (intent classification) live on the same instance, trained together. */
|
|
||||||
private readonly _manager: NlpManager;
|
|
||||||
/** Memoized training pass — `undefined` until the first call starts it, after which every caller (concurrent or not) awaits the same promise rather than retraining. */
|
/** Memoized training pass — `undefined` until the first call starts it, after which every caller (concurrent or not) awaits the same promise rather than retraining. */
|
||||||
private _trained: Promise<void> | undefined;
|
private _trained: Promise<void> | undefined;
|
||||||
/** Memoized `TechStep.key -> id` lookup — training data only knows techniques by their stable `uid`/`key`, resolved to the real DB id once, alongside training. */
|
/** Memoized `TechStep.key -> id` lookup — training data only knows techniques by their stable `uid`/`key`, resolved to the real DB id once, alongside training. */
|
||||||
private _techStepIdByUid: Map<string, number> | undefined;
|
private _techStepIdByUid: Map<string, number> | undefined;
|
||||||
|
|
||||||
public constructor() {
|
|
||||||
this._manager = new NlpManager({
|
|
||||||
languages: ["fr", "en"],
|
|
||||||
forceNER: true,
|
|
||||||
nlu: { log: false },
|
|
||||||
// node-nlp's enum-entity NER defaults to a fuzzy (Levenshtein-based)
|
|
||||||
// 0.8 accuracy threshold — loose enough that e.g. "faire" (the
|
|
||||||
// generic French helper verb in almost every recipe step) fuzzy-
|
|
||||||
// matches `fry`'s synonym "frire" at 0.80, a false positive found
|
|
||||||
// while tuning this against the real training corpus. `1` (exact,
|
|
||||||
// after node-nlp's own case/accent/stemming normalization — real
|
|
||||||
// conjugation variance is still covered by listing each form in
|
|
||||||
// `tech-step-training-data.ts`) removed it without losing any real
|
|
||||||
// match. Precision matters more than recall for this stage — NER
|
|
||||||
// only proposes candidate split points, `_classifyClause`'s trained
|
|
||||||
// model (not fuzzy string distance) is what actually has to be
|
|
||||||
// right.
|
|
||||||
ner: { threshold: 1 },
|
|
||||||
// node-nlp defaults to `autoSave`/`autoLoad: true` — silently
|
|
||||||
// persisting the trained model to a `model.nlp` file in the process's
|
|
||||||
// cwd, and *loading from that file instead of retraining* the next
|
|
||||||
// time a manager is constructed, if the file already exists. Found
|
|
||||||
// this the hard way: a stray `model.nlp` appeared at the repo root
|
|
||||||
// after running this locally. That's the opposite of what this
|
|
||||||
// service wants — `TECH_STEP_TRAINING_DATA` in code is the single
|
|
||||||
// source of truth this always trains fresh from (see this file's own
|
|
||||||
// doc comment) — a stale on-disk model silently shadowing a
|
|
||||||
// corpus/threshold update would be a nasty, hard-to-notice class of
|
|
||||||
// bug. Both off; nothing here should ever touch disk.
|
|
||||||
autoSave: false,
|
|
||||||
autoLoad: false,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Forces training plus node-nlp's own one-time lazy setup (loading its
|
* Forces training (a `POST /v1/train` call per locale to
|
||||||
* bundled per-language stemmers/tokenizers on the *first* real
|
* `services/tech-step-intent-service`) to happen now, synchronously with
|
||||||
* `NlpManager.process()` call takes a few seconds by itself, separate
|
* server startup (see `server.ts`, which also retries this against a
|
||||||
* from and much slower than the ~40ms `train()` pass — measured against
|
* not-yet-ready intent service), rather than stalling whichever request
|
||||||
* this corpus while tuning the pipeline) to happen now, synchronously
|
* happens to be first to save/preview a recipe.
|
||||||
* with server startup (see `server.ts`), rather than stalling whichever
|
|
||||||
* request happens to be first to save/preview a recipe.
|
|
||||||
*/
|
*/
|
||||||
public async warmUp(): Promise<void> {
|
public async warmUp(): Promise<void> {
|
||||||
try {
|
try {
|
||||||
|
|
@ -363,22 +330,18 @@ export class TechStepClassifierService {
|
||||||
await this._ensureTrained();
|
await this._ensureTrained();
|
||||||
if (description.trim().length === 0) return [];
|
if (description.trim().length === 0) return [];
|
||||||
|
|
||||||
const nerResult = await this._manager.process(locale, description);
|
// The intent service only ever returns enum-style candidates (its own
|
||||||
const candidates: TechniqueCandidate[] = nerResult.entities
|
// `PhraseMatcher`, built solely from `TECH_STEP_TRAINING_DATA`'s
|
||||||
// node-nlp's language plugins also auto-extract their own built-in
|
// `synonyms`) — unlike node-nlp, it never mixes in built-in
|
||||||
// entities (numbers, durations, dates…) alongside the enum
|
// numbers/durations/dates entities, so no `type === "enum"` filter is
|
||||||
// entities `_train` registered from `TECH_STEP_TRAINING_DATA` —
|
// needed here anymore. Its `start`/`end` are already `[start, end)`
|
||||||
// `type === "enum"` is what tells the two apart; without this
|
// (matching `String.prototype.slice`), unlike node-nlp's inclusive
|
||||||
// filter a step like "10 minutes" would hand `splitIntoClauses` a
|
// `end` — no `+ 1` needed either.
|
||||||
// bogus "duration" candidate that resolves to no real technique.
|
const nerResult = await intentServiceClient.process(locale, description);
|
||||||
.filter((entity) => entity.type === "enum")
|
const candidates: TechniqueCandidate[] = nerResult.entities.map((entity) => ({
|
||||||
.map((entity) => ({
|
uid: entity.uid,
|
||||||
uid: entity.entity,
|
|
||||||
start: entity.start,
|
start: entity.start,
|
||||||
// node-nlp's own `end` is inclusive (verified against a real
|
end: entity.end,
|
||||||
// trained model) — `+ 1` converts to this module's `[start, end)`
|
|
||||||
// convention, matching `String.prototype.slice`.
|
|
||||||
end: entity.end + 1,
|
|
||||||
}));
|
}));
|
||||||
|
|
||||||
const clauses = splitIntoClauses(description, candidates);
|
const clauses = splitIntoClauses(description, candidates);
|
||||||
|
|
@ -431,13 +394,11 @@ export class TechStepClassifierService {
|
||||||
await this._ensureTrained();
|
await this._ensureTrained();
|
||||||
if (description.trim().length === 0) return [];
|
if (description.trim().length === 0) return [];
|
||||||
|
|
||||||
const nerResult = await this._manager.process(locale, description);
|
const nerResult = await intentServiceClient.process(locale, description);
|
||||||
const candidates: TechniqueCandidate[] = nerResult.entities
|
const candidates: TechniqueCandidate[] = nerResult.entities.map((entity) => ({
|
||||||
.filter((entity) => entity.type === "enum")
|
uid: entity.uid,
|
||||||
.map((entity) => ({
|
|
||||||
uid: entity.entity,
|
|
||||||
start: entity.start,
|
start: entity.start,
|
||||||
end: entity.end + 1,
|
end: entity.end,
|
||||||
}));
|
}));
|
||||||
|
|
||||||
const clauses = splitIntoClauses(description, candidates);
|
const clauses = splitIntoClauses(description, candidates);
|
||||||
|
|
@ -456,15 +417,14 @@ export class TechStepClassifierService {
|
||||||
});
|
});
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
const result = await this._manager.process(locale, clauseText);
|
const result = await intentServiceClient.process(locale, clauseText);
|
||||||
const intentUid = result.intent !== "None" ? result.intent : null;
|
|
||||||
results.push({
|
results.push({
|
||||||
clauseText,
|
clauseText,
|
||||||
start: clause.start,
|
start: clause.start,
|
||||||
end: clause.end,
|
end: clause.end,
|
||||||
anchorUid,
|
anchorUid,
|
||||||
intentUid,
|
intentUid: result.intent,
|
||||||
score: intentUid === null ? 0 : result.score,
|
score: result.intent === null ? 0 : result.score,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
return results;
|
return results;
|
||||||
|
|
@ -506,8 +466,8 @@ export class TechStepClassifierService {
|
||||||
const clauseText = description.slice(clause.start, clause.end).trim();
|
const clauseText = description.slice(clause.start, clause.end).trim();
|
||||||
if (clauseText.length === 0) return clause.anchor?.uid ?? null;
|
if (clauseText.length === 0) return clause.anchor?.uid ?? null;
|
||||||
|
|
||||||
const result = await this._manager.process(locale, clauseText);
|
const result = await intentServiceClient.process(locale, clauseText);
|
||||||
if (result.intent !== "None" && result.score >= CONFIDENCE_THRESHOLD) {
|
if (result.intent !== null && result.score >= CONFIDENCE_THRESHOLD) {
|
||||||
return result.intent;
|
return result.intent;
|
||||||
}
|
}
|
||||||
return clause.anchor?.uid ?? null;
|
return clause.anchor?.uid ?? null;
|
||||||
|
|
@ -517,11 +477,12 @@ export class TechStepClassifierService {
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Trains `_manager` from {@link TECH_STEP_TRAINING_DATA} and resolves the
|
* Trains `services/tech-step-intent-service` from
|
||||||
* `uid -> TechStep.id` lookup, both exactly once — memoized on
|
* {@link TECH_STEP_TRAINING_DATA} and resolves the `uid -> TechStep.id`
|
||||||
* `_trained` so a burst of concurrent calls (several steps of the same
|
* lookup, both exactly once — memoized on `_trained` so a burst of
|
||||||
* recipe save, awaited via the same event loop tick) all await the one
|
* concurrent calls (several steps of the same recipe save, awaited via
|
||||||
* in-flight training pass rather than each kicking off their own.
|
* the same event loop tick) all await the one in-flight training pass
|
||||||
|
* rather than each kicking off their own.
|
||||||
*/
|
*/
|
||||||
private async _ensureTrained(): Promise<void> {
|
private async _ensureTrained(): Promise<void> {
|
||||||
if (this._trained === undefined) {
|
if (this._trained === undefined) {
|
||||||
|
|
@ -543,21 +504,14 @@ export class TechStepClassifierService {
|
||||||
const techSteps = await prisma.techStep.findMany({ select: { id: true, key: true } });
|
const techSteps = await prisma.techStep.findMany({ select: { id: true, key: true } });
|
||||||
this._techStepIdByUid = new Map(techSteps.map((techStep) => [techStep.key, techStep.id]));
|
this._techStepIdByUid = new Map(techSteps.map((techStep) => [techStep.key, techStep.id]));
|
||||||
|
|
||||||
for (const entry of TECH_STEP_TRAINING_DATA) {
|
for (const locale of ["fr", "en"] as const) {
|
||||||
for (const [locale, data] of [
|
const entries = TECH_STEP_TRAINING_DATA.map((entry) => ({
|
||||||
["fr", entry.fr],
|
uid: entry.uid,
|
||||||
["en", entry.en],
|
synonyms: entry[locale].synonyms,
|
||||||
] as const) {
|
utterances: entry[locale].utterances,
|
||||||
if (data.synonyms.length > 0) {
|
}));
|
||||||
this._manager.addNamedEntityText(entry.uid, entry.uid, [locale], data.synonyms);
|
await intentServiceClient.train(locale, entries);
|
||||||
}
|
}
|
||||||
for (const utterance of data.utterances) {
|
|
||||||
this._manager.addDocument(locale, utterance, entry.uid);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
await this._manager.train();
|
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
throw err; // see matchTechStepSpans()'s catch comment above
|
throw err; // see matchTechStepSpans()'s catch comment above
|
||||||
}
|
}
|
||||||
|
|
|
||||||
101
apps/api/src/scripts/calibrate-tech-step-threshold.ts
Normal file
101
apps/api/src/scripts/calibrate-tech-step-threshold.ts
Normal file
|
|
@ -0,0 +1,101 @@
|
||||||
|
import { prisma } from "../db/prisma.js";
|
||||||
|
import { TECH_STEP_EVAL_DATASET } from "../lib/recipe-matching/tech-step-eval-dataset.js";
|
||||||
|
import {
|
||||||
|
computeTechStepMetrics,
|
||||||
|
type TechStepEvalOutcome,
|
||||||
|
} from "../lib/recipe-matching/tech-step-evaluator.js";
|
||||||
|
import { techStepClassifier } from "../lib/recipe-matching/tech-step-matcher.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Candidate thresholds to sweep, `0.05` to `0.95` in `0.05` steps — fine
|
||||||
|
* enough to find a good value without an unreasonable number of full
|
||||||
|
* `TECH_STEP_EVAL_DATASET` passes (each threshold only needs one
|
||||||
|
* {@link techStepClassifier.classifyClauses} call per eval case, not a
|
||||||
|
* retrain — see this file's own doc comment for why).
|
||||||
|
*/
|
||||||
|
const CANDIDATE_THRESHOLDS = Array.from({ length: 19 }, (_, i) => Math.round((i + 1) * 5) / 100);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One-off maintainer tool for recalibrating `CONFIDENCE_THRESHOLD`
|
||||||
|
* (`tech-step-matcher.ts`) after a change to the underlying intent
|
||||||
|
* classifier — most notably, the migration from `node-nlp` to
|
||||||
|
* `services/tech-step-intent-service` (spaCy): a different model produces a
|
||||||
|
* differently-shaped confidence score distribution, so a threshold tuned
|
||||||
|
* against the old classifier has no reason to still be the right cutoff for
|
||||||
|
* the new one.
|
||||||
|
*
|
||||||
|
* Reuses `techStepClassifier.classifyClauses` — already public, and
|
||||||
|
* deliberately *not* threshold-applied (see that method's own doc comment)
|
||||||
|
* — to get every eval case's raw `{anchorUid, intentUid, score}` per clause
|
||||||
|
* exactly once, then replays `_classifyClause`'s own decision rule
|
||||||
|
* (`intentUid` if confident enough, `anchorUid` otherwise) locally in this
|
||||||
|
* script for every candidate threshold. This is what makes a full sweep
|
||||||
|
* cheap: one classifier pass per eval case regardless of how many
|
||||||
|
* thresholds are being compared, rather than one full pass *per threshold*.
|
||||||
|
*
|
||||||
|
* Prints a threshold -> precision/recall/F1 table and the threshold that
|
||||||
|
* maximizes aggregate F1 — does **not** edit `tech-step-matcher.ts` itself.
|
||||||
|
* A maintainer reads the table, updates `CONFIDENCE_THRESHOLD` by hand (with
|
||||||
|
* an updated doc comment recording what run/F1 the new value was calibrated
|
||||||
|
* against, same as the existing comment's own format), then re-runs
|
||||||
|
* `retrain-tech-steps.ts` to confirm the change clears `MIN_OVERALL_F1`.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
*
|
||||||
|
* pnpm --filter api exec tsx src/scripts/calibrate-tech-step-threshold.ts
|
||||||
|
*/
|
||||||
|
async function calibrateTechStepThreshold(): Promise<void> {
|
||||||
|
console.info(`Classifying ${TECH_STEP_EVAL_DATASET.length} eval case(s)...`);
|
||||||
|
|
||||||
|
// One classifier pass per eval case, all clauses' raw verdicts kept
|
||||||
|
// alongside the case's own `expectedKeys` — reused for every candidate
|
||||||
|
// threshold in the loop below.
|
||||||
|
const casesWithClauses = await Promise.all(
|
||||||
|
TECH_STEP_EVAL_DATASET.map(async (evalCase) => ({
|
||||||
|
expectedKeys: evalCase.expectedKeys,
|
||||||
|
clauses: await techStepClassifier.classifyClauses(evalCase.description, evalCase.locale),
|
||||||
|
})),
|
||||||
|
);
|
||||||
|
|
||||||
|
console.info("\nthreshold precision recall f1");
|
||||||
|
let bestThreshold = CANDIDATE_THRESHOLDS[0] ?? 0;
|
||||||
|
let bestF1 = -1;
|
||||||
|
|
||||||
|
for (const threshold of CANDIDATE_THRESHOLDS) {
|
||||||
|
const outcomes: TechStepEvalOutcome[] = casesWithClauses.map(({ expectedKeys, clauses }) => {
|
||||||
|
const actualKeys = clauses
|
||||||
|
// Mirrors `_classifyClause`'s own decision rule exactly (see that
|
||||||
|
// method, `tech-step-matcher.ts`) — the classifier's own verdict
|
||||||
|
// when confident enough, otherwise its clause's NER anchor, `null`
|
||||||
|
// when neither applies (no keyword, no confident classification).
|
||||||
|
.map((clause) =>
|
||||||
|
clause.intentUid !== null && clause.score >= threshold
|
||||||
|
? clause.intentUid
|
||||||
|
: clause.anchorUid,
|
||||||
|
)
|
||||||
|
.filter((key): key is string => key !== null);
|
||||||
|
return { expectedKeys, actualKeys };
|
||||||
|
});
|
||||||
|
|
||||||
|
const { overall } = computeTechStepMetrics(outcomes);
|
||||||
|
console.info(
|
||||||
|
`${threshold.toFixed(2)} ${overall.precision.toFixed(3)} ${overall.recall.toFixed(3)} ${overall.f1.toFixed(3)}`,
|
||||||
|
);
|
||||||
|
if (overall.f1 > bestF1) {
|
||||||
|
bestF1 = overall.f1;
|
||||||
|
bestThreshold = threshold;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.info(
|
||||||
|
`\nBest aggregate F1 ${bestF1.toFixed(3)} at threshold ${bestThreshold.toFixed(2)} — update CONFIDENCE_THRESHOLD in tech-step-matcher.ts by hand if this differs from the current value.`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
calibrateTechStepThreshold()
|
||||||
|
.then(() => prisma.$disconnect())
|
||||||
|
.catch(async (err) => {
|
||||||
|
console.error(err);
|
||||||
|
await prisma.$disconnect();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
|
@ -9,22 +9,45 @@ import { registerAllRecipeSources } from "./sources/index.js";
|
||||||
// doesn't happen inside app.ts/createServer() itself.
|
// doesn't happen inside app.ts/createServer() itself.
|
||||||
registerAllRecipeSources();
|
registerAllRecipeSources();
|
||||||
|
|
||||||
// Trains the tech-step classifier (and pays node-nlp's own one-time lazy
|
/**
|
||||||
// setup cost — see `TechStepClassifierService.warmUp`) before accepting
|
* Trains the tech-step classifier (a `POST /v1/train` round-trip per locale
|
||||||
// any traffic, so the first real recipe save/preview isn't the one stuck
|
* to `services/tech-step-intent-service` — see
|
||||||
// waiting several seconds for it.
|
* `TechStepClassifierService.warmUp`) before accepting any traffic, so the
|
||||||
try {
|
* first real recipe save/preview isn't the one stuck waiting for it.
|
||||||
|
*
|
||||||
|
* Retried with exponential backoff: in Docker Compose, `app`'s own
|
||||||
|
* `depends_on: tech-step-intent-service: condition: service_healthy`
|
||||||
|
* (`docker-compose.yml`) already means that service is up by the time this
|
||||||
|
* runs, but native dev (`pnpm dev:api`, no Compose ordering at all) can
|
||||||
|
* easily start this before the intent service has finished loading its
|
||||||
|
* spaCy models — a transient connection failure here shouldn't need a
|
||||||
|
* manual restart. Still non-fatal after every attempt is exhausted: the
|
||||||
|
* *next* real call retries training itself (see `_ensureTrained`'s own
|
||||||
|
* retry-on-failure comment), same graceful-degrade posture as before this
|
||||||
|
* retry loop existed.
|
||||||
|
*/
|
||||||
|
async function warmUpTechStepClassifier(): Promise<void> {
|
||||||
|
const maxAttempts = 5;
|
||||||
|
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||||
|
try {
|
||||||
await techStepClassifier.warmUp();
|
await techStepClassifier.warmUp();
|
||||||
} catch (err) {
|
return;
|
||||||
// Not fatal to startup — a failed warm-up just means the *next* call
|
} catch (err) {
|
||||||
// retries training itself (see `_ensureTrained`'s own retry-on-failure
|
if (attempt === maxAttempts) {
|
||||||
// comment), same graceful-degrade posture as everywhere else training
|
logger.error("Tech-step classifier warm-up failed after retries", {
|
||||||
// failures surface. Still worth a loud log: this shouldn't normally fail.
|
|
||||||
logger.error("Tech-step classifier warm-up failed", {
|
|
||||||
error: err instanceof Error ? err.message : String(err),
|
error: err instanceof Error ? err.message : String(err),
|
||||||
|
attempts: attempt,
|
||||||
});
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const delayMs = 1000 * 2 ** (attempt - 1);
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, delayMs));
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
await warmUpTechStepClassifier();
|
||||||
|
|
||||||
const server = createServer();
|
const server = createServer();
|
||||||
|
|
||||||
server.listen(env.PORT, () => {
|
server.listen(env.PORT, () => {
|
||||||
|
|
|
||||||
50
apps/api/src/types/node-nlp.d.ts
vendored
50
apps/api/src/types/node-nlp.d.ts
vendored
|
|
@ -1,50 +0,0 @@
|
||||||
/**
|
|
||||||
* Minimal ambient typing for `node-nlp` (no official/DefinitelyTyped types
|
|
||||||
* exist for it) — declares only the `NlpManager` surface
|
|
||||||
* `tech-step-matcher.ts` actually calls, verified against the real
|
|
||||||
* package (v4.27.0) rather than the library's full documented API, which
|
|
||||||
* this repo doesn't use the rest of.
|
|
||||||
*/
|
|
||||||
declare module "node-nlp" {
|
|
||||||
/** Constructor options this repo passes — `NlpManager` accepts more, only what's used here is typed. */
|
|
||||||
export interface NlpManagerOptions {
|
|
||||||
languages?: string[];
|
|
||||||
forceNER?: boolean;
|
|
||||||
nlu?: { log?: boolean };
|
|
||||||
ner?: { threshold?: number };
|
|
||||||
/** Defaults to `true` — persists the trained model to `modelFileName` (default `model.nlp`, in `process.cwd()`). See `tech-step-matcher.ts`'s own constructor comment for why this repo always sets it `false`. */
|
|
||||||
autoSave?: boolean;
|
|
||||||
/** Defaults to `true` — loads from `modelFileName` instead of training fresh if that file already exists. Always `false` here, same reasoning as `autoSave`. */
|
|
||||||
autoLoad?: boolean;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** One entity `NlpManager.process`'s result reports — see `tech-step-matcher.ts`'s own `NerEntity` for the subset this repo reads. */
|
|
||||||
export interface NlpEntity {
|
|
||||||
entity: string;
|
|
||||||
start: number;
|
|
||||||
end: number;
|
|
||||||
type: string;
|
|
||||||
accuracy?: number;
|
|
||||||
sourceText?: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** `NlpManager.process`'s result — trimmed to the fields this repo reads (the real object carries many more). */
|
|
||||||
export interface NlpProcessResult {
|
|
||||||
intent: string;
|
|
||||||
score: number;
|
|
||||||
entities: NlpEntity[];
|
|
||||||
}
|
|
||||||
|
|
||||||
export class NlpManager {
|
|
||||||
public constructor(options?: NlpManagerOptions);
|
|
||||||
public addNamedEntityText(
|
|
||||||
entityName: string,
|
|
||||||
optionName: string,
|
|
||||||
languages: string[],
|
|
||||||
texts: string[],
|
|
||||||
): void;
|
|
||||||
public addDocument(locale: string, utterance: string, intent: string): void;
|
|
||||||
public train(): Promise<void>;
|
|
||||||
public process(locale: string, text: string): Promise<NlpProcessResult>;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -121,9 +121,11 @@ describe("tech-step-matcher", () => {
|
||||||
// (`tech-step-training-data.ts`) and the real seeded `TechStep`
|
// (`tech-step-training-data.ts`) and the real seeded `TechStep`
|
||||||
// catalog, rather than synthetic injectable fixtures the old
|
// catalog, rather than synthetic injectable fixtures the old
|
||||||
// regex-based `matchTechStepSpans(description, mappings)` allowed.
|
// regex-based `matchTechStepSpans(description, mappings)` allowed.
|
||||||
// Training + node-nlp's own one-time per-language setup can take a
|
// Training now round-trips over HTTP to a real, locally running
|
||||||
// few seconds on the very first call in the whole suite (subsequent
|
// `services/tech-step-intent-service` (see that service's own README
|
||||||
// calls reuse the same trained model and are fast) — comfortably
|
// and `apps/api/.env.test`) — the very first call in the whole suite
|
||||||
|
// pays for that plus the service's own spaCy pipeline setup (subsequent
|
||||||
|
// calls reuse the already-trained pipeline and are fast) — comfortably
|
||||||
// inside this suite's default 10s timeout (.mocharc.json).
|
// inside this suite's default 10s timeout (.mocharc.json).
|
||||||
let simmerId: number;
|
let simmerId: number;
|
||||||
let cookId: number;
|
let cookId: number;
|
||||||
|
|
|
||||||
|
|
@ -27,9 +27,6 @@ services:
|
||||||
context: .
|
context: .
|
||||||
dockerfile: apps/api/Dockerfile
|
dockerfile: apps/api/Dockerfile
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
depends_on:
|
|
||||||
postgres:
|
|
||||||
condition: service_healthy
|
|
||||||
environment:
|
environment:
|
||||||
NODE_ENV: production
|
NODE_ENV: production
|
||||||
PORT: 3000
|
PORT: 3000
|
||||||
|
|
@ -51,8 +48,50 @@ services:
|
||||||
# default: `/internal/tech-steps/*` fails closed rather than open
|
# default: `/internal/tech-steps/*` fails closed rather than open
|
||||||
# for a deployment that doesn't run the worker at all.
|
# for a deployment that doesn't run the worker at all.
|
||||||
INTERNAL_WORKER_SECRET: ${INTERNAL_WORKER_SECRET:-}
|
INTERNAL_WORKER_SECRET: ${INTERNAL_WORKER_SECRET:-}
|
||||||
|
# Compose network service name, not localhost — same reasoning as
|
||||||
|
# DATABASE_URL above. Unlike INTERNAL_WORKER_SECRET, no `:-` fallback:
|
||||||
|
# tech-step-intent-service is a core dependency (see its own entry
|
||||||
|
# below), not an optional background job.
|
||||||
|
INTENT_SERVICE_BASE_URL: "http://tech-step-intent-service:8000"
|
||||||
|
INTENT_SERVICE_SECRET: ${INTENT_SERVICE_SECRET:?set INTENT_SERVICE_SECRET in .env}
|
||||||
ports:
|
ports:
|
||||||
- "${APP_PORT:-3000}:3000"
|
- "${APP_PORT:-3000}:3000"
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
tech-step-intent-service:
|
||||||
|
condition: service_healthy
|
||||||
|
|
||||||
|
# spaCy-based NER + intent classification microservice
|
||||||
|
# (services/tech-step-intent-service) — `app` delegates all tech-step
|
||||||
|
# detection to it over HTTP (see `IntentServiceClient`,
|
||||||
|
# apps/api/src/lib/recipe-matching/intent-service-client.ts). Unlike
|
||||||
|
# `tech-step-llm-worker` below, **not optional**: without it, `app` can no
|
||||||
|
# longer detect any cooking technique in a recipe step at all. No exposed
|
||||||
|
# port — reachable only from `app` on the compose network, nothing ever
|
||||||
|
# calls into it from outside.
|
||||||
|
tech-step-intent-service:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: services/tech-step-intent-service/Dockerfile
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
INTENT_SERVICE_SECRET: ${INTENT_SERVICE_SECRET:?set INTENT_SERVICE_SECRET in .env}
|
||||||
|
healthcheck:
|
||||||
|
# No curl/wget in the python:3.12-slim base image — a one-line Python
|
||||||
|
# request is the healthcheck for a service that's already guaranteed
|
||||||
|
# to have Python (see this service's Dockerfile).
|
||||||
|
test:
|
||||||
|
[
|
||||||
|
"CMD",
|
||||||
|
"python",
|
||||||
|
"-c",
|
||||||
|
"import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=2)",
|
||||||
|
]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 10
|
||||||
|
start_period: 15s
|
||||||
|
|
||||||
# Deliberately its own image, not built into `app`'s (see
|
# Deliberately its own image, not built into `app`'s (see
|
||||||
# services/tech-step-llm-worker/Dockerfile's own doc comment) — a
|
# services/tech-step-llm-worker/Dockerfile's own doc comment) — a
|
||||||
|
|
|
||||||
696
pnpm-lock.yaml
696
pnpm-lock.yaml
|
|
@ -44,9 +44,6 @@ importers:
|
||||||
jsonwebtoken:
|
jsonwebtoken:
|
||||||
specifier: ^9.0.3
|
specifier: ^9.0.3
|
||||||
version: 9.0.3
|
version: 9.0.3
|
||||||
node-nlp:
|
|
||||||
specifier: 4.27.0
|
|
||||||
version: 4.27.0
|
|
||||||
prisma:
|
prisma:
|
||||||
specifier: ^5.22.0
|
specifier: ^5.22.0
|
||||||
version: 5.22.0
|
version: 5.22.0
|
||||||
|
|
@ -853,224 +850,12 @@ packages:
|
||||||
resolution: {integrity: sha512-Yhlar6v9WQgUp/He7BdgzOz8lqMQ8sU+jkCq7Wx8Myc5YFJLbEe7lgui/V7G1qB1DJykHSGwreceSaD60Y0PUQ==, tarball: https://registry.npmjs.org/@mapbox/node-pre-gyp/-/node-pre-gyp-1.0.11.tgz}
|
resolution: {integrity: sha512-Yhlar6v9WQgUp/He7BdgzOz8lqMQ8sU+jkCq7Wx8Myc5YFJLbEe7lgui/V7G1qB1DJykHSGwreceSaD60Y0PUQ==, tarball: https://registry.npmjs.org/@mapbox/node-pre-gyp/-/node-pre-gyp-1.0.11.tgz}
|
||||||
hasBin: true
|
hasBin: true
|
||||||
|
|
||||||
'@microsoft/recognizers-text-choice@1.3.1':
|
|
||||||
resolution: {integrity: sha512-HubunMJVq/OetmdvcAmBh5skMlg+yiScm3V2wNyNZIVvLgli4+8nzbg/W/fI9dpaf6wv9ZQ7d2IYvn8swJBo3A==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text-choice/-/recognizers-text-choice-1.3.1.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-data-types-timex-expression@1.3.1':
|
|
||||||
resolution: {integrity: sha512-jarJIFIJZBqeofy3hh0vdQo1yOmTM+jCjj6/zmo9JunsQ6LO750eZHCg9eLptQhsvq321XCt5xdRNLCwU8YeNA==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text-data-types-timex-expression/-/recognizers-text-data-types-timex-expression-1.3.1.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-date-time@1.3.2':
|
|
||||||
resolution: {integrity: sha512-fUEGOTccS55ZY0erzjS1bunJYA9lGXjcZoru5oPOlnxbJS4Lk0ylgdH2Ub2EjAyqr8DIJhdLNOEesCdAXMvlNg==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text-date-time/-/recognizers-text-date-time-1.3.2.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-number-with-unit@1.3.1':
|
|
||||||
resolution: {integrity: sha512-gzCpPP4zQ5Vb+RHaWjzP2t1c+mj6GYOsFoI2NyJkm8OZ52XI+x9SJCgrrD2ujzjOd5/CQVC46rE22rfGwXLDkA==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text-number-with-unit/-/recognizers-text-number-with-unit-1.3.1.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-number@1.3.1':
|
|
||||||
resolution: {integrity: sha512-JBxhSdihdQLQilCtqISEBw5kM+CNGTXzy5j5hNoZECNUEvBUPkAGNEJAeQPMP5abrYks29aSklnSvSyLObXaNQ==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text-number/-/recognizers-text-number-1.3.1.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-sequence@1.3.1':
|
|
||||||
resolution: {integrity: sha512-J7Kg35hpm0NcFHmu69Bb4q7DPDiSpCd8ApUZqNm59itIjrQJHpSdl9HF6JxuQQz0Ftc/li5ZLqSuupJAmA/sgg==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text-sequence/-/recognizers-text-sequence-1.3.1.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-suite@1.3.0':
|
|
||||||
resolution: {integrity: sha512-uqG4vzy5N2CmBaeINny0bLdnGp0jDbT1moNoLC+Yim3G8kHOU9lpDfwA6VN6HTYaDM5854SNMEzLjJdS1TPFTw==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text-suite/-/recognizers-text-suite-1.3.0.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text@1.3.1':
|
|
||||||
resolution: {integrity: sha512-HikLoRUgSzM4OKP3JVBzUUp3Q7L4wgI17p/3rERF01HVmopcujY3i6wgx8PenCwbenyTNxjr1AwSDSVuFlYedQ==, tarball: https://registry.npmjs.org/@microsoft/recognizers-text/-/recognizers-text-1.3.1.tgz}
|
|
||||||
engines: {node: '>=10.3.0'}
|
|
||||||
|
|
||||||
'@napi-rs/lzma-linux-x64-gnu@1.5.1':
|
'@napi-rs/lzma-linux-x64-gnu@1.5.1':
|
||||||
resolution: {integrity: sha512-oTXEIha4SsuXdTA4Iyskj0kpdx2yVXdhd75c2v3xGrHFfVMsbhTPZU/nMPL4sWKo4pBHm3aucLaqGlF696dTyQ==, tarball: https://registry.npmjs.org/@napi-rs/lzma-linux-x64-gnu/-/lzma-linux-x64-gnu-1.5.1.tgz}
|
resolution: {integrity: sha512-oTXEIha4SsuXdTA4Iyskj0kpdx2yVXdhd75c2v3xGrHFfVMsbhTPZU/nMPL4sWKo4pBHm3aucLaqGlF696dTyQ==, tarball: https://registry.npmjs.org/@napi-rs/lzma-linux-x64-gnu/-/lzma-linux-x64-gnu-1.5.1.tgz}
|
||||||
engines: {node: ^22.20 || ^24.12 || >=25}
|
engines: {node: ^22.20 || ^24.12 || >=25}
|
||||||
cpu: [x64]
|
cpu: [x64]
|
||||||
os: [linux]
|
os: [linux]
|
||||||
|
|
||||||
'@nlpjs/builtin-duckling@4.26.1':
|
|
||||||
resolution: {integrity: sha512-3qkH955X2g5MXV1EqT3fTAT/lLEdiqqe5IgBDyr+MQB7FOV9R3YhqGIn3DFOl+TSm/tP5n/BAEptkTNn/TOpmQ==, tarball: https://registry.npmjs.org/@nlpjs/builtin-duckling/-/builtin-duckling-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/builtin-microsoft@4.26.1':
|
|
||||||
resolution: {integrity: sha512-AODgzTcfYUf5Ozm00aQnHImDum7Idtl0F9dSPoaXpfj7rZqP8hPZ7iWwdGTAvISH/da2YhjPOU65QSYk2YpjFA==, tarball: https://registry.npmjs.org/@nlpjs/builtin-microsoft/-/builtin-microsoft-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/core-loader@4.26.1':
|
|
||||||
resolution: {integrity: sha512-IiRtn65bdiUSQHy2kusco2fmhk39u2Mc2c5Fsm9+9EVG6BtJCmVEFU/btAzGDAmxEA/E4qKecaAT4LvcW6TPbA==, tarball: https://registry.npmjs.org/@nlpjs/core-loader/-/core-loader-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/core@4.26.1':
|
|
||||||
resolution: {integrity: sha512-M/PeFddsi3y7Z1piFJxsLGm5/xdMhcrpOsml7s6CTEgYo8iduaT30HDd61tZxDyvvJseU6uFqlXSn7XKkAcC1g==, tarball: https://registry.npmjs.org/@nlpjs/core/-/core-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/emoji@4.26.1':
|
|
||||||
resolution: {integrity: sha512-Q0PoXwIvaB1bnRXK4U/YD7mrqaz29Yfed3s2au0iXl1bffUgoG+hs4GORCvyy7DFCCLlc9d5yDM3oLIX/ggZ+Q==, tarball: https://registry.npmjs.org/@nlpjs/emoji/-/emoji-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/evaluator@4.26.1':
|
|
||||||
resolution: {integrity: sha512-WeUrC8qq7+V8Jhkkjc2yiXdzy9V0wbETv8/qasQmL0QmEuwBDJF+fvfl4z2vWpBb0vW07A8aNrFElKELzbpkdg==, tarball: https://registry.npmjs.org/@nlpjs/evaluator/-/evaluator-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-all@4.26.1':
|
|
||||||
resolution: {integrity: sha512-UzRm1JRRAyQqilEOxQ2ySMOitKbhPk5iKYbjD8FREDcPjreUvDxVuQsYUOvYucmEyFcZU2U/TdJx+fX9/bcaKQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-all/-/lang-all-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ar@4.26.1':
|
|
||||||
resolution: {integrity: sha512-MUlVtabt9ltG7WyzCQpFJymLJlnEqp3mxhgN9JHyFH7oZMK3REvMovFfvEUAbfiYrJEv/BN5KKLL7yrvUeaHtg==, tarball: https://registry.npmjs.org/@nlpjs/lang-ar/-/lang-ar-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-bn@4.26.1':
|
|
||||||
resolution: {integrity: sha512-sim1iZKBDdehi/yBUKrLW51QvS9uB+sXW7lj+THVqBy5UsnEQvt4gzE0NsC873uJMh66vt2AlHkhzgPH0qH/nQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-bn/-/lang-bn-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ca@4.26.1':
|
|
||||||
resolution: {integrity: sha512-fD4R5tcAB0uYtNxSEF20b1KmF6nUQSbiJqrIUJI5yis4ObjCYRQnSh4bjVDKUKxyONjbD6L8EaK5GrY1/jkwFQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-ca/-/lang-ca-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-cs@4.26.1':
|
|
||||||
resolution: {integrity: sha512-CqI6VB8toaJ/MlP1D4K9BctA6GpZJhMKyEy+OX9xavDe4r4ao/SxlSaIYK3izK0k+J38lJWC5lXYGazfCdTGjA==, tarball: https://registry.npmjs.org/@nlpjs/lang-cs/-/lang-cs-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-da@4.26.1':
|
|
||||||
resolution: {integrity: sha512-krI/ojeDSi329ENM/hLIsbUh1x4XRTKAbtPcbFxAY6XVhcSVoWPO7L77jFTL1NQeE1oGRFzGHaeC9hZJ8phVbA==, tarball: https://registry.npmjs.org/@nlpjs/lang-da/-/lang-da-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-de@4.26.1':
|
|
||||||
resolution: {integrity: sha512-HfZQwsE5FICq9taVZDiyktmdAePVF5948NM80et0d9mx43RWDFhHKQYgtJPwfQXtdCoQtOM5TOJ2FanGwzPeaA==, tarball: https://registry.npmjs.org/@nlpjs/lang-de/-/lang-de-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-el@4.26.1':
|
|
||||||
resolution: {integrity: sha512-pcOvuSwPCXxI+2xNZZzM4V5pTRDntYoJi0SP/ic2nV4IPQ0nU2j16dYfg1HlvET/E6iN1VTqghrCaf10SMkDGA==, tarball: https://registry.npmjs.org/@nlpjs/lang-el/-/lang-el-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-en-min@4.26.1':
|
|
||||||
resolution: {integrity: sha512-1sJZ7dy7ysqzbsB8IklguvB88J8EPIv4XGVkZCcwecKtOw+fp5LAsZ3TJVmEf18iK1gD4cEGr7qZg5fpPxTpWQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-en-min/-/lang-en-min-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-en@4.26.1':
|
|
||||||
resolution: {integrity: sha512-GVoJpOjyk5TtBAqo/fxsiuuH7jXycyakGT0gw5f01u9lOmUnpJegvXyGff/Nb0j14pXcGHXOhmpWrcTrG2B0LQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-en/-/lang-en-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-es@4.26.1':
|
|
||||||
resolution: {integrity: sha512-fIPQt+WPcNdyxZOCMkOPlMb4Y1iE585QxjB9IAdFz8ZtVg7mc4dlv5f46ud7ppdMh84iLOuOdo6pzu2Cqm14lw==, tarball: https://registry.npmjs.org/@nlpjs/lang-es/-/lang-es-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-eu@4.26.1':
|
|
||||||
resolution: {integrity: sha512-Ha8GHTbgQYd7dwHM8aWHDyxmbUNUcyu/5xlBKqqBOPxysDyZ6Ad0tvj0FmJBy6mYhqmFTPBnEAo69cfuFSqWIQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-eu/-/lang-eu-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-fa@4.26.1':
|
|
||||||
resolution: {integrity: sha512-qJCmNXgJZnfNXUnKnxvEGEzSFBdQT4XU7/rMxuFmSJqmQY7fH/Vsmi5CKF94VRBPOIV4ULlEJuLpUWHXRmOnVQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-fa/-/lang-fa-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-fi@4.26.1':
|
|
||||||
resolution: {integrity: sha512-W/rUcrzSh3KE07q2vOsssTpU1sbX32gbBzKPZfRJ2ZUF4afO+eHxmAywikXubP4kiU3JxVNLvXXEjuGD3SBUbA==, tarball: https://registry.npmjs.org/@nlpjs/lang-fi/-/lang-fi-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-fr@4.26.1':
|
|
||||||
resolution: {integrity: sha512-LTA852atCJnHtKDmtjx/ui5AnvEIkrPx+MJQ2mB3gn8ko6i2UITnJgPmJE9Kej5bLasVZOAJvU/SrfXEmnPGOw==, tarball: https://registry.npmjs.org/@nlpjs/lang-fr/-/lang-fr-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ga@4.26.1':
|
|
||||||
resolution: {integrity: sha512-JsP1CZ8r3Jd6o/Az7cN3exz0HDP3FNYLzh4Vi6ksEkdKF0yCjJ9G5dXZYqS9qFIN5ffemWn29G4WRELY6QH/cQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-ga/-/lang-ga-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-gl@4.26.1':
|
|
||||||
resolution: {integrity: sha512-y1NNu6NVy/6o5UNfihgg0WkSlVr4IvKA5W193CpRLZWS4FccQDmnFFhyYWRkshyDbgEsfsZ0Rs3BoE82+T2Ubg==, tarball: https://registry.npmjs.org/@nlpjs/lang-gl/-/lang-gl-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-hi@4.26.1':
|
|
||||||
resolution: {integrity: sha512-Fw9rXqF5l8q9etJG5uOlEFpnMVjQEWMaCIgQfEcA1yTvieSV8mpoSvQkEZl+DFhww+azareoJ7ZCkx0gJ9UDuQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-hi/-/lang-hi-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-hu@4.26.1':
|
|
||||||
resolution: {integrity: sha512-7dPUn5/ZpLZmsdRwO+dtORuMIiIpnsWbgSLIKdOLh8irhgUR+M2bYTfkdnKcrEcHzHPP8Svn7pU0xk7OKSUA1w==, tarball: https://registry.npmjs.org/@nlpjs/lang-hu/-/lang-hu-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-hy@4.26.1':
|
|
||||||
resolution: {integrity: sha512-T2brpLGDJryAwWmjtnmY8Ot6ZUkCz+/nRR9/QM1PybvZIqOVLjJqA49bqjJfT5DMN89HbwC7I/15NTT0y09i1Q==, tarball: https://registry.npmjs.org/@nlpjs/lang-hy/-/lang-hy-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-id@4.26.1':
|
|
||||||
resolution: {integrity: sha512-rVuIkYFKdltFhMT/a2ZxD9ovoZSVZF7OPuqYjTXW9xKd3Ff32yUrzcf/pHXlqmZOSltqOH3E5jZRRDkHvgUOjQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-id/-/lang-id-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-it@4.26.1':
|
|
||||||
resolution: {integrity: sha512-BZA3QnfQGW91gYaybRmHnCAPBvQggtmHZJrAmuBZUKUS12HoQm8uybjw2fZO+vahEeUQceKNDISRcT1eLLijog==, tarball: https://registry.npmjs.org/@nlpjs/lang-it/-/lang-it-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ja@4.26.1':
|
|
||||||
resolution: {integrity: sha512-QgkuJOkHguRFyfnckH2It5/Kg8zecnOMJsHxYeuDC4tBF7jL/5xqWis+679lYLsXtAkrG8+fjVcBbjyopP0KHg==, tarball: https://registry.npmjs.org/@nlpjs/lang-ja/-/lang-ja-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ko@4.26.1':
|
|
||||||
resolution: {integrity: sha512-Q0N8bLJJ829ILWCKH1UQWPSNyuLaEURAXCawkDju4pt33DBLcpqz9IzO9dnqiFc+fjSgVzZ7WMaLT18hXZQ9vg==, tarball: https://registry.npmjs.org/@nlpjs/lang-ko/-/lang-ko-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-lt@4.26.1':
|
|
||||||
resolution: {integrity: sha512-SeYZxRhdCy+ClQNnF/u0MAtcDui/ocdk4NtgNOCuwNTNuzhN3t3rfGeArfBGmZeg1SIeBLUDE9dsTxYCv5AOEg==, tarball: https://registry.npmjs.org/@nlpjs/lang-lt/-/lang-lt-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ms@4.26.1':
|
|
||||||
resolution: {integrity: sha512-KxWBS+tFY2U8z9UrjQIqMM40npGDOskP5DcWhaEE3zuhzf3RTDYjy8sdz34jVd0fBdbPihX133h3bFibg2Cm7w==, tarball: https://registry.npmjs.org/@nlpjs/lang-ms/-/lang-ms-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ne@4.26.1':
|
|
||||||
resolution: {integrity: sha512-K3E2l+0LTESv+dO+ZTIdvNa+zwMJvvnMiFYYkKvJst6lhc8JgvGOsPxGsjJn6PDhI3wyfQu+dg3b+bnVPu4FDA==, tarball: https://registry.npmjs.org/@nlpjs/lang-ne/-/lang-ne-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-nl@4.26.1':
|
|
||||||
resolution: {integrity: sha512-I/mP1RRbUN4BQ+8NXAl2FKaLHbb7f6S8JVjxHQ0sKHT4BgQ3+r0yO+DVcEsHg+vWRiY1Fyzh0gq0PhLVnF6HnA==, tarball: https://registry.npmjs.org/@nlpjs/lang-nl/-/lang-nl-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-no@4.26.1':
|
|
||||||
resolution: {integrity: sha512-a0CLL2c/OCzbg7J7ugyrsAksI96XhkQ3IeBbbx60o5o/9wsFNik6cPWrkpoE5xNtw7gLlAJWabwDiZXkl8Zrcw==, tarball: https://registry.npmjs.org/@nlpjs/lang-no/-/lang-no-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-pl@4.26.1':
|
|
||||||
resolution: {integrity: sha512-nrDXlq+TzQLE5IpXPIlFMzd8OpquvApWsouh6fmLsD9HZLZI4O3w1M4sXXLzE+9Ggu9Cy1m1QJ0/i7XCcv115g==, tarball: https://registry.npmjs.org/@nlpjs/lang-pl/-/lang-pl-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-pt@4.26.1':
|
|
||||||
resolution: {integrity: sha512-p6yZHaJ0e+n0avMHpdDw5PMk4HkKXjPbOMbrlg0dF+VRqChjxfH478Q423rDyzu/4MzDsIYB+p6KzL9AARKXpg==, tarball: https://registry.npmjs.org/@nlpjs/lang-pt/-/lang-pt-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ro@4.26.1':
|
|
||||||
resolution: {integrity: sha512-baUdTA0DWpDR0Tn6fxo+RDN/6gbuINLCARtHwap2UR/HKQWP2XoH/DIvcjZpwUTalr5MQjso31epcdeRRapczA==, tarball: https://registry.npmjs.org/@nlpjs/lang-ro/-/lang-ro-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ru@4.26.1':
|
|
||||||
resolution: {integrity: sha512-NaZ2DAOGxWG2Us9IyIDs3m6vhGpUaUJRVgzzHHyX3LO3xEYjZmtnA0jEpBaTOe2PuNHThv0WCZUNn9BSurV3PA==, tarball: https://registry.npmjs.org/@nlpjs/lang-ru/-/lang-ru-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-sl@4.26.1':
|
|
||||||
resolution: {integrity: sha512-QBJwcJt+oKUpAnHKNJkLkx9Xm1n4dUPC5GPYfAXTnJZf0hNWJSY21GicdWi7Vu/qFJ3ghIqtSP8D7KIPLnibNw==, tarball: https://registry.npmjs.org/@nlpjs/lang-sl/-/lang-sl-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-sr@4.26.1':
|
|
||||||
resolution: {integrity: sha512-drH3+UqTW637uLWsnLrcp8jEKUGxV61ZgCBjNkVQNEv1/jbpSg6IqgynSY2JyhtnlV0f870KS0HvSbyo5AD4Ng==, tarball: https://registry.npmjs.org/@nlpjs/lang-sr/-/lang-sr-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-sv@4.26.1':
|
|
||||||
resolution: {integrity: sha512-2axkrYFC02tAlxCWeiEKISbe4dSteciP1CIggO/dZglnnLWgdF+g7kOeYMn7abCfFVSnh5vLqfDkrwnyIqt7Ag==, tarball: https://registry.npmjs.org/@nlpjs/lang-sv/-/lang-sv-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-ta@4.26.1':
|
|
||||||
resolution: {integrity: sha512-keeh+croa1TAirV9Fd3OQMo5IkAlTGNWTNweHbi/htYMX0MKOPYxyqg+VH2bml+57VY2aUj/WYgV/p3ATx9EfQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-ta/-/lang-ta-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-th@4.26.1':
|
|
||||||
resolution: {integrity: sha512-2SWZhrln3rMw8/DsRc9yS5bi3qEdGfw2pq9Uejx/UYED5zvvL6kh9AiCJZT4k0wMBGEwWUV6HxJ0Pq/jOTHogg==, tarball: https://registry.npmjs.org/@nlpjs/lang-th/-/lang-th-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-tl@4.26.1':
|
|
||||||
resolution: {integrity: sha512-AzmLtg28tm0VXCm0Q0EY3OtA3m4oYxaqh4VX6uhB4J+PoEsIkm0py12SJxMNIsh/r98pobCumH8KH9bvHQoCAg==, tarball: https://registry.npmjs.org/@nlpjs/lang-tl/-/lang-tl-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-tr@4.26.1':
|
|
||||||
resolution: {integrity: sha512-p30uuXvE9pZeU/5XkrQfvxRgiAOBmP3EyBFGV/+P05PEogaqbsmmtVCgCnR63yeRvVnGbToPBPjRK3OO1y4AEQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-tr/-/lang-tr-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-uk@4.26.1':
|
|
||||||
resolution: {integrity: sha512-PVEvmlhvl6BL3e/Q4qjMPsnwON3cWEYvDh9dg+Si+sjD2Edu9tajolJKcQ6ZA4I8dXrld5xuXx+DEBH/uB4uWQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-uk/-/lang-uk-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/lang-zh@4.26.1':
|
|
||||||
resolution: {integrity: sha512-kwqeqeEgMAMvucVX9HNE1p6s/2APP23ZsS8Um/lNvtswb4gL5jjYF9kyCvRfqlPBQSWWdRv7wwcnNXOvXYkxcQ==, tarball: https://registry.npmjs.org/@nlpjs/lang-zh/-/lang-zh-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/language-min@4.25.0':
|
|
||||||
resolution: {integrity: sha512-g8jtbDbqtRm+dlD/1Vnb4VWfKbKteApEGVTqIMxYkk6N/HMhvLZ5J2svrxzrB98a/HZ0fb//YBfFgymnz9Oukg==, tarball: https://registry.npmjs.org/@nlpjs/language-min/-/language-min-4.25.0.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/language@4.25.0':
|
|
||||||
resolution: {integrity: sha512-tUF6QENoUQ/E26RYc32IgsttStSF9cNO4ySN+BQECn8VpjukWdwbMw073MlOLXzjfeobxa+3hCVrmPPcW+V3UA==, tarball: https://registry.npmjs.org/@nlpjs/language/-/language-4.25.0.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/ner@4.27.0':
|
|
||||||
resolution: {integrity: sha512-ptwkxriJdmgHSH9TfP10JQ1jviaSl2SupSFGUvTuWkuJhobQd3hbnlSq40V6XYvJNmqh9M9zEab/AKeghxYOTA==, tarball: https://registry.npmjs.org/@nlpjs/ner/-/ner-4.27.0.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/neural@4.25.0':
|
|
||||||
resolution: {integrity: sha512-Oz20denGiBe0DlQsS7lN4TNrATN1nXlHKc/HB6jJPegjVmgJVCugDaHwIGoV7qOWyA6F2fRRwOgD+quNT2gVpg==, tarball: https://registry.npmjs.org/@nlpjs/neural/-/neural-4.25.0.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/nlg@4.26.1':
|
|
||||||
resolution: {integrity: sha512-PCJWiZ7464ChXXUGvjBZIFtoqkC24Oy6X63HgQrSv+63svz22Y5Cmu1MYLk77Nb+4keWv+hKhFJKDkvJoOpBVg==, tarball: https://registry.npmjs.org/@nlpjs/nlg/-/nlg-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/nlp@4.27.0':
|
|
||||||
resolution: {integrity: sha512-q6X7sY6TYVnQRZJKF/6mfLFlNA5oRYLhgQ5k3i1IBqH9lbWTAZJr31w/dCf97HXaYaj+vJp3h0ucfNumme9EIw==, tarball: https://registry.npmjs.org/@nlpjs/nlp/-/nlp-4.27.0.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/nlu@4.27.0':
|
|
||||||
resolution: {integrity: sha512-j4DUdoXS/y/Xag6ysYXx7Ve8NBmUVViUSCJhj3r49+zGyYtyVAHuVcqSej5q0tJjn0JSMT+6+ip8klON1q8ixw==, tarball: https://registry.npmjs.org/@nlpjs/nlu/-/nlu-4.27.0.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/request@4.25.0':
|
|
||||||
resolution: {integrity: sha512-MPVYWfFZY03WyFL7GWkUkv8tw968OXsdxFSJEvjXHzhiCe/vAlPCWbvoR+VnoQTgzLHxs/KIF6sIF2s9AzsLmQ==, tarball: https://registry.npmjs.org/@nlpjs/request/-/request-4.25.0.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/sentiment@4.26.1':
|
|
||||||
resolution: {integrity: sha512-U2WmcW3w6yDDO45+Y7v5e6DPQj8e0x+RUUePPyRu2uIZmUtIKG+qCPMWnNLMmYQZoSQEFxmMMlLcGDC7tN7o3w==, tarball: https://registry.npmjs.org/@nlpjs/sentiment/-/sentiment-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/similarity@4.26.1':
|
|
||||||
resolution: {integrity: sha512-QutSBFGo/huNuz60PgqCjub0oBd9S8MLrjme33U5GzxuSvToQzXtn9/ynIia8qDm009D09VXV+LPeNE4h7yuSg==, tarball: https://registry.npmjs.org/@nlpjs/similarity/-/similarity-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/slot@4.26.1':
|
|
||||||
resolution: {integrity: sha512-mK8EEy5O+mRGne822PIKMxHSFh8j+iC7hGJ6T31XdFsNhFEYXLI/0dmeBstZgTSKBTe27HNFgCCwuGb77u0o9w==, tarball: https://registry.npmjs.org/@nlpjs/slot/-/slot-4.26.1.tgz}
|
|
||||||
|
|
||||||
'@nlpjs/xtables@4.25.0':
|
|
||||||
resolution: {integrity: sha512-+baCtMZIp+aDqODLQs8Wyyke5qUqQkL8AGWsZzwYuJV8S7xdW2+XklRnHnkFc3p3foC248TkzG5L8j9r6INOtg==, tarball: https://registry.npmjs.org/@nlpjs/xtables/-/xtables-4.25.0.tgz}
|
|
||||||
|
|
||||||
'@noble/hashes@1.8.0':
|
'@noble/hashes@1.8.0':
|
||||||
resolution: {integrity: sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==, tarball: https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz}
|
resolution: {integrity: sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==, tarball: https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz}
|
||||||
engines: {node: ^14.21.3 || >=16}
|
engines: {node: ^14.21.3 || >=16}
|
||||||
|
|
@ -1333,10 +1118,6 @@ packages:
|
||||||
resolution: {integrity: sha512-ID7fosbc50TbT0MK0EG12O+gAP3W3Aa/Pz4DaTtQtEvlc9Odaqi0de+xuZ7Li2GtK4HzEX7IuRWS/JmZLksR3Q==, tarball: https://registry.npmjs.org/@teppeis/multimaps/-/multimaps-3.0.0.tgz}
|
resolution: {integrity: sha512-ID7fosbc50TbT0MK0EG12O+gAP3W3Aa/Pz4DaTtQtEvlc9Odaqi0de+xuZ7Li2GtK4HzEX7IuRWS/JmZLksR3Q==, tarball: https://registry.npmjs.org/@teppeis/multimaps/-/multimaps-3.0.0.tgz}
|
||||||
engines: {node: '>=14'}
|
engines: {node: '>=14'}
|
||||||
|
|
||||||
'@tootallnate/once@2.0.1':
|
|
||||||
resolution: {integrity: sha512-HqmEUIGRJ5fSXchkVgR5F7qn48bDBzv0kWj/Kfu5e6uci4UlEeng4331LnBkWffb++Ei3FOVLxo8JJWMFBDMeQ==, tarball: https://registry.npmjs.org/@tootallnate/once/-/once-2.0.1.tgz}
|
|
||||||
engines: {node: '>= 10'}
|
|
||||||
|
|
||||||
'@types/babel__core@7.20.5':
|
'@types/babel__core@7.20.5':
|
||||||
resolution: {integrity: sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA==, tarball: https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz}
|
resolution: {integrity: sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA==, tarball: https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz}
|
||||||
|
|
||||||
|
|
@ -1506,10 +1287,6 @@ packages:
|
||||||
engines: {node: '>=0.4.0'}
|
engines: {node: '>=0.4.0'}
|
||||||
hasBin: true
|
hasBin: true
|
||||||
|
|
||||||
adler-32@1.3.1:
|
|
||||||
resolution: {integrity: sha512-ynZ4w/nUUv5rrsR8UUGoe1VC9hZj6V5hU9Qw1HlMDJGEJw5S7TfTErWTjMys6M7vr0YWcPqs3qAr4ss0nDfP+A==, tarball: https://registry.npmjs.org/adler-32/-/adler-32-1.3.1.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
|
|
||||||
agent-base@6.0.2:
|
agent-base@6.0.2:
|
||||||
resolution: {integrity: sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==, tarball: https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz}
|
resolution: {integrity: sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==, tarball: https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz}
|
||||||
engines: {node: '>= 6.0.0'}
|
engines: {node: '>= 6.0.0'}
|
||||||
|
|
@ -1613,9 +1390,6 @@ packages:
|
||||||
resolution: {integrity: sha512-Z7tMw1ytTXt5jqMcOP+OQteU1VuNK9Y02uuJtKQ1Sv69jXQKKg5cibLwGJow8yzZP+eAc18EmLGPal0bp36rvQ==, tarball: https://registry.npmjs.org/astral-regex/-/astral-regex-2.0.0.tgz}
|
resolution: {integrity: sha512-Z7tMw1ytTXt5jqMcOP+OQteU1VuNK9Y02uuJtKQ1Sv69jXQKKg5cibLwGJow8yzZP+eAc18EmLGPal0bp36rvQ==, tarball: https://registry.npmjs.org/astral-regex/-/astral-regex-2.0.0.tgz}
|
||||||
engines: {node: '>=8'}
|
engines: {node: '>=8'}
|
||||||
|
|
||||||
async@2.6.4:
|
|
||||||
resolution: {integrity: sha512-mzo5dfJYwAn29PeiJ0zvwTo04zj8HDJj0Mn8TD7sno7q12prdbnasKJHhkm2c1LgrhlJ0teaea8860oxi51mGA==, tarball: https://registry.npmjs.org/async/-/async-2.6.4.tgz}
|
|
||||||
|
|
||||||
async@3.2.6:
|
async@3.2.6:
|
||||||
resolution: {integrity: sha512-htCUDlxyyCLMgaM3xXg0C0LW2xqfuQ6p05pCEIsXuyQ+a1koYKTuBMzRNwmybfLgvJDMd0r1LTn4+E0Ti6C2AA==, tarball: https://registry.npmjs.org/async/-/async-3.2.6.tgz}
|
resolution: {integrity: sha512-htCUDlxyyCLMgaM3xXg0C0LW2xqfuQ6p05pCEIsXuyQ+a1koYKTuBMzRNwmybfLgvJDMd0r1LTn4+E0Ti6C2AA==, tarball: https://registry.npmjs.org/async/-/async-3.2.6.tgz}
|
||||||
|
|
||||||
|
|
@ -1657,9 +1431,6 @@ packages:
|
||||||
bcrypt-pbkdf@1.0.2:
|
bcrypt-pbkdf@1.0.2:
|
||||||
resolution: {integrity: sha512-qeFIXtP4MSoi6NLqO12WfqARWWuCKi2Rn/9hJLEmtB5yTNr9DqFWkJRCf2qShWzPeAMRnOgCrq0sg/KLv5ES9w==, tarball: https://registry.npmjs.org/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.2.tgz}
|
resolution: {integrity: sha512-qeFIXtP4MSoi6NLqO12WfqARWWuCKi2Rn/9hJLEmtB5yTNr9DqFWkJRCf2qShWzPeAMRnOgCrq0sg/KLv5ES9w==, tarball: https://registry.npmjs.org/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.2.tgz}
|
||||||
|
|
||||||
bignumber.js@7.2.1:
|
|
||||||
resolution: {integrity: sha512-S4XzBk5sMB+Rcb/LNcpzXr57VRTxgAvaAEDAl1AwRx27j00hT84O6OkteE7u8UB3NuaaygCRrEpqox4uDOrbdQ==, tarball: https://registry.npmjs.org/bignumber.js/-/bignumber.js-7.2.1.tgz}
|
|
||||||
|
|
||||||
binary-extensions@2.3.0:
|
binary-extensions@2.3.0:
|
||||||
resolution: {integrity: sha512-Ceh+7ox5qe7LJuLHoY0feh3pHuUDHAcRUeyL2VYghZwfpkNIy/+8Ocg0a3UuSoYzavmylwuLWQOf3hl0jjMMIw==, tarball: https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.3.0.tgz}
|
resolution: {integrity: sha512-Ceh+7ox5qe7LJuLHoY0feh3pHuUDHAcRUeyL2VYghZwfpkNIy/+8Ocg0a3UuSoYzavmylwuLWQOf3hl0jjMMIw==, tarball: https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.3.0.tgz}
|
||||||
engines: {node: '>=8'}
|
engines: {node: '>=8'}
|
||||||
|
|
@ -1748,10 +1519,6 @@ packages:
|
||||||
caseless@0.12.0:
|
caseless@0.12.0:
|
||||||
resolution: {integrity: sha512-4tYFyifaFfGacoiObjJegolkwSU4xQNGbVgUiNYVUxbQ2x2lUsFvY4hVgVzGiIe6WLOPqycWXA40l+PWsxthUw==, tarball: https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz}
|
resolution: {integrity: sha512-4tYFyifaFfGacoiObjJegolkwSU4xQNGbVgUiNYVUxbQ2x2lUsFvY4hVgVzGiIe6WLOPqycWXA40l+PWsxthUw==, tarball: https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz}
|
||||||
|
|
||||||
cfb@1.2.2:
|
|
||||||
resolution: {integrity: sha512-KfdUZsSOw19/ObEWasvBP/Ac4reZvAGauZhs6S/gqNhXhI7cKwvlH7ulj+dOEYnca4bm4SGo8C1bTAQvnTjgQA==, tarball: https://registry.npmjs.org/cfb/-/cfb-1.2.2.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
|
|
||||||
chai@5.3.3:
|
chai@5.3.3:
|
||||||
resolution: {integrity: sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==, tarball: https://registry.npmjs.org/chai/-/chai-5.3.3.tgz}
|
resolution: {integrity: sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==, tarball: https://registry.npmjs.org/chai/-/chai-5.3.3.tgz}
|
||||||
engines: {node: '>=18'}
|
engines: {node: '>=18'}
|
||||||
|
|
@ -1822,10 +1589,6 @@ packages:
|
||||||
resolution: {integrity: sha512-JQHZ2QMW6l3aH/j6xCqQThY/9OH4D/9ls34cgkUBiEeocRTU04tHfKPBsUK1PqZCUQM7GiA0IIXJSuXHI64Kbg==, tarball: https://registry.npmjs.org/clone/-/clone-1.0.4.tgz}
|
resolution: {integrity: sha512-JQHZ2QMW6l3aH/j6xCqQThY/9OH4D/9ls34cgkUBiEeocRTU04tHfKPBsUK1PqZCUQM7GiA0IIXJSuXHI64Kbg==, tarball: https://registry.npmjs.org/clone/-/clone-1.0.4.tgz}
|
||||||
engines: {node: '>=0.8'}
|
engines: {node: '>=0.8'}
|
||||||
|
|
||||||
codepage@1.15.0:
|
|
||||||
resolution: {integrity: sha512-3g6NUTPd/YtuuGrhMnOMRjFc+LJw/bnMp3+0r/Wcz3IXUuCosKRJvMphm5+Q+bvTVGcJJuRvVLuYba+WojaFaA==, tarball: https://registry.npmjs.org/codepage/-/codepage-1.15.0.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
|
|
||||||
color-convert@2.0.1:
|
color-convert@2.0.1:
|
||||||
resolution: {integrity: sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==, tarball: https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz}
|
resolution: {integrity: sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==, tarball: https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz}
|
||||||
engines: {node: '>=7.0.0'}
|
engines: {node: '>=7.0.0'}
|
||||||
|
|
@ -1940,11 +1703,6 @@ packages:
|
||||||
typescript:
|
typescript:
|
||||||
optional: true
|
optional: true
|
||||||
|
|
||||||
crc-32@1.2.2:
|
|
||||||
resolution: {integrity: sha512-ROmzCKrTnOwybPcJApAA6WBWij23HVfGVNKqqrZpuyZOHqK2CwHSvpGuyt/UNNvaIjEd8X5IFGp4Mh+Ie1IHJQ==, tarball: https://registry.npmjs.org/crc-32/-/crc-32-1.2.2.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
hasBin: true
|
|
||||||
|
|
||||||
cross-env@10.1.0:
|
cross-env@10.1.0:
|
||||||
resolution: {integrity: sha512-GsYosgnACZTADcmEyJctkJIoqAhHjttw7RsFrVoJNXbsWWqaq6Ym+7kZjq6mS45O0jij6vtiReppKQEtqWy6Dw==, tarball: https://registry.npmjs.org/cross-env/-/cross-env-10.1.0.tgz}
|
resolution: {integrity: sha512-GsYosgnACZTADcmEyJctkJIoqAhHjttw7RsFrVoJNXbsWWqaq6Ym+7kZjq6mS45O0jij6vtiReppKQEtqWy6Dw==, tarball: https://registry.npmjs.org/cross-env/-/cross-env-10.1.0.tgz}
|
||||||
engines: {node: '>=20'}
|
engines: {node: '>=20'}
|
||||||
|
|
@ -2132,9 +1890,6 @@ packages:
|
||||||
resolution: {integrity: sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==, tarball: https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz}
|
resolution: {integrity: sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==, tarball: https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz}
|
||||||
engines: {node: '>=12'}
|
engines: {node: '>=12'}
|
||||||
|
|
||||||
doublearray@0.0.2:
|
|
||||||
resolution: {integrity: sha512-aw55FtZzT6AmiamEj2kvmR6BuFqvYgKZUkfQ7teqVRNqD5UE0rw8IeW/3gieHNKQ5sPuDKlljWEn4bzv5+1bHw==, tarball: https://registry.npmjs.org/doublearray/-/doublearray-0.0.2.tgz}
|
|
||||||
|
|
||||||
dunder-proto@1.0.1:
|
dunder-proto@1.0.1:
|
||||||
resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==, tarball: https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz}
|
resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==, tarball: https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz}
|
||||||
engines: {node: '>= 0.4'}
|
engines: {node: '>= 0.4'}
|
||||||
|
|
@ -2401,10 +2156,6 @@ packages:
|
||||||
resolution: {integrity: sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==, tarball: https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz}
|
resolution: {integrity: sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==, tarball: https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz}
|
||||||
engines: {node: '>= 0.6'}
|
engines: {node: '>= 0.6'}
|
||||||
|
|
||||||
frac@1.1.2:
|
|
||||||
resolution: {integrity: sha512-w/XBfkibaTl3YDqASwfDUqkna4Z2p9cFSr1aHDt0WoMTECnRfBOv2WArlZILlqgWlmdIlALXGpM2AOhEk5W3IA==, tarball: https://registry.npmjs.org/frac/-/frac-1.1.2.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
|
|
||||||
fresh@0.5.2:
|
fresh@0.5.2:
|
||||||
resolution: {integrity: sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q==, tarball: https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz}
|
resolution: {integrity: sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q==, tarball: https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz}
|
||||||
engines: {node: '>= 0.6'}
|
engines: {node: '>= 0.6'}
|
||||||
|
|
@ -2510,9 +2261,6 @@ packages:
|
||||||
graceful-fs@4.2.11:
|
graceful-fs@4.2.11:
|
||||||
resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==, tarball: https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz}
|
resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==, tarball: https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz}
|
||||||
|
|
||||||
grapheme-splitter@1.0.4:
|
|
||||||
resolution: {integrity: sha512-bzh50DW9kTPM00T8y4o8vQg89Di9oLJVLW/KaOGIXJWP/iqCN6WKYkbNOF04vFLJhwcpYUh9ydh/+5vpOqV4YQ==, tarball: https://registry.npmjs.org/grapheme-splitter/-/grapheme-splitter-1.0.4.tgz}
|
|
||||||
|
|
||||||
has-ansi@4.0.1:
|
has-ansi@4.0.1:
|
||||||
resolution: {integrity: sha512-Qr4RtTm30xvEdqUXbSBVWDu+PrTokJOwe/FU+VdfJPk+MXAPoeOzKpRyrDTnZIJwAkQ4oBLTU53nu0HrkF/Z2A==, tarball: https://registry.npmjs.org/has-ansi/-/has-ansi-4.0.1.tgz}
|
resolution: {integrity: sha512-Qr4RtTm30xvEdqUXbSBVWDu+PrTokJOwe/FU+VdfJPk+MXAPoeOzKpRyrDTnZIJwAkQ4oBLTU53nu0HrkF/Z2A==, tarball: https://registry.npmjs.org/has-ansi/-/has-ansi-4.0.1.tgz}
|
||||||
engines: {node: '>=8'}
|
engines: {node: '>=8'}
|
||||||
|
|
@ -2558,10 +2306,6 @@ packages:
|
||||||
resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==, tarball: https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz}
|
resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==, tarball: https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz}
|
||||||
engines: {node: '>= 0.8'}
|
engines: {node: '>= 0.8'}
|
||||||
|
|
||||||
http-proxy-agent@5.0.0:
|
|
||||||
resolution: {integrity: sha512-n2hY8YdoRE1i7r6M0w9DIw5GgZN0G25P8zLCRQ8rjXtTU3vsNFBI/vWK/UIeE6g5MUUz6avwAPXmL6Fy9D/90w==, tarball: https://registry.npmjs.org/http-proxy-agent/-/http-proxy-agent-5.0.0.tgz}
|
|
||||||
engines: {node: '>= 6'}
|
|
||||||
|
|
||||||
http-signature@1.4.0:
|
http-signature@1.4.0:
|
||||||
resolution: {integrity: sha512-G5akfn7eKbpDN+8nPS/cb57YeA1jLTVxjpCj7tmm3QKPdyDy7T+qSC40e9ptydSWvkwjSXw1VbkpyEm39ukeAg==, tarball: https://registry.npmjs.org/http-signature/-/http-signature-1.4.0.tgz}
|
resolution: {integrity: sha512-G5akfn7eKbpDN+8nPS/cb57YeA1jLTVxjpCj7tmm3QKPdyDy7T+qSC40e9ptydSWvkwjSXw1VbkpyEm39ukeAg==, tarball: https://registry.npmjs.org/http-signature/-/http-signature-1.4.0.tgz}
|
||||||
engines: {node: '>=0.10'}
|
engines: {node: '>=0.10'}
|
||||||
|
|
@ -2814,9 +2558,6 @@ packages:
|
||||||
knuth-shuffle-seeded@1.0.6:
|
knuth-shuffle-seeded@1.0.6:
|
||||||
resolution: {integrity: sha512-9pFH0SplrfyKyojCLxZfMcvkhf5hH0d+UwR9nTVJ/DDQJGuzcXjTwB7TP7sDfehSudlGGaOLblmEWqv04ERVWg==, tarball: https://registry.npmjs.org/knuth-shuffle-seeded/-/knuth-shuffle-seeded-1.0.6.tgz}
|
resolution: {integrity: sha512-9pFH0SplrfyKyojCLxZfMcvkhf5hH0d+UwR9nTVJ/DDQJGuzcXjTwB7TP7sDfehSudlGGaOLblmEWqv04ERVWg==, tarball: https://registry.npmjs.org/knuth-shuffle-seeded/-/knuth-shuffle-seeded-1.0.6.tgz}
|
||||||
|
|
||||||
kuromoji@0.1.2:
|
|
||||||
resolution: {integrity: sha512-V0dUf+C2LpcPEXhoHLMAop/bOht16Dyr+mDiIE39yX3vqau7p80De/koFqpiTcL1zzdZlc3xuHZ8u5gjYRfFaQ==, tarball: https://registry.npmjs.org/kuromoji/-/kuromoji-0.1.2.tgz}
|
|
||||||
|
|
||||||
lazy-ass@1.6.0:
|
lazy-ass@1.6.0:
|
||||||
resolution: {integrity: sha512-cc8oEVoctTvsFZ/Oje/kGnHbpWHYBe8IAJe4C0QNc3t8uM/0Y8+erSz/7Y1ALuXTEZTMvxXwO6YbX1ey3ujiZw==, tarball: https://registry.npmjs.org/lazy-ass/-/lazy-ass-1.6.0.tgz}
|
resolution: {integrity: sha512-cc8oEVoctTvsFZ/Oje/kGnHbpWHYBe8IAJe4C0QNc3t8uM/0Y8+erSz/7Y1ALuXTEZTMvxXwO6YbX1ey3ujiZw==, tarball: https://registry.npmjs.org/lazy-ass/-/lazy-ass-1.6.0.tgz}
|
||||||
engines: {node: '> 0.8'}
|
engines: {node: '> 0.8'}
|
||||||
|
|
@ -3081,9 +2822,6 @@ packages:
|
||||||
node-html-parser@5.3.3:
|
node-html-parser@5.3.3:
|
||||||
resolution: {integrity: sha512-ncg1033CaX9UexbyA7e1N0aAoAYRDiV8jkTvzEnfd1GDvzFdrsXLzR4p4ik8mwLgnaKP/jyUFWDy9q3jvRT2Jw==, tarball: https://registry.npmjs.org/node-html-parser/-/node-html-parser-5.3.3.tgz}
|
resolution: {integrity: sha512-ncg1033CaX9UexbyA7e1N0aAoAYRDiV8jkTvzEnfd1GDvzFdrsXLzR4p4ik8mwLgnaKP/jyUFWDy9q3jvRT2Jw==, tarball: https://registry.npmjs.org/node-html-parser/-/node-html-parser-5.3.3.tgz}
|
||||||
|
|
||||||
node-nlp@4.27.0:
|
|
||||||
resolution: {integrity: sha512-LnkhOUPXX0CMFbSzJ1gHI+7Yb3ULLip5gRsqedXb6pryjcRCbNzPgHXcH/6G9B1vSbDfO+y3X2B4QZpfP12OyQ==, tarball: https://registry.npmjs.org/node-nlp/-/node-nlp-4.27.0.tgz}
|
|
||||||
|
|
||||||
node-releases@2.0.53:
|
node-releases@2.0.53:
|
||||||
resolution: {integrity: sha512-D9UOmYG3UH1V+ENW56t5QXBwJw1YEY18ruVeus89Rw+SyIgjPkCO84bRzO3uNIYosJbNwiabWVn48o3uJLjxFQ==, tarball: https://registry.npmjs.org/node-releases/-/node-releases-2.0.53.tgz}
|
resolution: {integrity: sha512-D9UOmYG3UH1V+ENW56t5QXBwJw1YEY18ruVeus89Rw+SyIgjPkCO84bRzO3uNIYosJbNwiabWVn48o3uJLjxFQ==, tarball: https://registry.npmjs.org/node-releases/-/node-releases-2.0.53.tgz}
|
||||||
engines: {node: '>=18'}
|
engines: {node: '>=18'}
|
||||||
|
|
@ -3633,10 +3371,6 @@ packages:
|
||||||
split@1.0.1:
|
split@1.0.1:
|
||||||
resolution: {integrity: sha512-mTyOoPbrivtXnwnIxZRFYRrPNtEFKlpB2fvjSnCQUiAA6qAZzqwna5envK4uk6OIeP17CsdF3rSBGYVBsU0Tkg==, tarball: https://registry.npmjs.org/split/-/split-1.0.1.tgz}
|
resolution: {integrity: sha512-mTyOoPbrivtXnwnIxZRFYRrPNtEFKlpB2fvjSnCQUiAA6qAZzqwna5envK4uk6OIeP17CsdF3rSBGYVBsU0Tkg==, tarball: https://registry.npmjs.org/split/-/split-1.0.1.tgz}
|
||||||
|
|
||||||
ssf@0.11.2:
|
|
||||||
resolution: {integrity: sha512-+idbmIXoYET47hH+d7dfm2epdOMUDjqcB4648sTZ+t2JwoyBFL/insLfB/racrDmsKB3diwsDA696pZMieAC5g==, tarball: https://registry.npmjs.org/ssf/-/ssf-0.11.2.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
|
|
||||||
sshpk@1.18.0:
|
sshpk@1.18.0:
|
||||||
resolution: {integrity: sha512-2p2KJZTSqQ/I3+HX42EpYOa2l3f8Erv8MWKsy2I9uf4wA7yFIkXRffYdsx86y6z4vHtV8u7g+pPlr8/4ouAxsQ==, tarball: https://registry.npmjs.org/sshpk/-/sshpk-1.18.0.tgz}
|
resolution: {integrity: sha512-2p2KJZTSqQ/I3+HX42EpYOa2l3f8Erv8MWKsy2I9uf4wA7yFIkXRffYdsx86y6z4vHtV8u7g+pPlr8/4ouAxsQ==, tarball: https://registry.npmjs.org/sshpk/-/sshpk-1.18.0.tgz}
|
||||||
engines: {node: '>=0.10.0'}
|
engines: {node: '>=0.10.0'}
|
||||||
|
|
@ -3977,14 +3711,6 @@ packages:
|
||||||
wide-align@1.1.5:
|
wide-align@1.1.5:
|
||||||
resolution: {integrity: sha512-eDMORYaPNZ4sQIuuYPDHdQvf4gyCF9rEEV/yPxGfwPkRodwEgiMUUXTx/dex+Me0wxx53S+NgUHaP7y3MGlDmg==, tarball: https://registry.npmjs.org/wide-align/-/wide-align-1.1.5.tgz}
|
resolution: {integrity: sha512-eDMORYaPNZ4sQIuuYPDHdQvf4gyCF9rEEV/yPxGfwPkRodwEgiMUUXTx/dex+Me0wxx53S+NgUHaP7y3MGlDmg==, tarball: https://registry.npmjs.org/wide-align/-/wide-align-1.1.5.tgz}
|
||||||
|
|
||||||
wmf@1.0.2:
|
|
||||||
resolution: {integrity: sha512-/p9K7bEh0Dj6WbXg4JG0xvLQmIadrner1bi45VMJTfnbVHsc7yIajZyoSoK60/dtVBs12Fm6WkUI5/3WAVsNMw==, tarball: https://registry.npmjs.org/wmf/-/wmf-1.0.2.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
|
|
||||||
word@0.3.0:
|
|
||||||
resolution: {integrity: sha512-OELeY0Q61OXpdUfTp+oweA/vtLVg5VDOXh+3he3PNzLGG/y0oylSOC1xRVj0+l4vQ3tj/bB1HVHv1ocXkQceFA==, tarball: https://registry.npmjs.org/word/-/word-0.3.0.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
|
|
||||||
workerpool@6.5.1:
|
workerpool@6.5.1:
|
||||||
resolution: {integrity: sha512-Fs4dNYcsdpYSAfVxhnl1L5zTksjvOJxtC5hzMNl+1t9B8hTJTdKDyZ5ju7ztgPy+ft9tBFXoOlDNiOT9WUXZlA==, tarball: https://registry.npmjs.org/workerpool/-/workerpool-6.5.1.tgz}
|
resolution: {integrity: sha512-Fs4dNYcsdpYSAfVxhnl1L5zTksjvOJxtC5hzMNl+1t9B8hTJTdKDyZ5ju7ztgPy+ft9tBFXoOlDNiOT9WUXZlA==, tarball: https://registry.npmjs.org/workerpool/-/workerpool-6.5.1.tgz}
|
||||||
|
|
||||||
|
|
@ -4006,11 +3732,6 @@ packages:
|
||||||
wrappy@1.0.2:
|
wrappy@1.0.2:
|
||||||
resolution: {integrity: sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==, tarball: https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz}
|
resolution: {integrity: sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==, tarball: https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz}
|
||||||
|
|
||||||
xlsx@0.18.5:
|
|
||||||
resolution: {integrity: sha512-dmg3LCjBPHZnQp5/F/+nnTa+miPJxUXB6vtk42YjBBKayDNagxGEeIdWApkYPOf3Z3pm3k62Knjzp7lMeTEtFQ==, tarball: https://registry.npmjs.org/xlsx/-/xlsx-0.18.5.tgz}
|
|
||||||
engines: {node: '>=0.8'}
|
|
||||||
hasBin: true
|
|
||||||
|
|
||||||
xmlbuilder@15.1.1:
|
xmlbuilder@15.1.1:
|
||||||
resolution: {integrity: sha512-yMqGBqtXyeN1e3TGYvgNgDVZ3j84W4cwkOXQswghol6APgZWaff9lnbvN7MHYJOiXsvGPXtjTYJEiC9J2wv9Eg==, tarball: https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-15.1.1.tgz}
|
resolution: {integrity: sha512-yMqGBqtXyeN1e3TGYvgNgDVZ3j84W4cwkOXQswghol6APgZWaff9lnbvN7MHYJOiXsvGPXtjTYJEiC9J2wv9Eg==, tarball: https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-15.1.1.tgz}
|
||||||
engines: {node: '>=8.0'}
|
engines: {node: '>=8.0'}
|
||||||
|
|
@ -4064,9 +3785,6 @@ packages:
|
||||||
yup@1.6.1:
|
yup@1.6.1:
|
||||||
resolution: {integrity: sha512-JED8pB50qbA4FOkDol0bYF/p60qSEDQqBD0/qeIrUCG1KbPBIQ776fCUNb9ldbPcSTxA69g/47XTo4TqWiuXOA==, tarball: https://registry.npmjs.org/yup/-/yup-1.6.1.tgz}
|
resolution: {integrity: sha512-JED8pB50qbA4FOkDol0bYF/p60qSEDQqBD0/qeIrUCG1KbPBIQ776fCUNb9ldbPcSTxA69g/47XTo4TqWiuXOA==, tarball: https://registry.npmjs.org/yup/-/yup-1.6.1.tgz}
|
||||||
|
|
||||||
zlibjs@0.3.1:
|
|
||||||
resolution: {integrity: sha512-+J9RrgTKOmlxFSDHo0pI1xM6BLVUv+o0ZT9ANtCxGkjIVCCUdx9alUF8Gm+dGLKbkkkidWIHFDZHDMpfITt4+w==, tarball: https://registry.npmjs.org/zlibjs/-/zlibjs-0.3.1.tgz}
|
|
||||||
|
|
||||||
zod@3.25.76:
|
zod@3.25.76:
|
||||||
resolution: {integrity: sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==, tarball: https://registry.npmjs.org/zod/-/zod-3.25.76.tgz}
|
resolution: {integrity: sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==, tarball: https://registry.npmjs.org/zod/-/zod-3.25.76.tgz}
|
||||||
|
|
||||||
|
|
@ -4675,344 +4393,9 @@ snapshots:
|
||||||
- encoding
|
- encoding
|
||||||
- supports-color
|
- supports-color
|
||||||
|
|
||||||
'@microsoft/recognizers-text-choice@1.3.1':
|
|
||||||
dependencies:
|
|
||||||
'@microsoft/recognizers-text': 1.3.1
|
|
||||||
grapheme-splitter: 1.0.4
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-data-types-timex-expression@1.3.1': {}
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-date-time@1.3.2':
|
|
||||||
dependencies:
|
|
||||||
'@microsoft/recognizers-text': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-number': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-number-with-unit': 1.3.1
|
|
||||||
lodash: 4.18.1
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-number-with-unit@1.3.1':
|
|
||||||
dependencies:
|
|
||||||
'@microsoft/recognizers-text': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-number': 1.3.1
|
|
||||||
lodash: 4.18.1
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-number@1.3.1':
|
|
||||||
dependencies:
|
|
||||||
'@microsoft/recognizers-text': 1.3.1
|
|
||||||
bignumber.js: 7.2.1
|
|
||||||
lodash: 4.18.1
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-sequence@1.3.1':
|
|
||||||
dependencies:
|
|
||||||
'@microsoft/recognizers-text': 1.3.1
|
|
||||||
grapheme-splitter: 1.0.4
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text-suite@1.3.0':
|
|
||||||
dependencies:
|
|
||||||
'@microsoft/recognizers-text': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-choice': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-data-types-timex-expression': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-date-time': 1.3.2
|
|
||||||
'@microsoft/recognizers-text-number': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-number-with-unit': 1.3.1
|
|
||||||
'@microsoft/recognizers-text-sequence': 1.3.1
|
|
||||||
|
|
||||||
'@microsoft/recognizers-text@1.3.1': {}
|
|
||||||
|
|
||||||
'@napi-rs/lzma-linux-x64-gnu@1.5.1':
|
'@napi-rs/lzma-linux-x64-gnu@1.5.1':
|
||||||
optional: true
|
optional: true
|
||||||
|
|
||||||
'@nlpjs/builtin-duckling@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/builtin-microsoft@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@microsoft/recognizers-text-suite': 1.3.0
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/core-loader@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/request': 4.25.0
|
|
||||||
transitivePeerDependencies:
|
|
||||||
- supports-color
|
|
||||||
|
|
||||||
'@nlpjs/core@4.26.1': {}
|
|
||||||
|
|
||||||
'@nlpjs/emoji@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/evaluator@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
escodegen: 2.1.0
|
|
||||||
esprima: 4.0.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-all@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/lang-ar': 4.26.1
|
|
||||||
'@nlpjs/lang-bn': 4.26.1
|
|
||||||
'@nlpjs/lang-ca': 4.26.1
|
|
||||||
'@nlpjs/lang-cs': 4.26.1
|
|
||||||
'@nlpjs/lang-da': 4.26.1
|
|
||||||
'@nlpjs/lang-de': 4.26.1
|
|
||||||
'@nlpjs/lang-el': 4.26.1
|
|
||||||
'@nlpjs/lang-en': 4.26.1
|
|
||||||
'@nlpjs/lang-es': 4.26.1
|
|
||||||
'@nlpjs/lang-eu': 4.26.1
|
|
||||||
'@nlpjs/lang-fa': 4.26.1
|
|
||||||
'@nlpjs/lang-fi': 4.26.1
|
|
||||||
'@nlpjs/lang-fr': 4.26.1
|
|
||||||
'@nlpjs/lang-ga': 4.26.1
|
|
||||||
'@nlpjs/lang-gl': 4.26.1
|
|
||||||
'@nlpjs/lang-hi': 4.26.1
|
|
||||||
'@nlpjs/lang-hu': 4.26.1
|
|
||||||
'@nlpjs/lang-hy': 4.26.1
|
|
||||||
'@nlpjs/lang-id': 4.26.1
|
|
||||||
'@nlpjs/lang-it': 4.26.1
|
|
||||||
'@nlpjs/lang-ja': 4.26.1
|
|
||||||
'@nlpjs/lang-ko': 4.26.1
|
|
||||||
'@nlpjs/lang-lt': 4.26.1
|
|
||||||
'@nlpjs/lang-ms': 4.26.1
|
|
||||||
'@nlpjs/lang-ne': 4.26.1
|
|
||||||
'@nlpjs/lang-nl': 4.26.1
|
|
||||||
'@nlpjs/lang-no': 4.26.1
|
|
||||||
'@nlpjs/lang-pl': 4.26.1
|
|
||||||
'@nlpjs/lang-pt': 4.26.1
|
|
||||||
'@nlpjs/lang-ro': 4.26.1
|
|
||||||
'@nlpjs/lang-ru': 4.26.1
|
|
||||||
'@nlpjs/lang-sl': 4.26.1
|
|
||||||
'@nlpjs/lang-sr': 4.26.1
|
|
||||||
'@nlpjs/lang-sv': 4.26.1
|
|
||||||
'@nlpjs/lang-ta': 4.26.1
|
|
||||||
'@nlpjs/lang-th': 4.26.1
|
|
||||||
'@nlpjs/lang-tl': 4.26.1
|
|
||||||
'@nlpjs/lang-tr': 4.26.1
|
|
||||||
'@nlpjs/lang-uk': 4.26.1
|
|
||||||
'@nlpjs/lang-zh': 4.26.1
|
|
||||||
'@nlpjs/language': 4.25.0
|
|
||||||
|
|
||||||
'@nlpjs/lang-ar@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-bn@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ca@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-cs@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-da@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-de@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-el@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-en-min@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-en@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/lang-en-min': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-es@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-eu@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-fa@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-fi@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-fr@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ga@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-gl@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-hi@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-hu@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-hy@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-id@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-it@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ja@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
kuromoji: 0.1.2
|
|
||||||
|
|
||||||
'@nlpjs/lang-ko@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-lt@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ms@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/lang-id': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ne@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-nl@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-no@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-pl@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-pt@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ro@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ru@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-sl@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-sr@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-sv@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-ta@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-th@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-tl@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-tr@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-uk@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/lang-zh@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/language-min@4.25.0': {}
|
|
||||||
|
|
||||||
'@nlpjs/language@4.25.0': {}
|
|
||||||
|
|
||||||
'@nlpjs/ner@4.27.0':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/language-min': 4.25.0
|
|
||||||
'@nlpjs/similarity': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/neural@4.25.0': {}
|
|
||||||
|
|
||||||
'@nlpjs/nlg@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/nlp@4.27.0':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/ner': 4.27.0
|
|
||||||
'@nlpjs/nlg': 4.26.1
|
|
||||||
'@nlpjs/nlu': 4.27.0
|
|
||||||
'@nlpjs/sentiment': 4.26.1
|
|
||||||
'@nlpjs/slot': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/nlu@4.27.0':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/language-min': 4.25.0
|
|
||||||
'@nlpjs/neural': 4.25.0
|
|
||||||
'@nlpjs/similarity': 4.26.1
|
|
||||||
|
|
||||||
'@nlpjs/request@4.25.0':
|
|
||||||
dependencies:
|
|
||||||
http-proxy-agent: 5.0.0
|
|
||||||
https-proxy-agent: 5.0.1
|
|
||||||
transitivePeerDependencies:
|
|
||||||
- supports-color
|
|
||||||
|
|
||||||
'@nlpjs/sentiment@4.26.1':
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/core': 4.26.1
|
|
||||||
'@nlpjs/language-min': 4.25.0
|
|
||||||
'@nlpjs/neural': 4.25.0
|
|
||||||
|
|
||||||
'@nlpjs/similarity@4.26.1': {}
|
|
||||||
|
|
||||||
'@nlpjs/slot@4.26.1': {}
|
|
||||||
|
|
||||||
'@nlpjs/xtables@4.25.0':
|
|
||||||
dependencies:
|
|
||||||
xlsx: 0.18.5
|
|
||||||
|
|
||||||
'@noble/hashes@1.8.0': {}
|
'@noble/hashes@1.8.0': {}
|
||||||
|
|
||||||
'@nodelib/fs.scandir@2.1.5':
|
'@nodelib/fs.scandir@2.1.5':
|
||||||
|
|
@ -5199,8 +4582,6 @@ snapshots:
|
||||||
|
|
||||||
'@teppeis/multimaps@3.0.0': {}
|
'@teppeis/multimaps@3.0.0': {}
|
||||||
|
|
||||||
'@tootallnate/once@2.0.1': {}
|
|
||||||
|
|
||||||
'@types/babel__core@7.20.5':
|
'@types/babel__core@7.20.5':
|
||||||
dependencies:
|
dependencies:
|
||||||
'@babel/parser': 7.29.8
|
'@babel/parser': 7.29.8
|
||||||
|
|
@ -5423,8 +4804,6 @@ snapshots:
|
||||||
|
|
||||||
acorn@8.18.0: {}
|
acorn@8.18.0: {}
|
||||||
|
|
||||||
adler-32@1.3.1: {}
|
|
||||||
|
|
||||||
agent-base@6.0.2:
|
agent-base@6.0.2:
|
||||||
dependencies:
|
dependencies:
|
||||||
debug: 4.4.3(supports-color@8.1.1)
|
debug: 4.4.3(supports-color@8.1.1)
|
||||||
|
|
@ -5514,10 +4893,6 @@ snapshots:
|
||||||
|
|
||||||
astral-regex@2.0.0: {}
|
astral-regex@2.0.0: {}
|
||||||
|
|
||||||
async@2.6.4:
|
|
||||||
dependencies:
|
|
||||||
lodash: 4.18.1
|
|
||||||
|
|
||||||
async@3.2.6: {}
|
async@3.2.6: {}
|
||||||
|
|
||||||
asynckit@0.4.0: {}
|
asynckit@0.4.0: {}
|
||||||
|
|
@ -5554,8 +4929,6 @@ snapshots:
|
||||||
dependencies:
|
dependencies:
|
||||||
tweetnacl: 0.14.5
|
tweetnacl: 0.14.5
|
||||||
|
|
||||||
bignumber.js@7.2.1: {}
|
|
||||||
|
|
||||||
binary-extensions@2.3.0: {}
|
binary-extensions@2.3.0: {}
|
||||||
|
|
||||||
blob-util@2.0.2: {}
|
blob-util@2.0.2: {}
|
||||||
|
|
@ -5654,11 +5027,6 @@ snapshots:
|
||||||
|
|
||||||
caseless@0.12.0: {}
|
caseless@0.12.0: {}
|
||||||
|
|
||||||
cfb@1.2.2:
|
|
||||||
dependencies:
|
|
||||||
adler-32: 1.3.1
|
|
||||||
crc-32: 1.2.2
|
|
||||||
|
|
||||||
chai@5.3.3:
|
chai@5.3.3:
|
||||||
dependencies:
|
dependencies:
|
||||||
assertion-error: 2.0.1
|
assertion-error: 2.0.1
|
||||||
|
|
@ -5738,8 +5106,6 @@ snapshots:
|
||||||
clone@1.0.4:
|
clone@1.0.4:
|
||||||
optional: true
|
optional: true
|
||||||
|
|
||||||
codepage@1.15.0: {}
|
|
||||||
|
|
||||||
color-convert@2.0.1:
|
color-convert@2.0.1:
|
||||||
dependencies:
|
dependencies:
|
||||||
color-name: 1.1.4
|
color-name: 1.1.4
|
||||||
|
|
@ -5821,8 +5187,6 @@ snapshots:
|
||||||
optionalDependencies:
|
optionalDependencies:
|
||||||
typescript: 5.9.3
|
typescript: 5.9.3
|
||||||
|
|
||||||
crc-32@1.2.2: {}
|
|
||||||
|
|
||||||
cross-env@10.1.0:
|
cross-env@10.1.0:
|
||||||
dependencies:
|
dependencies:
|
||||||
'@epic-web/invariant': 1.0.0
|
'@epic-web/invariant': 1.0.0
|
||||||
|
|
@ -6067,8 +5431,6 @@ snapshots:
|
||||||
|
|
||||||
dotenv@16.6.1: {}
|
dotenv@16.6.1: {}
|
||||||
|
|
||||||
doublearray@0.0.2: {}
|
|
||||||
|
|
||||||
dunder-proto@1.0.1:
|
dunder-proto@1.0.1:
|
||||||
dependencies:
|
dependencies:
|
||||||
call-bind-apply-helpers: 1.0.2
|
call-bind-apply-helpers: 1.0.2
|
||||||
|
|
@ -6461,8 +5823,6 @@ snapshots:
|
||||||
|
|
||||||
forwarded@0.2.0: {}
|
forwarded@0.2.0: {}
|
||||||
|
|
||||||
frac@1.1.2: {}
|
|
||||||
|
|
||||||
fresh@0.5.2: {}
|
fresh@0.5.2: {}
|
||||||
|
|
||||||
from@0.1.7: {}
|
from@0.1.7: {}
|
||||||
|
|
@ -6584,8 +5944,6 @@ snapshots:
|
||||||
|
|
||||||
graceful-fs@4.2.11: {}
|
graceful-fs@4.2.11: {}
|
||||||
|
|
||||||
grapheme-splitter@1.0.4: {}
|
|
||||||
|
|
||||||
has-ansi@4.0.1:
|
has-ansi@4.0.1:
|
||||||
dependencies:
|
dependencies:
|
||||||
ansi-regex: 4.1.1
|
ansi-regex: 4.1.1
|
||||||
|
|
@ -6626,14 +5984,6 @@ snapshots:
|
||||||
statuses: 2.0.2
|
statuses: 2.0.2
|
||||||
toidentifier: 1.0.1
|
toidentifier: 1.0.1
|
||||||
|
|
||||||
http-proxy-agent@5.0.0:
|
|
||||||
dependencies:
|
|
||||||
'@tootallnate/once': 2.0.1
|
|
||||||
agent-base: 6.0.2
|
|
||||||
debug: 4.4.3(supports-color@8.1.1)
|
|
||||||
transitivePeerDependencies:
|
|
||||||
- supports-color
|
|
||||||
|
|
||||||
http-signature@1.4.0:
|
http-signature@1.4.0:
|
||||||
dependencies:
|
dependencies:
|
||||||
assert-plus: 1.0.0
|
assert-plus: 1.0.0
|
||||||
|
|
@ -6876,12 +6226,6 @@ snapshots:
|
||||||
dependencies:
|
dependencies:
|
||||||
seed-random: 2.2.0
|
seed-random: 2.2.0
|
||||||
|
|
||||||
kuromoji@0.1.2:
|
|
||||||
dependencies:
|
|
||||||
async: 2.6.4
|
|
||||||
doublearray: 0.0.2
|
|
||||||
zlibjs: 0.3.1
|
|
||||||
|
|
||||||
lazy-ass@1.6.0: {}
|
lazy-ass@1.6.0: {}
|
||||||
|
|
||||||
lazy-ass@2.0.3: {}
|
lazy-ass@2.0.3: {}
|
||||||
|
|
@ -7133,26 +6477,6 @@ snapshots:
|
||||||
css-select: 4.3.0
|
css-select: 4.3.0
|
||||||
he: 1.2.0
|
he: 1.2.0
|
||||||
|
|
||||||
node-nlp@4.27.0:
|
|
||||||
dependencies:
|
|
||||||
'@nlpjs/builtin-duckling': 4.26.1
|
|
||||||
'@nlpjs/builtin-microsoft': 4.26.1
|
|
||||||
'@nlpjs/core-loader': 4.26.1
|
|
||||||
'@nlpjs/emoji': 4.26.1
|
|
||||||
'@nlpjs/evaluator': 4.26.1
|
|
||||||
'@nlpjs/lang-all': 4.26.1
|
|
||||||
'@nlpjs/language': 4.25.0
|
|
||||||
'@nlpjs/neural': 4.25.0
|
|
||||||
'@nlpjs/nlg': 4.26.1
|
|
||||||
'@nlpjs/nlp': 4.27.0
|
|
||||||
'@nlpjs/nlu': 4.27.0
|
|
||||||
'@nlpjs/request': 4.25.0
|
|
||||||
'@nlpjs/sentiment': 4.26.1
|
|
||||||
'@nlpjs/similarity': 4.26.1
|
|
||||||
'@nlpjs/xtables': 4.25.0
|
|
||||||
transitivePeerDependencies:
|
|
||||||
- supports-color
|
|
||||||
|
|
||||||
node-releases@2.0.53: {}
|
node-releases@2.0.53: {}
|
||||||
|
|
||||||
node-source-walk@7.0.2:
|
node-source-walk@7.0.2:
|
||||||
|
|
@ -7751,10 +7075,6 @@ snapshots:
|
||||||
dependencies:
|
dependencies:
|
||||||
through: 2.3.8
|
through: 2.3.8
|
||||||
|
|
||||||
ssf@0.11.2:
|
|
||||||
dependencies:
|
|
||||||
frac: 1.1.2
|
|
||||||
|
|
||||||
sshpk@1.18.0:
|
sshpk@1.18.0:
|
||||||
dependencies:
|
dependencies:
|
||||||
asn1: 0.2.6
|
asn1: 0.2.6
|
||||||
|
|
@ -8079,10 +7399,6 @@ snapshots:
|
||||||
dependencies:
|
dependencies:
|
||||||
string-width: 4.2.3
|
string-width: 4.2.3
|
||||||
|
|
||||||
wmf@1.0.2: {}
|
|
||||||
|
|
||||||
word@0.3.0: {}
|
|
||||||
|
|
||||||
workerpool@6.5.1: {}
|
workerpool@6.5.1: {}
|
||||||
|
|
||||||
workerpool@9.3.4: {}
|
workerpool@9.3.4: {}
|
||||||
|
|
@ -8107,16 +7423,6 @@ snapshots:
|
||||||
|
|
||||||
wrappy@1.0.2: {}
|
wrappy@1.0.2: {}
|
||||||
|
|
||||||
xlsx@0.18.5:
|
|
||||||
dependencies:
|
|
||||||
adler-32: 1.3.1
|
|
||||||
cfb: 1.2.2
|
|
||||||
codepage: 1.15.0
|
|
||||||
crc-32: 1.2.2
|
|
||||||
ssf: 0.11.2
|
|
||||||
wmf: 1.0.2
|
|
||||||
word: 0.3.0
|
|
||||||
|
|
||||||
xmlbuilder@15.1.1: {}
|
xmlbuilder@15.1.1: {}
|
||||||
|
|
||||||
y18n@5.0.8: {}
|
y18n@5.0.8: {}
|
||||||
|
|
@ -8174,6 +7480,4 @@ snapshots:
|
||||||
toposort: 2.0.2
|
toposort: 2.0.2
|
||||||
type-fest: 2.19.0
|
type-fest: 2.19.0
|
||||||
|
|
||||||
zlibjs@0.3.1: {}
|
|
||||||
|
|
||||||
zod@3.25.76: {}
|
zod@3.25.76: {}
|
||||||
|
|
|
||||||
5
services/tech-step-intent-service/.env.example
Normal file
5
services/tech-step-intent-service/.env.example
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
# Secret partagé attendu sur le header `X-Intent-Service-Secret` de chaque
|
||||||
|
# requête (sauf `GET /health`) — doit matcher `INTENT_SERVICE_SECRET` côté
|
||||||
|
# apps/api/.env (voir apps/api/src/config/env.ts). Requis, pas de valeur par
|
||||||
|
# défaut : `Settings` (intent_service/config.py) refuse de démarrer sans.
|
||||||
|
INTENT_SERVICE_SECRET=changeme-generate-a-real-random-secret-at-least-32-chars
|
||||||
5
services/tech-step-intent-service/.gitignore
vendored
Normal file
5
services/tech-step-intent-service/.gitignore
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
.venv/
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
.pytest_cache/
|
||||||
|
.env
|
||||||
34
services/tech-step-intent-service/Dockerfile
Normal file
34
services/tech-step-intent-service/Dockerfile
Normal file
|
|
@ -0,0 +1,34 @@
|
||||||
|
# Standalone image for services/tech-step-intent-service — hors du build
|
||||||
|
# apps/api (voir services/tech-step-llm-worker/Dockerfile pour le précédent
|
||||||
|
# direct : un service Python/spaCy n'a rien à faire dans l'image Node de
|
||||||
|
# l'API, et inversement). Rien n'est persisté sur disque (pas de VOLUME,
|
||||||
|
# contrairement au worker LLM) : tout l'état (textcat/matcher entraînés)
|
||||||
|
# vit en mémoire, reconstruit à chaque `/v1/train` depuis un corpus que ce
|
||||||
|
# service ne possède pas lui-même (voir intent_service/README.md).
|
||||||
|
FROM python:3.12-slim AS base
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*
|
||||||
|
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
|
||||||
|
WORKDIR /service
|
||||||
|
|
||||||
|
FROM base AS build
|
||||||
|
# `uv.lock` est commité pour ce service (même rigueur que
|
||||||
|
# `pnpm-lock.yaml`/`--frozen-lockfile` pour apps/api et
|
||||||
|
# services/tech-step-llm-worker) — `--frozen` échoue bruyamment si
|
||||||
|
# `pyproject.toml` a dérivé du lock plutôt que de re-résoudre en silence.
|
||||||
|
# `--no-install-project` sépare l'installation des dépendances (dont les
|
||||||
|
# wheels de modèles spaCy, pinnés par URL dans pyproject.toml) de la copie
|
||||||
|
# du code applicatif, pour que le cache de layer Docker survive à un
|
||||||
|
# changement dans intent_service/ sans retélécharger ~80 Mo de modèles.
|
||||||
|
# Chemins préfixés par `services/tech-step-intent-service/` : le contexte
|
||||||
|
# de build est la racine du repo (`docker-compose.yml`'s `build.context: .`),
|
||||||
|
# même convention que `services/tech-step-llm-worker/Dockerfile`.
|
||||||
|
COPY services/tech-step-intent-service/pyproject.toml services/tech-step-intent-service/uv.lock ./
|
||||||
|
RUN uv sync --frozen --no-install-project --no-dev
|
||||||
|
COPY services/tech-step-intent-service/intent_service ./intent_service
|
||||||
|
RUN uv sync --frozen --no-dev
|
||||||
|
|
||||||
|
FROM base AS runtime
|
||||||
|
ENV PYTHONUNBUFFERED=1
|
||||||
|
COPY --from=build /service /service
|
||||||
|
EXPOSE 8000
|
||||||
|
CMD ["uv", "run", "uvicorn", "intent_service.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||||
112
services/tech-step-intent-service/README.md
Normal file
112
services/tech-step-intent-service/README.md
Normal file
|
|
@ -0,0 +1,112 @@
|
||||||
|
# tech-step-intent-service
|
||||||
|
|
||||||
|
Microservice de détection d'intention (technique de cuisine) — remplace le
|
||||||
|
pipeline `node-nlp` qui vivait dans `apps/api`
|
||||||
|
(`TechStepClassifierService`, `apps/api/src/lib/recipe-matching/tech-step-matcher.ts`) :
|
||||||
|
|
||||||
|
1. **NER par phrases** (`spacy.matcher.PhraseMatcher`) — trouve les mentions
|
||||||
|
candidates d'une technique dans un texte, à partir des `synonyms` de
|
||||||
|
chaque technique.
|
||||||
|
2. **Classification d'intention** (`textcat` spaCy, bag-of-words) — verdict
|
||||||
|
de la technique qu'une clause de texte *signifie*, entraîné sur les
|
||||||
|
`utterances` de chaque technique (y compris des paraphrases n'utilisant
|
||||||
|
jamais le mot-clé lui-même).
|
||||||
|
|
||||||
|
Basé sur **spaCy** (`fr_core_news_md`/`en_core_web_md`) plutôt que node-nlp —
|
||||||
|
écosystème NLP plus robuste/maintenu, avec l'ambition à terme (hors scope de
|
||||||
|
ce service en l'état) de pouvoir aussi absorber ce que fait aujourd'hui
|
||||||
|
`services/tech-step-llm-worker` une fois ce pipeline assez riche pour s'en
|
||||||
|
passer (les modèles `md`, avec vecteurs de mots, sont conservés dans ce but,
|
||||||
|
même si rien ici ne s'en sert encore).
|
||||||
|
|
||||||
|
## Pourquoi ce service ne possède aucune donnée d'entraînement
|
||||||
|
|
||||||
|
Contrairement à un service NLP habituel, **ce service ne connaît aucune
|
||||||
|
technique par lui-même** — `apps/api` reste l'unique source de vérité du
|
||||||
|
corpus (`TECH_STEP_TRAINING_DATA`,
|
||||||
|
`apps/api/src/lib/recipe-matching/tech-step-training-data.ts`, revu par PR
|
||||||
|
comme le reste du code). Il pousse l'intégralité du corpus ici via
|
||||||
|
`POST /v1/train` à chaque warm-up serveur (`TechStepClassifierService._train`)
|
||||||
|
— ce service (re)construit alors son pipeline en mémoire, sans jamais rien
|
||||||
|
persister sur disque. Le workflow mainteneur existant
|
||||||
|
(`apps/api/src/scripts/retrain-tech-steps.ts`, édition manuelle du corpus)
|
||||||
|
n'a pas changé.
|
||||||
|
|
||||||
|
## Pourquoi ce service vit hors du workspace pnpm
|
||||||
|
|
||||||
|
Même raisonnement que `services/tech-step-llm-worker` : un service Python
|
||||||
|
n'a rien à faire dans `pnpm-workspace.yaml` (qui ne couvre que
|
||||||
|
`apps/*`/`packages/*`), et ses dépendances (spaCy, ses modèles) ne doivent
|
||||||
|
jamais se retrouver dans l'image `apps/api`. **Aucun accès direct à
|
||||||
|
Postgres** non plus — la résolution `TechStep.key -> id` reste entièrement
|
||||||
|
côté `apps/api` (`TechStepClassifierService._train`), ce service ne
|
||||||
|
manipule que des `uid` (chaînes opaques) tout du long.
|
||||||
|
|
||||||
|
## Contrat HTTP
|
||||||
|
|
||||||
|
Voir `intent_service/schemas.py` pour le détail exact. En résumé :
|
||||||
|
|
||||||
|
- `GET /health` — sans authentification, `200` une fois les modèles spaCy
|
||||||
|
de base chargés (pas de lazy-load, voir `intent_service/main.py`).
|
||||||
|
- `POST /v1/train` — `{ locale, entries: [{ uid, synonyms, utterances }] }`
|
||||||
|
→ reconstruit le pipeline de `locale` à neuf.
|
||||||
|
- `POST /v1/process` — `{ locale, text }` → `{ entities: [{ uid, start, end }], intent, score }`.
|
||||||
|
|
||||||
|
`/v1/train` et `/v1/process` exigent le header `X-Intent-Service-Secret`
|
||||||
|
(voir `intent_service/security.py`), qui doit matcher `INTENT_SERVICE_SECRET`
|
||||||
|
côté `apps/api`.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Ce service utilise [`uv`](https://docs.astral.sh/uv/) pour ses dépendances
|
||||||
|
(`uv.lock` committé, `uv sync --frozen` partout — Dockerfile, CI, dev).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd services/tech-step-intent-service
|
||||||
|
uv sync
|
||||||
|
cp .env.example .env
|
||||||
|
# édite .env : génère un INTENT_SERVICE_SECRET, identique à celui d'apps/api
|
||||||
|
uv run uvicorn intent_service.main:app --reload --port 8000
|
||||||
|
```
|
||||||
|
|
||||||
|
`apps/api` (natif, `pnpm dev:api`, ou sa suite Mocha) doit pointer
|
||||||
|
`INTENT_SERVICE_BASE_URL=http://localhost:8000` et le même
|
||||||
|
`INTENT_SERVICE_SECRET` (voir `apps/api/.env.example`).
|
||||||
|
|
||||||
|
## Running via Docker Compose
|
||||||
|
|
||||||
|
`docker-compose.yml` (racine) définit un service `tech-step-intent-service`
|
||||||
|
aux côtés de `postgres`/`app`/`tech-step-llm-worker` — **pas optionnel**,
|
||||||
|
contrairement au worker LLM : sans lui, `apps/api` ne peut plus détecter
|
||||||
|
aucune technique de cuisine. `app` attend qu'il soit `healthy`
|
||||||
|
(`depends_on: condition: service_healthy`) avant de démarrer.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
`tests/test_locale_pipeline_entities.py` rejoue les cas d'offsets caractère
|
||||||
|
exacts et d'insensibilité accents/casse de
|
||||||
|
`apps/api/test/recipe-matching/tech-step-matcher.test.ts` — le point de
|
||||||
|
fidélité le plus critique de ce service (voir le plan de migration).
|
||||||
|
|
||||||
|
Aucun test ici ne dépend d'une vraie base Postgres ni d'`apps/api` en
|
||||||
|
service — à l'inverse, la suite Mocha d'`apps/api`
|
||||||
|
(`tech-step-matcher.test.ts`/`recipe-translation.test.ts`) exige elle une
|
||||||
|
vraie instance de ce service tournant (voir `apps/api/.env.test`), conforme
|
||||||
|
à la convention du repo de ne jamais mocker un service interne.
|
||||||
|
|
||||||
|
## Limitations connues (première version)
|
||||||
|
|
||||||
|
- **Textcat bag-of-words** (`spacy.TextCatBOW.v3`) — suffisant/rapide pour
|
||||||
|
le corpus actuel, mais n'exploite pas les vecteurs de mots des modèles
|
||||||
|
`md` chargés. Migrable vers une architecture tok2vec/similarité sans
|
||||||
|
changer le contrat HTTP, si le F1 mesuré par
|
||||||
|
`apps/api/src/scripts/calibrate-tech-step-threshold.ts` le justifie un
|
||||||
|
jour.
|
||||||
|
- **Reconstruit tout le pipeline à chaque `/v1/train`** (pas de fusion
|
||||||
|
incrémentale) — un choix délibéré (voir `LocalePipeline.train`), pas une
|
||||||
|
limitation à lever : `TECH_STEP_TRAINING_DATA` doit toujours rester
|
||||||
|
l'unique source de vérité, jamais un état local qui dérive.
|
||||||
12
services/tech-step-intent-service/intent_service/__init__.py
Normal file
12
services/tech-step-intent-service/intent_service/__init__.py
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
"""Microservice de détection d'intention (technique de cuisine).
|
||||||
|
|
||||||
|
Remplace le pipeline `node-nlp` qui vivait dans `apps/api`
|
||||||
|
(`TechStepClassifierService`, `apps/api/src/lib/recipe-matching/tech-step-matcher.ts`) :
|
||||||
|
NER par phrases (synonymes) + classification d'intention (textcat), les deux
|
||||||
|
entraînés à la demande depuis un corpus qui reste possédé par `apps/api`
|
||||||
|
(`TECH_STEP_TRAINING_DATA`) et poussé ici via `POST /v1/train`.
|
||||||
|
|
||||||
|
Ce service ne touche jamais Postgres — voir `services/tech-step-llm-worker`
|
||||||
|
pour le précédent architectural (même posture : aucun accès DB direct,
|
||||||
|
tout passe par HTTP, la résolution `TechStep.key -> id` reste côté `apps/api`).
|
||||||
|
"""
|
||||||
44
services/tech-step-intent-service/intent_service/config.py
Normal file
44
services/tech-step-intent-service/intent_service/config.py
Normal file
|
|
@ -0,0 +1,44 @@
|
||||||
|
"""Configuration du service, lue depuis l'environnement (`pydantic-settings`).
|
||||||
|
|
||||||
|
Contrairement à `requireInternalWorker` côté `apps/api`
|
||||||
|
(`apps/api/src/middlewares/require-internal-worker.ts`), qui tolère un
|
||||||
|
`INTERNAL_WORKER_SECRET` absent (le worker LLM est un job de fond
|
||||||
|
optionnel) et échoue "juste" requête par requête dans ce cas, ce service est
|
||||||
|
une dépendance coeur : `INTENT_SERVICE_SECRET` absent doit empêcher
|
||||||
|
`uvicorn` de démarrer du tout plutôt que de démarrer dans un état où chaque
|
||||||
|
requête échouerait silencieusement en boucle — `Settings` n'a donc aucune
|
||||||
|
valeur par défaut ni type optionnel pour ce champ, la validation Pydantic
|
||||||
|
lève dès l'import de ce module si la variable manque.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
# `env_file=".env"` : lu uniquement en dev natif (`cp .env.example .env`,
|
||||||
|
# voir le README de ce service) — sans effet en Docker, où
|
||||||
|
# docker-compose.yml passe les variables directement en `environment:`
|
||||||
|
# et où aucun `.env` n'est copié dans l'image. Un `.env` absent n'est pas
|
||||||
|
# une erreur ici (pydantic-settings ignore silencieusement un fichier
|
||||||
|
# manquant) ; c'est bien `intent_service_secret` ci-dessous, sans valeur
|
||||||
|
# par défaut, qui fait échouer le démarrage si la variable n'est
|
||||||
|
# disponible par aucune des deux voies.
|
||||||
|
#
|
||||||
|
# `case_sensitive` par défaut (False) : `INTENT_SERVICE_SECRET` (la
|
||||||
|
# convention majuscule utilisée partout ailleurs dans le repo, cf.
|
||||||
|
# `docker-compose.yml`/`.env.example`) matche bien le champ
|
||||||
|
# `intent_service_secret` ci-dessous.
|
||||||
|
model_config = SettingsConfigDict(env_file=".env")
|
||||||
|
|
||||||
|
# Secret partagé attendu sur le header `X-Intent-Service-Secret` de
|
||||||
|
# chaque requête (sauf `GET /health`) — voir `security.py`. Doit matcher
|
||||||
|
# `INTENT_SERVICE_SECRET` côté `apps/api/src/config/env.ts`.
|
||||||
|
intent_service_secret: str
|
||||||
|
|
||||||
|
# Pas de `port` ici : `uvicorn` prend son port en argument de ligne de
|
||||||
|
# commande (`--port`, voir le Dockerfile et le README de ce service),
|
||||||
|
# jamais lu depuis `Settings` — une variable d'env dupliquant ce que la
|
||||||
|
# commande de démarrage fixe déjà explicitement n'aurait aucun lecteur.
|
||||||
|
|
||||||
|
|
||||||
|
settings = Settings()
|
||||||
|
|
@ -0,0 +1,288 @@
|
||||||
|
"""Pipeline spaCy pour UNE locale — l'équivalent Python de ce que
|
||||||
|
`node-nlp`'s `NlpManager` faisait pour cette locale dans
|
||||||
|
`TechStepClassifierService` (`apps/api/src/lib/recipe-matching/tech-step-matcher.ts`) :
|
||||||
|
NER par entités enum (ici un `PhraseMatcher`) + classification d'intention
|
||||||
|
(ici un `textcat`), les deux entraînés à partir du même corpus
|
||||||
|
(`TECH_STEP_TRAINING_DATA`, poussé par `apps/api` via `POST /v1/train`).
|
||||||
|
|
||||||
|
Le modèle de base spaCy (tokenizer + vecteurs + le composant
|
||||||
|
`diacritics_normalizer` défini plus bas) est chargé une seule fois
|
||||||
|
(`preload()`, appelé au démarrage du process — voir `main.py` — pas
|
||||||
|
paresseusement au premier `train()`, pour que `GET /health` ne devienne
|
||||||
|
`200` qu'une fois ce coût payé) puis réutilisé à chaque `train()` : seul le
|
||||||
|
`textcat` (retiré puis rajouté à neuf) et le `PhraseMatcher` (remplacé) sont
|
||||||
|
reconstruits à chaque appel, jamais le tokenizer/les vecteurs. Rien n'est
|
||||||
|
jamais persisté sur disque — même posture que `autoSave`/`autoLoad: false`
|
||||||
|
sur l'ancien `NlpManager` : `TECH_STEP_TRAINING_DATA` (côté `apps/api`) reste
|
||||||
|
l'unique source de vérité.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import random
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
import spacy
|
||||||
|
from spacy.language import Language
|
||||||
|
from spacy.matcher import PhraseMatcher
|
||||||
|
from spacy.tokens import Doc
|
||||||
|
from spacy.training import Example
|
||||||
|
from spacy.util import minibatch
|
||||||
|
|
||||||
|
from .text_normalization import normalize_text
|
||||||
|
|
||||||
|
# Modèle spaCy de base par locale — voir pyproject.toml pour la version
|
||||||
|
# pinnée exacte. `md` (pas `sm`) : conserve les vecteurs de mots, inutilisés
|
||||||
|
# par le pipeline v1 (textcat bag-of-words) mais retenus pour l'ambition
|
||||||
|
# future de similarité sémantique (voir le README de ce service).
|
||||||
|
SUPPORTED_LOCALES = {
|
||||||
|
"fr": "fr_core_news_md",
|
||||||
|
"en": "en_core_web_md",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Composants du modèle de base non utilisés par ce pipeline (on ne s'appuie
|
||||||
|
# ni sur le NER générique de spaCy, ni sur l'analyse syntaxique/morphologique
|
||||||
|
# — seuls le tokenizer et les vecteurs de mots restent nécessaires) : les
|
||||||
|
# exclure au chargement évite le coût mémoire/CPU de composants qui ne
|
||||||
|
# tourneraient jamais.
|
||||||
|
_EXCLUDED_COMPONENTS = ["parser", "ner", "tagger", "morphologizer", "attribute_ruler", "lemmatizer"]
|
||||||
|
|
||||||
|
_TEXTCAT_PIPE_NAME = "textcat"
|
||||||
|
|
||||||
|
# Nombre d'itérations d'entraînement du textcat — calibré empiriquement pour
|
||||||
|
# converger sur un corpus de cette taille (quelques centaines d'utterances
|
||||||
|
# par locale) sans allonger inutilement le warm-up. À revoir si
|
||||||
|
# `calibrate-tech-step-threshold.ts` (côté apps/api) montre un F1 anormalement
|
||||||
|
# bas qui s'améliore avec plus d'itérations.
|
||||||
|
_TRAINING_ITERATIONS = 30
|
||||||
|
_TRAINING_BATCH_SIZE = 8
|
||||||
|
_TRAINING_DROPOUT = 0.2
|
||||||
|
# Seed fixe — un warm-up reproductible d'un redémarrage à l'autre (même
|
||||||
|
# corpus en entrée) est préférable à un score qui varie légèrement à chaque
|
||||||
|
# déploiement pour la même donnée, en particulier pendant la calibration du
|
||||||
|
# seuil de confiance côté apps/api.
|
||||||
|
_TRAINING_SEED = 0
|
||||||
|
|
||||||
|
|
||||||
|
@Language.factory("diacritics_normalizer")
|
||||||
|
def _create_diacritics_normalizer(nlp: Language, name: str) -> "_DiacriticsNormalizer":
|
||||||
|
return _DiacriticsNormalizer()
|
||||||
|
|
||||||
|
|
||||||
|
class _DiacriticsNormalizer:
|
||||||
|
"""Composant de pipeline réécrivant `token.norm_` avec `normalize_text()`
|
||||||
|
(le port Python de `normalizeText()` côté `apps/api`) pour chaque token.
|
||||||
|
|
||||||
|
Point clé : ce composant tourne aussi bien sur les `Doc` construits pour
|
||||||
|
les *patterns* du `PhraseMatcher` (voir `LocalePipeline.train`) que sur
|
||||||
|
le *texte cible* passé à `process()` — les deux passent donc par
|
||||||
|
exactement la même normalisation, ce qui garantit qu'un synonyme comme
|
||||||
|
"mijoter" matche indifféremment "MIJOTER"/"mijoté"/"Mijotée" dans le
|
||||||
|
texte, reproduisant le comportement `ner.threshold: 1` (exact après
|
||||||
|
normalisation, sans tolérance floue Levenshtein) de l'ancien `NlpManager`.
|
||||||
|
Indépendant des `entries` entraînées — ajouté une seule fois par
|
||||||
|
`preload()`, jamais retiré/rajouté par `train()`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __call__(self, doc: Doc) -> Doc:
|
||||||
|
for token in doc:
|
||||||
|
token.norm_ = normalize_text(token.text)
|
||||||
|
return doc
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TrainEntry:
|
||||||
|
"""Une technique à entraîner pour une locale — miroir de
|
||||||
|
`TrainEntryPayload` (`schemas.py`)."""
|
||||||
|
|
||||||
|
uid: str
|
||||||
|
synonyms: list[str] = field(default_factory=list)
|
||||||
|
utterances: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Entity:
|
||||||
|
"""Une mention candidate trouvée par le `PhraseMatcher` — offsets
|
||||||
|
caractère `[start, end)`, miroir de `EntityPayload` (`schemas.py`)."""
|
||||||
|
|
||||||
|
uid: str
|
||||||
|
start: int
|
||||||
|
end: int
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ProcessResult:
|
||||||
|
"""Résultat complet d'un `process()` — miroir de `ProcessResponse`
|
||||||
|
(`schemas.py`)."""
|
||||||
|
|
||||||
|
entities: list[Entity]
|
||||||
|
intent: str | None
|
||||||
|
score: float
|
||||||
|
|
||||||
|
|
||||||
|
class UnsupportedLocaleError(ValueError):
|
||||||
|
"""`locale` ne correspond à aucun modèle spaCy connu (voir
|
||||||
|
`SUPPORTED_LOCALES`) — distinct d'une locale simplement "pas encore
|
||||||
|
entraînée" (`LocalePipeline.is_trained is False`), qui n'est pas une
|
||||||
|
erreur (voir `process()`)."""
|
||||||
|
|
||||||
|
|
||||||
|
class LocalePipeline:
|
||||||
|
"""Pipeline spaCy (NER par phrases + textcat) pour une locale donnée.
|
||||||
|
Un `PipelineRegistry` (voir `pipeline_registry.py`) en détient une
|
||||||
|
instance par locale supportée.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, locale: str) -> None:
|
||||||
|
if locale not in SUPPORTED_LOCALES:
|
||||||
|
raise UnsupportedLocaleError(f"Unsupported locale: {locale!r}")
|
||||||
|
self._locale = locale
|
||||||
|
self._model_name = SUPPORTED_LOCALES[locale]
|
||||||
|
# `None` tant que `preload()` n'a pas tourné.
|
||||||
|
self._base_nlp: Language | None = None
|
||||||
|
# `None` tant qu'aucun `train()` n'a réussi — `process()` traite ça
|
||||||
|
# comme "rien à trouver" plutôt qu'une erreur, exactement le
|
||||||
|
# comportement testé côté `apps/api` pour "une locale jamais
|
||||||
|
# entraînée".
|
||||||
|
self._matcher: PhraseMatcher | None = None
|
||||||
|
self._trained = False
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_trained(self) -> bool:
|
||||||
|
return self._trained
|
||||||
|
|
||||||
|
def preload(self) -> None:
|
||||||
|
"""Charge le modèle spaCy de base (tokenizer + vecteurs) et le
|
||||||
|
composant `diacritics_normalizer` — idempotent, sans effet si déjà
|
||||||
|
chargé. Appelé au démarrage du process pour les deux locales
|
||||||
|
connues (voir `main.py`), pas paresseusement au premier `train()`.
|
||||||
|
"""
|
||||||
|
if self._base_nlp is not None:
|
||||||
|
return
|
||||||
|
nlp = spacy.load(self._model_name, exclude=_EXCLUDED_COMPONENTS)
|
||||||
|
nlp.add_pipe("diacritics_normalizer", first=True)
|
||||||
|
self._base_nlp = nlp
|
||||||
|
|
||||||
|
def train(self, entries: list[TrainEntry]) -> tuple[int, int, int]:
|
||||||
|
"""Reconstruit le `textcat` et le `PhraseMatcher` de ce pipeline à
|
||||||
|
partir de `entries` (le tokenizer/les vecteurs restent ceux chargés
|
||||||
|
par `preload()`). Retourne `(label_count, utterance_count,
|
||||||
|
synonym_count)` pour la réponse `/v1/train`.
|
||||||
|
|
||||||
|
`entries` vide retombe à `is_trained == False` plutôt que de lever —
|
||||||
|
un appelant qui n'a rien à entraîner pour cette locale obtient le
|
||||||
|
même comportement que "jamais entraîné", pas une erreur 500.
|
||||||
|
"""
|
||||||
|
self.preload()
|
||||||
|
assert self._base_nlp is not None # garanti par preload() ci-dessus
|
||||||
|
|
||||||
|
if _TEXTCAT_PIPE_NAME in self._base_nlp.pipe_names:
|
||||||
|
self._base_nlp.remove_pipe(_TEXTCAT_PIPE_NAME)
|
||||||
|
|
||||||
|
if not entries:
|
||||||
|
self._matcher = None
|
||||||
|
self._trained = False
|
||||||
|
return (0, 0, 0)
|
||||||
|
|
||||||
|
nlp = self._base_nlp
|
||||||
|
# `nlp.make_doc()` ne fait tourner *que* le tokenizer, pas les
|
||||||
|
# composants du pipeline — le `diacritics_normalizer` ajouté par
|
||||||
|
# `preload()` ne tournerait donc jamais sur les `Doc` de patterns
|
||||||
|
# s'ils n'étaient construits qu'avec `make_doc()`, alors que
|
||||||
|
# `process()` appelle `nlp(text)` (le pipeline complet) sur le texte
|
||||||
|
# cible. Sans ce correctif, un synonyme accentué comme "préchauffer"
|
||||||
|
# n'aurait jamais matché "PRÉCHAUFFER"/"Préchauffer" : trouvé en
|
||||||
|
# calibrant contre les cas exacts de `tech-step-matcher.test.ts`
|
||||||
|
# (fr, la locale la plus concernée par les accents) — un synonyme
|
||||||
|
# sans diacritique comme "faire fondre" masquait le bug en semblant
|
||||||
|
# fonctionner par coïncidence. Appliquer explicitement le même
|
||||||
|
# composant aux deux côtés garantit qu'ils passent par la même
|
||||||
|
# normalisation.
|
||||||
|
diacritics_normalizer = nlp.get_pipe("diacritics_normalizer")
|
||||||
|
|
||||||
|
matcher = PhraseMatcher(nlp.vocab, attr="NORM")
|
||||||
|
synonym_count = 0
|
||||||
|
for entry in entries:
|
||||||
|
if not entry.synonyms:
|
||||||
|
continue
|
||||||
|
patterns = [diacritics_normalizer(nlp.make_doc(synonym)) for synonym in entry.synonyms]
|
||||||
|
matcher.add(entry.uid, patterns)
|
||||||
|
synonym_count += len(entry.synonyms)
|
||||||
|
|
||||||
|
# `textcat` (exclusive_classes) exige au moins deux labels (voir
|
||||||
|
# spaCy's error E867) — jamais un problème avec le vrai corpus
|
||||||
|
# (`TECH_STEP_TRAINING_DATA` a ~27 techniques), mais un `entries` à
|
||||||
|
# un seul élément resterait structurellement valide pour le NER
|
||||||
|
# seul : ne pas planter, juste ne pas construire de textcat du tout
|
||||||
|
# (`process()` retombe alors sur `intent: null` via son garde
|
||||||
|
# `if not cats`, exactement comme "rien à classifier").
|
||||||
|
examples: list[Example] = []
|
||||||
|
if len(entries) >= 2:
|
||||||
|
textcat = nlp.add_pipe(
|
||||||
|
_TEXTCAT_PIPE_NAME,
|
||||||
|
config={
|
||||||
|
"model": {
|
||||||
|
"@architectures": "spacy.TextCatBOW.v3",
|
||||||
|
"exclusive_classes": True,
|
||||||
|
"ngram_size": 1,
|
||||||
|
"no_output_layer": False,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
for entry in entries:
|
||||||
|
textcat.add_label(entry.uid)
|
||||||
|
|
||||||
|
for entry in entries:
|
||||||
|
for utterance in entry.utterances:
|
||||||
|
doc = nlp.make_doc(utterance)
|
||||||
|
cats = {other.uid: 0.0 for other in entries}
|
||||||
|
cats[entry.uid] = 1.0
|
||||||
|
examples.append(Example.from_dict(doc, {"cats": cats}))
|
||||||
|
|
||||||
|
rng = random.Random(_TRAINING_SEED)
|
||||||
|
if examples:
|
||||||
|
optimizer = nlp.initialize(lambda: examples)
|
||||||
|
for _ in range(_TRAINING_ITERATIONS):
|
||||||
|
rng.shuffle(examples)
|
||||||
|
for batch in minibatch(examples, size=_TRAINING_BATCH_SIZE):
|
||||||
|
nlp.update(batch, sgd=optimizer, drop=_TRAINING_DROPOUT)
|
||||||
|
else:
|
||||||
|
# Des `entries` avec des `uid` mais aucune `utterance` nulle
|
||||||
|
# part (corpus incomplet) : le textcat a des labels mais rien
|
||||||
|
# pour apprendre à les distinguer — toujours initialisé pour
|
||||||
|
# rester un pipeline valide ; `process()` renverra alors un
|
||||||
|
# score ~uniforme entre labels. Ce n'est pas ce module qui doit
|
||||||
|
# juger la qualité du corpus reçu (voir `tech-step-eval-runner.ts`
|
||||||
|
# côté apps/api pour ce rôle).
|
||||||
|
nlp.initialize()
|
||||||
|
|
||||||
|
self._matcher = matcher
|
||||||
|
self._trained = True
|
||||||
|
return (len(entries), len(examples), synonym_count)
|
||||||
|
|
||||||
|
def process(self, text: str) -> ProcessResult:
|
||||||
|
"""Reproduit la forme de `NlpManager.process(locale, text)` : les
|
||||||
|
entités candidates (NER) et le verdict du classifieur d'intention
|
||||||
|
sur `text` tel quel — que ce soit la description complète ou une
|
||||||
|
clause déjà découpée côté `apps/api`, ce module ne le sait pas et ne
|
||||||
|
s'en soucie pas, exactement comme l'ancien `NlpManager`.
|
||||||
|
"""
|
||||||
|
if not self._trained or self._base_nlp is None or self._matcher is None or not text.strip():
|
||||||
|
return ProcessResult(entities=[], intent=None, score=0.0)
|
||||||
|
|
||||||
|
doc = self._base_nlp(text)
|
||||||
|
|
||||||
|
entities = [
|
||||||
|
Entity(
|
||||||
|
uid=self._base_nlp.vocab.strings[match_id],
|
||||||
|
start=doc[start].idx,
|
||||||
|
end=doc[end - 1].idx + len(doc[end - 1].text),
|
||||||
|
)
|
||||||
|
for match_id, start, end in self._matcher(doc)
|
||||||
|
]
|
||||||
|
|
||||||
|
cats = doc.cats
|
||||||
|
if not cats:
|
||||||
|
return ProcessResult(entities=entities, intent=None, score=0.0)
|
||||||
|
intent = max(cats, key=cats.get)
|
||||||
|
return ProcessResult(entities=entities, intent=intent, score=cats[intent])
|
||||||
29
services/tech-step-intent-service/intent_service/main.py
Normal file
29
services/tech-step-intent-service/intent_service/main.py
Normal file
|
|
@ -0,0 +1,29 @@
|
||||||
|
"""Point d'entrée FastAPI — `uv run uvicorn intent_service.main:app` (voir
|
||||||
|
le Dockerfile et le README de ce service).
|
||||||
|
|
||||||
|
Le chargement des modèles spaCy de base (`PipelineRegistry.preload_all`) se
|
||||||
|
fait dans le handler `lifespan` ci-dessous, *avant* qu'uvicorn n'accepte de
|
||||||
|
requêtes — `GET /health` ne répond donc `200` qu'une fois ce coût payé,
|
||||||
|
jamais pendant un chargement encore en cours (uvicorn ne sert aucune requête
|
||||||
|
tant que le `lifespan` de démarrage n'est pas terminé).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from .pipeline_registry import registry
|
||||||
|
from .routes import health, process, train
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI):
|
||||||
|
registry.preload_all()
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
app = FastAPI(title="tech-step-intent-service", lifespan=lifespan)
|
||||||
|
|
||||||
|
app.include_router(health.router)
|
||||||
|
app.include_router(train.router)
|
||||||
|
app.include_router(process.router)
|
||||||
|
|
@ -0,0 +1,48 @@
|
||||||
|
"""Détient un `LocalePipeline` par locale supportée — le seul état mutable
|
||||||
|
partagé du process (une instance vit pour toute la durée de vie d'`uvicorn`,
|
||||||
|
montée sur `app.state`, voir `main.py`).
|
||||||
|
|
||||||
|
Volontairement une classe "registre" séparée de `LocalePipeline` lui-même :
|
||||||
|
`LocalePipeline` ne connaît qu'une seule locale, ce module route
|
||||||
|
`train`/`process` vers la bonne instance selon le `locale` reçu dans la
|
||||||
|
requête — même séparation de responsabilité que `TechStepClassifierService`
|
||||||
|
(une seule instance, un seul `NlpManager` multi-langues) avait implicitement
|
||||||
|
via node-nlp, explicitée ici puisque spaCy charge un modèle par langue.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from .locale_pipeline import SUPPORTED_LOCALES, LocalePipeline, ProcessResult, TrainEntry, UnsupportedLocaleError
|
||||||
|
|
||||||
|
|
||||||
|
class PipelineRegistry:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._pipelines: dict[str, LocalePipeline] = {
|
||||||
|
locale: LocalePipeline(locale) for locale in SUPPORTED_LOCALES
|
||||||
|
}
|
||||||
|
|
||||||
|
def preload_all(self) -> None:
|
||||||
|
"""Charge le modèle spaCy de base de chaque locale connue — appelé
|
||||||
|
une fois au démarrage du process (`main.py`), pas paresseusement au
|
||||||
|
premier appel, pour que `GET /health` ne réponde `200` qu'une fois
|
||||||
|
ce coût payé (voir `LocalePipeline.preload`)."""
|
||||||
|
for pipeline in self._pipelines.values():
|
||||||
|
pipeline.preload()
|
||||||
|
|
||||||
|
def train(self, locale: str, entries: list[TrainEntry]) -> tuple[int, int, int]:
|
||||||
|
pipeline = self._pipelines.get(locale)
|
||||||
|
if pipeline is None:
|
||||||
|
raise UnsupportedLocaleError(f"Unsupported locale: {locale!r}")
|
||||||
|
return pipeline.train(entries)
|
||||||
|
|
||||||
|
def process(self, locale: str, text: str) -> ProcessResult:
|
||||||
|
pipeline = self._pipelines.get(locale)
|
||||||
|
if pipeline is None:
|
||||||
|
# Une locale que ce service ne sait structurellement pas
|
||||||
|
# charger (pas de modèle spaCy connu) se comporte comme une
|
||||||
|
# locale "jamais entraînée" côté `process` — reproduit le test
|
||||||
|
# `apps/api` existant ("returns an empty sequence for a locale
|
||||||
|
# nothing was trained on"), qui ne distingue pas les deux cas.
|
||||||
|
return ProcessResult(entities=[], intent=None, score=0.0)
|
||||||
|
return pipeline.process(text)
|
||||||
|
|
||||||
|
|
||||||
|
registry = PipelineRegistry()
|
||||||
|
|
@ -0,0 +1,18 @@
|
||||||
|
"""`GET /health` — sondé par le `healthcheck` Docker (`docker-compose.yml`)
|
||||||
|
et par l'étape CI qui attend que ce service soit prêt avant de lancer la
|
||||||
|
suite Mocha de `apps/api` (voir `.github/workflows/ci.yml`). Volontairement
|
||||||
|
sans authentification, même posture que le `GET /health` existant côté
|
||||||
|
`apps/api` (`app.ts`) — un healthcheck qui exigerait un secret compliquerait
|
||||||
|
sa configuration pour un gain de sécurité nul (il ne renvoie aucune donnée).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from ..schemas import HealthResponse
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/health", response_model=HealthResponse)
|
||||||
|
def health() -> HealthResponse:
|
||||||
|
return HealthResponse(status="ok")
|
||||||
|
|
@ -0,0 +1,23 @@
|
||||||
|
"""`POST /v1/process` — appelé par `apps/api` (`IntentServiceClient.process`)
|
||||||
|
en remplacement direct de l'ancien `NlpManager.process(locale, text)`. Voir
|
||||||
|
`LocalePipeline.process` pour la sémantique exacte (locale non entraînée ou
|
||||||
|
`text` vide -> résultat vide, jamais une erreur).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends
|
||||||
|
|
||||||
|
from ..pipeline_registry import registry
|
||||||
|
from ..schemas import EntityPayload, ProcessRequest, ProcessResponse
|
||||||
|
from ..security import require_valid_secret
|
||||||
|
|
||||||
|
router = APIRouter(dependencies=[Depends(require_valid_secret)])
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/v1/process", response_model=ProcessResponse)
|
||||||
|
def process(request: ProcessRequest) -> ProcessResponse:
|
||||||
|
result = registry.process(request.locale, request.text)
|
||||||
|
return ProcessResponse(
|
||||||
|
entities=[EntityPayload(uid=entity.uid, start=entity.start, end=entity.end) for entity in result.entities],
|
||||||
|
intent=result.intent,
|
||||||
|
score=result.score,
|
||||||
|
)
|
||||||
|
|
@ -0,0 +1,38 @@
|
||||||
|
"""`POST /v1/train` — appelé par `apps/api` (`IntentServiceClient.train`,
|
||||||
|
`TechStepClassifierService._train`) une fois par locale à chaque warm-up
|
||||||
|
serveur, avec l'intégralité de `TECH_STEP_TRAINING_DATA` filtrée pour cette
|
||||||
|
locale. Voir `LocalePipeline.train` pour ce que "reconstruit à neuf" signifie
|
||||||
|
concrètement.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
|
||||||
|
from ..locale_pipeline import TrainEntry, UnsupportedLocaleError
|
||||||
|
from ..pipeline_registry import registry
|
||||||
|
from ..schemas import TrainRequest, TrainResponse
|
||||||
|
from ..security import require_valid_secret
|
||||||
|
|
||||||
|
router = APIRouter(dependencies=[Depends(require_valid_secret)])
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/v1/train", response_model=TrainResponse)
|
||||||
|
def train(request: TrainRequest) -> TrainResponse:
|
||||||
|
entries = [
|
||||||
|
TrainEntry(uid=entry.uid, synonyms=entry.synonyms, utterances=entry.utterances)
|
||||||
|
for entry in request.entries
|
||||||
|
]
|
||||||
|
try:
|
||||||
|
label_count, utterance_count, synonym_count = registry.train(request.locale, entries)
|
||||||
|
except UnsupportedLocaleError as err:
|
||||||
|
# 422, pas 500 : une locale non supportée dans une requête de
|
||||||
|
# `apps/api` est une erreur de configuration/version-skew entre les
|
||||||
|
# deux services (voir le contrat documenté dans le plan de
|
||||||
|
# migration), pas un échec inattendu du service lui-même.
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(err)) from err
|
||||||
|
|
||||||
|
return TrainResponse(
|
||||||
|
locale=request.locale,
|
||||||
|
label_count=label_count,
|
||||||
|
utterance_count=utterance_count,
|
||||||
|
synonym_count=synonym_count,
|
||||||
|
)
|
||||||
75
services/tech-step-intent-service/intent_service/schemas.py
Normal file
75
services/tech-step-intent-service/intent_service/schemas.py
Normal file
|
|
@ -0,0 +1,75 @@
|
||||||
|
"""Modèles Pydantic du contrat HTTP — voir le plan de migration pour le
|
||||||
|
contrat exact attendu côté `apps/api` (`IntentServiceClient`,
|
||||||
|
`apps/api/src/lib/recipe-matching/intent-service-client.ts`).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
from pydantic.alias_generators import to_camel
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# POST /v1/train
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TrainEntryPayload(BaseModel):
|
||||||
|
"""Une technique — mêmes champs qu'une entrée de `TECH_STEP_TRAINING_DATA`
|
||||||
|
(`apps/api/src/lib/recipe-matching/tech-step-training-data.ts`) pour une
|
||||||
|
locale donnée."""
|
||||||
|
|
||||||
|
uid: str
|
||||||
|
synonyms: list[str] = Field(default_factory=list)
|
||||||
|
utterances: list[str] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class TrainRequest(BaseModel):
|
||||||
|
locale: str
|
||||||
|
entries: list[TrainEntryPayload]
|
||||||
|
|
||||||
|
|
||||||
|
class TrainResponse(BaseModel):
|
||||||
|
# camelCase en sortie (`labelCount`, pas `label_count`) — cohérent avec
|
||||||
|
# la convention JSON déjà en place côté `apps/api`/`packages/shared`
|
||||||
|
# (voir `TechStepAuditClauseView` etc.), même si le code Python interne
|
||||||
|
# reste en snake_case (convention PEP 8).
|
||||||
|
model_config = ConfigDict(alias_generator=to_camel, populate_by_name=True)
|
||||||
|
|
||||||
|
locale: str
|
||||||
|
label_count: int
|
||||||
|
utterance_count: int
|
||||||
|
synonym_count: int
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# POST /v1/process
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class ProcessRequest(BaseModel):
|
||||||
|
locale: str
|
||||||
|
text: str
|
||||||
|
|
||||||
|
|
||||||
|
class EntityPayload(BaseModel):
|
||||||
|
"""Une mention candidate d'une technique — offsets caractère `[start, end)`
|
||||||
|
dans `text`, convention identique à `String.prototype.slice` côté
|
||||||
|
`apps/api` (pas de décalage `+1` à appliquer côté Node, contrairement à
|
||||||
|
l'ancien `NlpManager` de node-nlp)."""
|
||||||
|
|
||||||
|
uid: str
|
||||||
|
start: int
|
||||||
|
end: int
|
||||||
|
|
||||||
|
|
||||||
|
class ProcessResponse(BaseModel):
|
||||||
|
entities: list[EntityPayload]
|
||||||
|
intent: str | None
|
||||||
|
score: float
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# GET /health
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class HealthResponse(BaseModel):
|
||||||
|
status: str
|
||||||
35
services/tech-step-intent-service/intent_service/security.py
Normal file
35
services/tech-step-intent-service/intent_service/security.py
Normal file
|
|
@ -0,0 +1,35 @@
|
||||||
|
"""Authentification des appels entrants — miroir inversé de `requireInternalWorker`
|
||||||
|
(`apps/api/src/middlewares/require-internal-worker.ts`) : ici c'est
|
||||||
|
`apps/api` qui appelle *ce* service, donc c'est ce service qui vérifie le
|
||||||
|
secret plutôt que de l'envoyer.
|
||||||
|
|
||||||
|
Comparaison à temps constant (`hmac.compare_digest`, l'équivalent Python du
|
||||||
|
`timingSafeEqual` de Node utilisé côté `apps/api`) — même raisonnement :
|
||||||
|
un attaquant ne doit rien apprendre de la durée de la comparaison au-delà de
|
||||||
|
ce qu'une différence de longueur révèle déjà.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import hmac
|
||||||
|
|
||||||
|
from fastapi import Header, HTTPException, status
|
||||||
|
|
||||||
|
from .config import settings
|
||||||
|
|
||||||
|
_SECRET_HEADER_NAME = "x-intent-service-secret"
|
||||||
|
|
||||||
|
|
||||||
|
def require_valid_secret(
|
||||||
|
x_intent_service_secret: str | None = Header(default=None, alias=_SECRET_HEADER_NAME),
|
||||||
|
) -> None:
|
||||||
|
"""Dépendance FastAPI montée sur chaque route protégée (`/v1/*`) — pas
|
||||||
|
`GET /health`, sondé par le healthcheck Docker sans configuration
|
||||||
|
d'auth propre.
|
||||||
|
|
||||||
|
`settings.intent_service_secret` est garanti non vide par `config.py`
|
||||||
|
(pas de valeur par défaut dans `Settings`) — le seul cas à traiter ici
|
||||||
|
est un header manquant ou incorrect côté appelant.
|
||||||
|
"""
|
||||||
|
if x_intent_service_secret is None or not hmac.compare_digest(
|
||||||
|
x_intent_service_secret, settings.intent_service_secret
|
||||||
|
):
|
||||||
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
|
||||||
|
|
@ -0,0 +1,32 @@
|
||||||
|
"""Port Python de `normalizeText` (`apps/api/src/lib/recipe-matching/tech-step-matcher.ts`).
|
||||||
|
|
||||||
|
Doit rester bit-pour-bit équivalent à sa contrepartie TypeScript — c'est ce
|
||||||
|
qui garantit qu'un synonyme matché ici tombe exactement sur les mêmes
|
||||||
|
positions caractère que ce que `apps/api` attendait de node-nlp (voir
|
||||||
|
`LocalePipeline`'s `diacritics_normalizer`, qui applique cette fonction aux
|
||||||
|
patterns *et* au texte cible pour les faire matcher identiquement).
|
||||||
|
|
||||||
|
TypeScript original :
|
||||||
|
|
||||||
|
const COMBINING_DIACRITICS_PATTERN = /\\p{Diacritic}/gu;
|
||||||
|
export function normalizeText(text: string): string {
|
||||||
|
return text.normalize("NFD").replace(COMBINING_DIACRITICS_PATTERN, "").toLowerCase();
|
||||||
|
}
|
||||||
|
|
||||||
|
`unicodedata.combining(ch) != 0` (catégories Unicode Mn/Mc, la classe de
|
||||||
|
combinaison canonique) est l'idiome Python standard pour "strip accents
|
||||||
|
after NFD" — légèrement plus étroit que `\\p{Diacritic}` en théorie (qui
|
||||||
|
couvre aussi quelques diacritiques autonomes hors caractères combinants),
|
||||||
|
mais strictement équivalent pour tout caractère latin accentué usuel
|
||||||
|
(français/anglais) une fois décomposé en NFD, ce qui est le seul cas
|
||||||
|
réellement exercé par ce corpus.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import unicodedata
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_text(text: str) -> str:
|
||||||
|
"""Décompose en NFD, retire les marques combinantes (accents), met en minuscule."""
|
||||||
|
decomposed = unicodedata.normalize("NFD", text)
|
||||||
|
stripped = "".join(char for char in decomposed if not unicodedata.combining(char))
|
||||||
|
return stripped.lower()
|
||||||
53
services/tech-step-intent-service/pyproject.toml
Normal file
53
services/tech-step-intent-service/pyproject.toml
Normal file
|
|
@ -0,0 +1,53 @@
|
||||||
|
[project]
|
||||||
|
name = "tech-step-intent-service"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "Microservice de détection d'intention (technique de cuisine) — remplace node-nlp côté apps/api."
|
||||||
|
requires-python = ">=3.12"
|
||||||
|
dependencies = [
|
||||||
|
"fastapi>=0.115,<0.116",
|
||||||
|
"uvicorn[standard]>=0.32,<0.33",
|
||||||
|
"spacy>=3.8,<3.9",
|
||||||
|
# Fournit les tables de lookup ("lexeme_norm" notamment) que
|
||||||
|
# `nlp.initialize()` réclame pour l'anglais lors de l'entraînement du
|
||||||
|
# textcat (`en_core_web_md` ne les embarque pas lui-même, contrairement à
|
||||||
|
# `fr_core_news_md`) — sans ce paquet, entraîner un pipeline "en" lève
|
||||||
|
# `E955`.
|
||||||
|
"spacy-lookups-data>=1.0,<1.1",
|
||||||
|
"pydantic-settings>=2.6,<3",
|
||||||
|
# Modèles spaCy installés comme des dépendances pip normales, pinnées par
|
||||||
|
# URL de release GitHub (pas via `python -m spacy download`, qui résout
|
||||||
|
# "la dernière version compatible" et n'est pas verrouillable par
|
||||||
|
# `uv.lock`). `uv sync --frozen` installe donc déjà les modèles — aucune
|
||||||
|
# étape `spacy download` séparée, ni au Dockerfile ni en CI. Version
|
||||||
|
# 3.8.0 choisie pour matcher la ligne spaCy 3.8 pinnée ci-dessus (voir
|
||||||
|
# https://github.com/explosion/spacy-models/releases).
|
||||||
|
"fr_core_news_md @ https://github.com/explosion/spacy-models/releases/download/fr_core_news_md-3.8.0/fr_core_news_md-3.8.0-py3-none-any.whl",
|
||||||
|
"en_core_web_md @ https://github.com/explosion/spacy-models/releases/download/en_core_web_md-3.8.0/en_core_web_md-3.8.0-py3-none-any.whl",
|
||||||
|
]
|
||||||
|
|
||||||
|
[dependency-groups]
|
||||||
|
dev = [
|
||||||
|
"pytest>=8,<9",
|
||||||
|
# Requis par fastapi.testclient.TestClient (httpx en interne depuis FastAPI 0.110+).
|
||||||
|
"httpx>=0.27,<0.28",
|
||||||
|
]
|
||||||
|
|
||||||
|
[tool.uv]
|
||||||
|
# Les deux modèles ci-dessus sont publiés comme des builds "any" universels
|
||||||
|
# (pas de wheel spécifique par plateforme) — rien à déclarer de plus ici,
|
||||||
|
# contrairement à un paquet avec des extras natifs par OS/arch.
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["hatchling"]
|
||||||
|
build-backend = "hatchling.build"
|
||||||
|
|
||||||
|
[tool.hatch.build.targets.wheel]
|
||||||
|
packages = ["intent_service"]
|
||||||
|
|
||||||
|
[tool.hatch.metadata]
|
||||||
|
# Requis par hatchling pour accepter des dépendances pinnées par URL directe
|
||||||
|
# (les wheels de modèles spaCy ci-dessus) plutôt qu'un nom+version résolu
|
||||||
|
# depuis un index PyPI — voir la note sur `pyproject.toml` dans le plan de
|
||||||
|
# migration pour pourquoi ces modèles sont déclarés ainsi plutôt que via
|
||||||
|
# `python -m spacy download`.
|
||||||
|
allow-direct-references = true
|
||||||
10
services/tech-step-intent-service/tests/conftest.py
Normal file
10
services/tech-step-intent-service/tests/conftest.py
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
"""`Settings` (`intent_service/config.py`) lève dès l'import si
|
||||||
|
`INTENT_SERVICE_SECRET` est absent — cette variable doit donc être définie
|
||||||
|
avant le tout premier `import intent_service...` de la session pytest.
|
||||||
|
`conftest.py` est chargé par pytest avant la collecte des modules de test,
|
||||||
|
donc avant que `test_routes_*.py`/`test_security.py` n'importent
|
||||||
|
`intent_service.main`."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
os.environ.setdefault("INTENT_SERVICE_SECRET", "pytest-only-secret-not-used-anywhere-else-32ch")
|
||||||
|
|
@ -0,0 +1,126 @@
|
||||||
|
"""Rejoue les cas d'offsets caractère exacts et d'insensibilité accents/casse
|
||||||
|
de `tech-step-matcher.test.ts` (`apps/api/test/recipe-matching/tech-step-matcher.test.ts`)
|
||||||
|
contre le `PhraseMatcher`/`diacritics_normalizer` de `LocalePipeline` — le
|
||||||
|
point de fidélité le plus critique de cette migration (voir le plan). Doit
|
||||||
|
être vert *avant* de brancher `apps/api` dessus.
|
||||||
|
|
||||||
|
Ces tests entraînent un pipeline minimal (pas le corpus complet
|
||||||
|
`TECH_STEP_TRAINING_DATA`, propriété de `apps/api`) avec juste assez de
|
||||||
|
`synonyms`/`utterances` pour reproduire chaque cas — le textcat n'est pas ce
|
||||||
|
qui est vérifié ici (voir `test_locale_pipeline_intent.py`).
|
||||||
|
"""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from intent_service.locale_pipeline import LocalePipeline, TrainEntry
|
||||||
|
|
||||||
|
# Un jeu d'entrées minimal mais réaliste, reprenant les synonymes réels de
|
||||||
|
# `tech-step-training-data.ts` pour "preheat"/"melt" qui rendent les cas
|
||||||
|
# `tech-step-matcher.test.ts` exacts (voir ce fichier, lignes 155/787).
|
||||||
|
_FR_ENTRIES = [
|
||||||
|
TrainEntry(
|
||||||
|
uid="preheat",
|
||||||
|
synonyms=["préchauffer", "poêle chaude"],
|
||||||
|
utterances=["préchauffer le four à 180 degrés", "mettre la poêle sur feu vif"],
|
||||||
|
),
|
||||||
|
TrainEntry(
|
||||||
|
uid="melt",
|
||||||
|
synonyms=["faire fondre", "faire chauffer"],
|
||||||
|
utterances=["faire fondre le beurre", "faire chauffer une noix de beurre"],
|
||||||
|
),
|
||||||
|
TrainEntry(
|
||||||
|
uid="simmer",
|
||||||
|
synonyms=["mijoter"],
|
||||||
|
utterances=["faire mijoter à feu doux", "laisser mijoter à couvert"],
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def fr_pipeline() -> LocalePipeline:
|
||||||
|
pipeline = LocalePipeline("fr")
|
||||||
|
pipeline.train(_FR_ENTRIES)
|
||||||
|
return pipeline
|
||||||
|
|
||||||
|
|
||||||
|
def test_matches_an_exact_expression(fr_pipeline: LocalePipeline):
|
||||||
|
result = fr_pipeline.process("Faire mijoter à feu doux")
|
||||||
|
assert [entity.uid for entity in result.entities] == ["simmer"]
|
||||||
|
entity = result.entities[0]
|
||||||
|
text = "Faire mijoter à feu doux"
|
||||||
|
assert text[entity.start : entity.end].lower() == "mijoter"
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_case_and_accent_insensitive(fr_pipeline: LocalePipeline):
|
||||||
|
result = fr_pipeline.process("FAIRE MIJOTER")
|
||||||
|
assert [entity.uid for entity in result.entities] == ["simmer"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_returns_no_entities_when_nothing_matches(fr_pipeline: LocalePipeline):
|
||||||
|
result = fr_pipeline.process("Ranger les couverts dans le tiroir")
|
||||||
|
assert result.entities == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_returns_empty_for_an_empty_text(fr_pipeline: LocalePipeline):
|
||||||
|
result = fr_pipeline.process("")
|
||||||
|
assert result.entities == []
|
||||||
|
assert result.intent is None
|
||||||
|
assert result.score == 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_untrained_locale_returns_empty_without_error():
|
||||||
|
pipeline = LocalePipeline("en")
|
||||||
|
result = pipeline.process("melt the butter")
|
||||||
|
assert result.entities == []
|
||||||
|
assert result.intent is None
|
||||||
|
assert result.score == 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_detects_two_techniques_with_exact_tight_spans_reading_order(fr_pipeline: LocalePipeline):
|
||||||
|
text = "Préchauffer la poêle, puis faire fondre le beurre"
|
||||||
|
result = fr_pipeline.process(text)
|
||||||
|
|
||||||
|
uids_by_start = sorted(((entity.start, entity.uid) for entity in result.entities))
|
||||||
|
assert [uid for _, uid in uids_by_start] == ["preheat", "melt"]
|
||||||
|
|
||||||
|
preheat_entity = next(e for e in result.entities if e.uid == "preheat")
|
||||||
|
melt_entity = next(e for e in result.entities if e.uid == "melt")
|
||||||
|
assert text[preheat_entity.start : preheat_entity.end].lower() == "préchauffer"
|
||||||
|
assert text[melt_entity.start : melt_entity.end].lower() == "faire fondre"
|
||||||
|
|
||||||
|
|
||||||
|
def test_matches_the_classic_poele_chaude_example_with_exact_offsets(fr_pipeline: LocalePipeline):
|
||||||
|
# Le cas motivant les context spans côté apps/api (tech-step-matcher.test.ts) :
|
||||||
|
# le mot-clé de `preheat` est un groupe nominal ("poêle chaude"), pas un
|
||||||
|
# verbe. Offsets attendus IDENTIQUES à ceux du test TS d'origine :
|
||||||
|
# preheat -> [9, 21) ("poêle chaude"), melt -> [23, 37) ("faire chauffer").
|
||||||
|
text = "Dans une poêle chaude, faire chauffer une noix de beurre"
|
||||||
|
result = fr_pipeline.process(text)
|
||||||
|
|
||||||
|
preheat_entity = next(e for e in result.entities if e.uid == "preheat")
|
||||||
|
melt_entity = next(e for e in result.entities if e.uid == "melt")
|
||||||
|
|
||||||
|
assert (preheat_entity.start, preheat_entity.end) == (9, 21)
|
||||||
|
assert text[preheat_entity.start : preheat_entity.end] == "poêle chaude"
|
||||||
|
|
||||||
|
assert (melt_entity.start, melt_entity.end) == (23, 37)
|
||||||
|
assert text[melt_entity.start : melt_entity.end] == "faire chauffer"
|
||||||
|
|
||||||
|
|
||||||
|
def test_chop_matches_english_text_tight_span():
|
||||||
|
pipeline = LocalePipeline("en")
|
||||||
|
pipeline.train(
|
||||||
|
[
|
||||||
|
TrainEntry(
|
||||||
|
uid="chop",
|
||||||
|
synonyms=["chop"],
|
||||||
|
utterances=["chop the onions finely", "finely chop the garlic"],
|
||||||
|
),
|
||||||
|
TrainEntry(uid="boil", synonyms=["boil"], utterances=["bring to the boil", "boil the water"]),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
text = "Chop the onions finely"
|
||||||
|
result = pipeline.process(text)
|
||||||
|
chop_entity = next(e for e in result.entities if e.uid == "chop")
|
||||||
|
assert (chop_entity.start, chop_entity.end) == (0, 4)
|
||||||
|
assert text[chop_entity.start : chop_entity.end] == "Chop"
|
||||||
|
|
@ -0,0 +1,76 @@
|
||||||
|
"""Vérifie le round-trip entraînement -> prédiction du `textcat` (la partie
|
||||||
|
"comprendre le sens, pas juste les mots clés" du pipeline — voir le
|
||||||
|
commentaire de `tech-step-matcher.ts` côté apps/api pour la motivation
|
||||||
|
d'origine)."""
|
||||||
|
|
||||||
|
from intent_service.locale_pipeline import LocalePipeline, TrainEntry
|
||||||
|
|
||||||
|
_ENTRIES = [
|
||||||
|
TrainEntry(
|
||||||
|
uid="melt",
|
||||||
|
synonyms=["faire fondre"],
|
||||||
|
utterances=[
|
||||||
|
"faire fondre le beurre à feu doux",
|
||||||
|
"laisser fondre le beurre dans la poêle",
|
||||||
|
"jusqu'à ce que le beurre ait disparu",
|
||||||
|
"jusqu'à ce que le beurre ait complètement disparu dans la poêle",
|
||||||
|
],
|
||||||
|
),
|
||||||
|
TrainEntry(
|
||||||
|
uid="boil",
|
||||||
|
synonyms=["bouillir"],
|
||||||
|
utterances=[
|
||||||
|
"porter l'eau à ébullition",
|
||||||
|
"faire bouillir l'eau salée",
|
||||||
|
"laisser bouillir quelques minutes",
|
||||||
|
"porter à ébullition puis baisser le feu",
|
||||||
|
],
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_train_returns_label_utterance_and_synonym_counts():
|
||||||
|
pipeline = LocalePipeline("fr")
|
||||||
|
label_count, utterance_count, synonym_count = pipeline.train(_ENTRIES)
|
||||||
|
assert label_count == 2
|
||||||
|
assert utterance_count == sum(len(entry.utterances) for entry in _ENTRIES)
|
||||||
|
assert synonym_count == sum(len(entry.synonyms) for entry in _ENTRIES)
|
||||||
|
assert pipeline.is_trained is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_classifies_a_paraphrase_never_using_the_techniques_own_verb():
|
||||||
|
# Le cas motivant tout le pipeline (voir tech-step-matcher.ts) : aucune
|
||||||
|
# forme de "fondre" dans cette phrase, mais elle ne peut raisonnablement
|
||||||
|
# signifier que `melt` une fois le textcat entraîné sur les paraphrases
|
||||||
|
# ci-dessus.
|
||||||
|
pipeline = LocalePipeline("fr")
|
||||||
|
pipeline.train(_ENTRIES)
|
||||||
|
|
||||||
|
result = pipeline.process("jusqu'à ce que le beurre ait disparu dans la poêle")
|
||||||
|
assert result.intent == "melt"
|
||||||
|
assert result.score > 0.5
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_entries_leaves_the_pipeline_untrained():
|
||||||
|
pipeline = LocalePipeline("fr")
|
||||||
|
pipeline.train([])
|
||||||
|
assert pipeline.is_trained is False
|
||||||
|
result = pipeline.process("faire fondre le beurre")
|
||||||
|
assert result.intent is None
|
||||||
|
assert result.entities == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_retraining_replaces_the_previous_textcat_rather_than_accumulating():
|
||||||
|
# `textcat` (exclusive_classes) exige >= 2 labels (voir la note dans
|
||||||
|
# LocalePipeline.train) — le second entraînement garde donc 2 entrées,
|
||||||
|
# mais remplace "boil" par une technique différente ("chop"), pour
|
||||||
|
# vérifier que "boil" ne peut plus jamais ressortir après coup (pas de
|
||||||
|
# fusion incrémentale — voir la doc de `LocalePipeline.train`).
|
||||||
|
pipeline = LocalePipeline("fr")
|
||||||
|
pipeline.train(_ENTRIES)
|
||||||
|
|
||||||
|
chop_entry = TrainEntry(uid="chop", synonyms=["couper"], utterances=["couper les légumes en dés"])
|
||||||
|
pipeline.train([_ENTRIES[0], chop_entry])
|
||||||
|
|
||||||
|
result = pipeline.process("porter l'eau à ébullition")
|
||||||
|
assert result.intent != "boil"
|
||||||
|
|
@ -0,0 +1,71 @@
|
||||||
|
"""Contrat JSON de `POST /v1/process` — voir `schemas.py`/`routes/process.py`."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from intent_service.config import settings
|
||||||
|
from intent_service.main import app
|
||||||
|
|
||||||
|
_HEADERS = {"X-Intent-Service-Secret": settings.intent_service_secret}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client():
|
||||||
|
with TestClient(app) as test_client:
|
||||||
|
yield test_client
|
||||||
|
|
||||||
|
|
||||||
|
def test_process_against_an_untrained_locale_returns_empty_result(client: TestClient):
|
||||||
|
response = client.post(
|
||||||
|
"/v1/process", headers=_HEADERS, json={"locale": "fr", "text": "faire mijoter à feu doux"}
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json() == {"entities": [], "intent": None, "score": 0.0}
|
||||||
|
|
||||||
|
|
||||||
|
def test_process_after_train_returns_entities_and_intent(client: TestClient):
|
||||||
|
client.post(
|
||||||
|
"/v1/train",
|
||||||
|
headers=_HEADERS,
|
||||||
|
json={
|
||||||
|
"locale": "fr",
|
||||||
|
"entries": [
|
||||||
|
# `textcat` (exclusive_classes) exige >= 2 labels (voir
|
||||||
|
# LocalePipeline.train) — un second label est nécessaire
|
||||||
|
# même si ce test ne vérifie que celui de "simmer".
|
||||||
|
{
|
||||||
|
"uid": "simmer",
|
||||||
|
"synonyms": ["mijoter"],
|
||||||
|
"utterances": ["faire mijoter à feu doux", "laisser mijoter à couvert"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"uid": "boil",
|
||||||
|
"synonyms": ["bouillir"],
|
||||||
|
"utterances": ["faire bouillir l'eau", "porter à ébullition"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/v1/process", headers=_HEADERS, json={"locale": "fr", "text": "Faire mijoter à feu doux"}
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
body = response.json()
|
||||||
|
assert body["intent"] == "simmer"
|
||||||
|
assert body["score"] > 0
|
||||||
|
assert [entity["uid"] for entity in body["entities"]] == ["simmer"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_process_with_blank_text_returns_empty_result(client: TestClient):
|
||||||
|
client.post(
|
||||||
|
"/v1/train",
|
||||||
|
headers=_HEADERS,
|
||||||
|
json={
|
||||||
|
"locale": "en",
|
||||||
|
"entries": [{"uid": "boil", "synonyms": ["boil"], "utterances": ["bring to the boil"]}],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
response = client.post("/v1/process", headers=_HEADERS, json={"locale": "en", "text": " "})
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json() == {"entities": [], "intent": None, "score": 0.0}
|
||||||
47
services/tech-step-intent-service/tests/test_routes_train.py
Normal file
47
services/tech-step-intent-service/tests/test_routes_train.py
Normal file
|
|
@ -0,0 +1,47 @@
|
||||||
|
"""Contrat JSON de `POST /v1/train` — voir `schemas.py`/`routes/train.py`."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from intent_service.config import settings
|
||||||
|
from intent_service.main import app
|
||||||
|
|
||||||
|
_HEADERS = {"X-Intent-Service-Secret": settings.intent_service_secret}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client():
|
||||||
|
with TestClient(app) as test_client:
|
||||||
|
yield test_client
|
||||||
|
|
||||||
|
|
||||||
|
def test_train_returns_counts(client: TestClient):
|
||||||
|
response = client.post(
|
||||||
|
"/v1/train",
|
||||||
|
headers=_HEADERS,
|
||||||
|
json={
|
||||||
|
"locale": "fr",
|
||||||
|
"entries": [
|
||||||
|
{"uid": "melt", "synonyms": ["faire fondre"], "utterances": ["faire fondre le beurre"]},
|
||||||
|
{"uid": "boil", "synonyms": ["bouillir"], "utterances": ["faire bouillir l'eau"]},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
body = response.json()
|
||||||
|
assert body == {"locale": "fr", "labelCount": 2, "utteranceCount": 2, "synonymCount": 2}
|
||||||
|
|
||||||
|
|
||||||
|
def test_train_with_unsupported_locale_returns_422(client: TestClient):
|
||||||
|
response = client.post(
|
||||||
|
"/v1/train",
|
||||||
|
headers=_HEADERS,
|
||||||
|
json={"locale": "de", "entries": [{"uid": "melt", "synonyms": [], "utterances": []}]},
|
||||||
|
)
|
||||||
|
assert response.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_train_with_empty_entries_returns_zero_counts(client: TestClient):
|
||||||
|
response = client.post("/v1/train", headers=_HEADERS, json={"locale": "en", "entries": []})
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json() == {"locale": "en", "labelCount": 0, "utteranceCount": 0, "synonymCount": 0}
|
||||||
42
services/tech-step-intent-service/tests/test_security.py
Normal file
42
services/tech-step-intent-service/tests/test_security.py
Normal file
|
|
@ -0,0 +1,42 @@
|
||||||
|
"""`require_valid_secret` — miroir inversé de
|
||||||
|
`require-internal-worker.test.ts` côté `apps/api`."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from intent_service.config import settings
|
||||||
|
from intent_service.main import app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client():
|
||||||
|
with TestClient(app) as test_client:
|
||||||
|
yield test_client
|
||||||
|
|
||||||
|
|
||||||
|
def test_rejects_a_missing_secret(client: TestClient):
|
||||||
|
response = client.post("/v1/process", json={"locale": "fr", "text": "faire fondre"})
|
||||||
|
assert response.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
def test_rejects_a_wrong_secret(client: TestClient):
|
||||||
|
response = client.post(
|
||||||
|
"/v1/process",
|
||||||
|
json={"locale": "fr", "text": "faire fondre"},
|
||||||
|
headers={"X-Intent-Service-Secret": "not-the-right-secret"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
def test_accepts_the_configured_secret(client: TestClient):
|
||||||
|
response = client.post(
|
||||||
|
"/v1/process",
|
||||||
|
json={"locale": "fr", "text": "faire fondre"},
|
||||||
|
headers={"X-Intent-Service-Secret": settings.intent_service_secret},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_requires_no_secret(client: TestClient):
|
||||||
|
response = client.get("/health")
|
||||||
|
assert response.status_code == 200
|
||||||
|
|
@ -0,0 +1,21 @@
|
||||||
|
"""Réplique les cas de `normalizeText` de `tech-step-matcher.test.ts`
|
||||||
|
(`apps/api/test/recipe-matching/tech-step-matcher.test.ts`) contre le port
|
||||||
|
Python — les deux fonctions doivent rester bit-pour-bit équivalentes."""
|
||||||
|
|
||||||
|
from intent_service.text_normalization import normalize_text
|
||||||
|
|
||||||
|
|
||||||
|
def test_lowercases_and_strips_accents():
|
||||||
|
assert normalize_text("Déglacer AU FOUR") == "deglacer au four"
|
||||||
|
|
||||||
|
|
||||||
|
def test_strips_a_variety_of_diacritics_including_cedilla():
|
||||||
|
assert normalize_text("Façon Œuf à l'Étouffée") == "facon œuf a l'etouffee"
|
||||||
|
|
||||||
|
|
||||||
|
def test_leaves_already_plain_text_unchanged_aside_from_casing():
|
||||||
|
assert normalize_text("Mix everything") == "mix everything"
|
||||||
|
|
||||||
|
|
||||||
|
def test_returns_an_empty_string_for_an_empty_input():
|
||||||
|
assert normalize_text("") == ""
|
||||||
1560
services/tech-step-intent-service/uv.lock
Normal file
1560
services/tech-step-intent-service/uv.lock
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -473,49 +473,58 @@ sens est sans ambiguïté. `TechStepMapping` a été supprimée (migration
|
||||||
interrogée/éditée à l'exécution, les données de matching vivent en code
|
interrogée/éditée à l'exécution, les données de matching vivent en code
|
||||||
(`tech-step-training-data.ts`).
|
(`tech-step-training-data.ts`).
|
||||||
|
|
||||||
|
`node-nlp` a ensuite été remplacé à son tour par un microservice Python dédié,
|
||||||
|
`services/tech-step-intent-service` (spaCy — `PhraseMatcher` + `textcat`),
|
||||||
|
appelé en HTTP par `TechStepClassifierService` via `IntentServiceClient`
|
||||||
|
(`intent-service-client.ts`) — `node-nlp` était peu maintenu et tournait
|
||||||
|
in-process dans l'event loop Node ; spaCy offre un écosystème NLP plus
|
||||||
|
robuste, dans un processus séparé, avec l'ambition à terme de pouvoir aussi
|
||||||
|
absorber ce que fait `services/tech-step-llm-worker`. `TECH_STEP_TRAINING_DATA`
|
||||||
|
reste possédé par `apps/api` (revu par PR comme le reste du code) et poussé
|
||||||
|
intégralement à ce service via `POST /v1/train` à chaque warm-up — ce service
|
||||||
|
ne touche jamais Postgres lui-même (voir son propre README).
|
||||||
|
|
||||||
`normalizeText` (décomposition NFD + suppression des diacritiques + minuscule)
|
`normalizeText` (décomposition NFD + suppression des diacritiques + minuscule)
|
||||||
reste utilisée par `ingredient-matcher.ts`, mais n'intervient plus dans la
|
reste utilisée par `ingredient-matcher.ts`, mais n'intervient plus dans la
|
||||||
détection des techniques elle-même — node-nlp gère sa propre normalisation
|
détection des techniques elle-même — un port Python de cette même fonction
|
||||||
par langue.
|
(`intent_service/text_normalization.py`) alimente le composant de
|
||||||
|
normalisation du pipeline spaCy côté service.
|
||||||
|
|
||||||
**Pipeline en 3 étapes** (`TechStepClassifierService.matchTechStepSpans`) :
|
**Pipeline en 3 étapes** (`TechStepClassifierService.matchTechStepSpans`) :
|
||||||
1. **NER** (entités enum node-nlp, `synonyms` de `TECH_STEP_TRAINING_DATA`)
|
1. **NER** (le `PhraseMatcher` du service, construit depuis les `synonyms` de
|
||||||
trouve chaque mention *candidate* d'une technique dans la description
|
`TECH_STEP_TRAINING_DATA`) trouve chaque mention *candidate* d'une
|
||||||
entière, avec sa position exacte — équivalent mécanique des anciennes
|
technique dans la description entière, avec sa position exacte —
|
||||||
regex, en listes de synonymes plutôt qu'en patterns écrits à la main.
|
équivalent mécanique des anciennes regex, en listes de synonymes plutôt
|
||||||
`ner.threshold: 1` (exact après normalisation, pas de tolérance floue
|
qu'en patterns écrits à la main. Le matching se fait sur une normalisation
|
||||||
Levenshtein) — le défaut à 0.8 faisait matcher "faire" (verbe auxiliaire
|
stricte (accents/casse) sans tolérance floue de type Levenshtein — voir
|
||||||
omniprésent en français) contre le synonyme "frire" de `fry` par pure
|
`services/tech-step-intent-service/intent_service/locale_pipeline.py`.
|
||||||
proximité de chaîne, un faux positif détecté en calibrant contre le
|
|
||||||
corpus réel.
|
|
||||||
2. La description est découpée en clauses autour de ces candidats
|
2. La description est découpée en clauses autour de ces candidats
|
||||||
(`splitIntoClauses`, pure/testable sans modèle) — une étape nommant deux
|
(`splitIntoClauses`, pure/testable sans modèle) — une étape nommant deux
|
||||||
techniques a besoin que chacune soit jugée sur son propre contexte, pas
|
techniques a besoin que chacune soit jugée sur son propre contexte, pas
|
||||||
la phrase entière classée d'un bloc.
|
la phrase entière classée d'un bloc.
|
||||||
3. **Classification d'intention NLP** (le même `NlpManager`, entraîné sur les
|
3. **Classification d'intention NLP** (le `textcat` du service, entraîné sur
|
||||||
`utterances` de `TECH_STEP_TRAINING_DATA`) classe chaque clause
|
les `utterances` de `TECH_STEP_TRAINING_DATA`) classe chaque clause
|
||||||
individuellement — c'est ce qui apporte la compréhension du **sens** :
|
individuellement — c'est ce qui apporte la compréhension du **sens** :
|
||||||
le corpus d'entraînement mélange volontairement des tournures ancrées sur
|
le corpus d'entraînement mélange volontairement des tournures ancrées sur
|
||||||
le mot-clé et des paraphrases qui ne l'emploient jamais (ex. "jusqu'à ce
|
le mot-clé et des paraphrases qui ne l'emploient jamais (ex. "jusqu'à ce
|
||||||
que le beurre ait disparu" pour `melt`), donc le verdict final d'une
|
que le beurre ait disparu" pour `melt`), donc le verdict final d'une
|
||||||
clause vient de ce que le modèle reconnaît comme *signifiant* la
|
clause vient de ce que le modèle reconnaît comme *signifiant* la
|
||||||
technique, pas du mot littéral qui a déclenché son découpage. En dessous
|
technique, pas du mot littéral qui a déclenché son découpage. En dessous
|
||||||
de `CONFIDENCE_THRESHOLD` (0.65 — ajusté empiriquement contre le corpus
|
de `CONFIDENCE_THRESHOLD` (voir la constante dans `tech-step-matcher.ts`
|
||||||
réel, voir `test/tech-step-matcher.test.ts`), retombe sur la technique
|
pour la valeur courante et comment elle a été calibrée), retombe sur la
|
||||||
impliquée par l'ancre NER de la clause plutôt que d'abandonner un match
|
technique impliquée par l'ancre NER de la clause plutôt que d'abandonner
|
||||||
clairement ancré sur un mot-clé juste parce qu'un petit modèle n'est pas
|
un match clairement ancré sur un mot-clé juste parce que le modèle n'est
|
||||||
assez confiant.
|
pas assez confiant.
|
||||||
|
|
||||||
Entraînement (`_train`) et résolution `TechStep.key -> id` sont mémoïsés une
|
Entraînement (`_train`) et résolution `TechStep.key -> id` sont mémoïsés une
|
||||||
seule fois sur le singleton partagé `techStepClassifier` (jamais par requête).
|
seule fois sur le singleton partagé `techStepClassifier` (jamais par requête).
|
||||||
Le tout premier appel réel à `NlpManager.process()` déclenche aussi le
|
`server.ts` appelle `techStepClassifier.warmUp()` avant d'accepter du trafic,
|
||||||
chargement paresseux des ressources par langue de node-nlp (plusieurs
|
avec retry/backoff si `services/tech-step-intent-service` n'est pas encore
|
||||||
secondes, mesuré) — `server.ts` appelle `techStepClassifier.warmUp()` avant
|
prêt (le cas normal en Docker Compose, où `app` attend qu'il soit `healthy`
|
||||||
d'accepter du trafic pour que ce ne soit jamais la première vraie requête qui
|
avant même de démarrer — voir `docker-compose.yml`).
|
||||||
attend.
|
|
||||||
|
|
||||||
**Deux pièges rencontrés en construisant ce pipeline**, tous deux corrigés
|
**Pièges rencontrés en construisant ce pipeline**, tous corrigés dans le code
|
||||||
dans le code (pas juste contournés) :
|
(pas juste contournés) :
|
||||||
- `db/prisma.ts` construisait `new PrismaClient()` sans jamais importer
|
- `db/prisma.ts` construisait `new PrismaClient()` sans jamais importer
|
||||||
`config/env.ts` — dans le run de test complet, un *autre* fichier
|
`config/env.ts` — dans le run de test complet, un *autre* fichier
|
||||||
chargeait toujours `config/env.ts` (donc `.env.test`) en premier par pur
|
chargeait toujours `config/env.ts` (donc `.env.test`) en premier par pur
|
||||||
|
|
@ -525,12 +534,20 @@ dans le code (pas juste contournés) :
|
||||||
tant que `resetDatabase()` ne throw pas (heureusement son garde-fou le
|
tant que `resetDatabase()` ne throw pas (heureusement son garde-fou le
|
||||||
fait). Fixé en import `config/env.js` pour effet de bord tout en haut de
|
fait). Fixé en import `config/env.js` pour effet de bord tout en haut de
|
||||||
`prisma.ts`, avant `new PrismaClient()`.
|
`prisma.ts`, avant `new PrismaClient()`.
|
||||||
- `NlpManager` a `autoSave`/`autoLoad: true` par défaut — persiste le
|
- (historique, node-nlp) `NlpManager` avait `autoSave`/`autoLoad: true` par
|
||||||
modèle entraîné dans un fichier `model.nlp` (cwd du process) et le
|
défaut — persistait le modèle entraîné dans un fichier `model.nlp` (cwd du
|
||||||
recharge *au lieu de* ré-entraîner au prochain démarrage s'il existe déjà.
|
process) et le rechargeait *au lieu de* ré-entraîner au prochain démarrage
|
||||||
Un modèle obsolète sur disque masquerait silencieusement toute mise à
|
s'il existait déjà. Un modèle obsolète sur disque aurait masqué
|
||||||
jour de `TECH_STEP_TRAINING_DATA`/`CONFIDENCE_THRESHOLD`. Les deux sont
|
silencieusement toute mise à jour de `TECH_STEP_TRAINING_DATA`. Non
|
||||||
explicitement à `false` dans le constructeur de `TechStepClassifierService`.
|
applicable au service Python actuel : `POST /v1/train` reconstruit tout en
|
||||||
|
mémoire à chaque appel, sans jamais rien persister sur disque (voir ce
|
||||||
|
service's own README).
|
||||||
|
- `nlp.make_doc()` (spaCy) ne fait tourner que le tokenizer, pas les
|
||||||
|
composants du pipeline — un piège trouvé en construisant le `PhraseMatcher`
|
||||||
|
du nouveau service : les patterns de synonymes doivent explicitement
|
||||||
|
repasser par le composant de normalisation, sinon un synonyme accentué
|
||||||
|
("préchauffer") ne matche jamais sa forme normalisée dans le texte cible
|
||||||
|
(voir le commentaire dans `locale_pipeline.py`'s `train()`).
|
||||||
|
|
||||||
### Résolution ingrédients/unités — `ingredient-matcher.ts`
|
### Résolution ingrédients/unités — `ingredient-matcher.ts`
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -353,7 +353,8 @@ fiable.
|
||||||
`tech_step` (`TechStep`, `key` unique, ex. `"simmer"`) est le catalogue des
|
`tech_step` (`TechStep`, `key` unique, ex. `"simmer"`) est le catalogue des
|
||||||
techniques (mijoter, préchauffer…) — juste un id/clé stable référencé par
|
techniques (mijoter, préchauffer…) — juste un id/clé stable référencé par
|
||||||
`step_tech_step`. Les données de détection elles-mêmes (synonymes + phrases
|
`step_tech_step`. Les données de détection elles-mêmes (synonymes + phrases
|
||||||
d'exemple par langue, entraînant un classifieur `node-nlp`) vivent en code
|
d'exemple par langue, entraînant le microservice spaCy
|
||||||
|
`services/tech-step-intent-service`) vivent en code
|
||||||
(`tech-step-training-data.ts`), pas dans une table — l'ancienne
|
(`tech-step-training-data.ts`), pas dans une table — l'ancienne
|
||||||
`tech_step_mapping` (`TechStepMapping`, une regex par technique/locale) a
|
`tech_step_mapping` (`TechStepMapping`, une regex par technique/locale) a
|
||||||
été supprimée une fois constaté que les regex ne généralisaient jamais
|
été supprimée une fois constaté que les regex ne généralisaient jamais
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue