Nicolas
0355b1cd99
Les essais 1-2 de cette branche (nom de conteneur Postgres par run) ont laissé des conteneurs `postgres-122` / `postgres-125` en marche sur le runner persistant : leur teardown `if: always()` n'a pas tourné (ce runner saute les étapes finales après un échec) et rien d'autre ne cible ce préfixe. Un 2e Postgres inactif affame la RAM du runner → requêtes ~5× plus lentes → `resetDatabase()` dépasse le timeout Mocha. L'étape « Start Postgres » retire maintenant tout `postgres-*` en plus du `postgres` nu ; plus rien ne crée ce préfixe, donc ça ne touche que ces résidus. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
267 lines
13 KiB
YAML
267 lines
13 KiB
YAML
name: CI
|
|
|
|
# Every push, on every branch — not just main — so build breakage shows up
|
|
# on the first commit of a branch, not only once a PR targets main. No
|
|
# separate pull_request trigger: for a PR from a branch on this same repo,
|
|
# push already fires on that branch, and GitHub attaches the run to the PR
|
|
# by commit SHA regardless of which event triggered it — adding
|
|
# pull_request too would just run every job twice per push. (Re-add it,
|
|
# scoped to forks, only if this repo starts accepting fork PRs — push
|
|
# events from a fork never reach here.)
|
|
on:
|
|
push:
|
|
|
|
env:
|
|
# Hostname `postgres` = the container the `test` job starts itself (see
|
|
# that job's "Start Postgres" step) — a plain `docker run`, not a
|
|
# `services:` block, for the same reason the intent-service below can't be
|
|
# one: this repo's Forgejo `act` runner does not keep a `services:`
|
|
# container as the *same* instance across a job's steps, so `migrate
|
|
# deploy` would land the schema in one Postgres and `pnpm --filter api
|
|
# test`'s `beforeAll` would then hit an empty one ("table … does not
|
|
# exist" right after "migrations successfully applied"). One long-lived
|
|
# container in the job, shared by every step, removes that failure mode.
|
|
DATABASE_URL: "postgresql://ci:ci@postgres:5432/batchcooking_ci?schema=public"
|
|
# Test-only secret, never used outside CI — real deployments must set their own.
|
|
JWT_SECRET: "ci-only-secret-not-used-anywhere-else-32chars+"
|
|
# Same reasoning as JWT_SECRET above — lets tech-step-worker.routes.test.ts
|
|
# exercise the success path (matching secret), not just the "unset"
|
|
# rejection every environment that doesn't set this gets by default.
|
|
INTERNAL_WORKER_SECRET: "ci-only-worker-secret-not-used-anywhere-else-32chars+"
|
|
# Same reasoning — lets admin-auth.test.ts exercise the admin login
|
|
# success path + the requireAdmin-guarded routes, not just the "no admin
|
|
# secret configured -> 401" path.
|
|
ADMIN_JWT_SECRET: "ci-only-admin-secret-not-used-anywhere-else-32chars+"
|
|
# Shared between the `test` job's own uvicorn step (below) and apps/api's
|
|
# IntentServiceClient — that step runs `uvicorn` as a plain background
|
|
# process (not a `services:` block or even a container) because CI can
|
|
# only pull a published image, not build
|
|
# services/tech-step-intent-service/Dockerfile from this checkout.
|
|
INTENT_SERVICE_BASE_URL: "http://localhost:8000"
|
|
INTENT_SERVICE_SECRET: "ci-only-intent-secret-not-used-anywhere-else-32chars+"
|
|
|
|
jobs:
|
|
# Five independent jobs, no needs: between them — each starts in parallel
|
|
# and reports as its own check, instead of the previous single chained
|
|
# "lint-and-test then e2e" pipeline.
|
|
lint:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm lint
|
|
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
# ~16-19 min of actual work (runs 117-124); the extra budget covers time
|
|
# spent queued in "Wait for any earlier test job" below. Also a runaway
|
|
# ceiling — Mocha has no `--exit` (`apps/api/.mocharc.json`), so one
|
|
# leaked handle would otherwise hang the job until the runner's global
|
|
# timeout (a `test` job zombied ~1h once). If a hung job hits this
|
|
# ceiling the runner removes its container, which unblocks the queue.
|
|
timeout-minutes: 45
|
|
steps:
|
|
- name: Install curl and the docker CLI
|
|
run: |
|
|
apt-get update && apt-get install -y curl docker.io
|
|
|
|
# Serialize `test` across every concurrent run. Two `test` jobs must
|
|
# never overlap: they share one `--name postgres` container on the
|
|
# runner's shared docker network (next step), so their `prisma migrate
|
|
# deploy` + `seedReferenceData` (root Mocha `beforeAll`) race —
|
|
# `Unique constraint failed on (key)` in `seedReferenceData`, "No
|
|
# pending migrations" when there should be some, `tech_step`/`diet` id
|
|
# shifts breaking unrelated assertions; and even past that they starve
|
|
# each other's RAM/CPU (each also trains a spaCy model) until
|
|
# `resetDatabase()` overruns Mocha's 10s hook timeout and cascades.
|
|
# Hit repeatedly (PRs #47, #53, #54), each needing an empty-commit
|
|
# re-trigger once the runner was idle.
|
|
#
|
|
# GitHub's `concurrency:` is the native fix but this Forgejo (9.0.3)
|
|
# predates Actions `concurrency` — it arrived in Forgejo v14 / Gitea
|
|
# 1.26 and is silently ignored here. A lock object (network, container,
|
|
# file) would leak: this runner (forgejo-runner v3.5.0) does NOT run a
|
|
# trailing `if: always()` step after an earlier step fails, so there is
|
|
# no reliable in-job release. Instead, key off the one thing the runner
|
|
# *does* always clean up — the job container itself (removed on pass,
|
|
# fail, and cancel alike): wait while any lower-numbered `test` job's
|
|
# container is still running. Lower task id goes first, so two jobs
|
|
# that start together can't deadlock; nothing to release.
|
|
- name: Wait for any earlier test job
|
|
run: |
|
|
my_id="$(printf '%s\n' "${JOB_CONTAINER_NAME:-}" | sed -n 's/.*TASK-\([0-9]\{1,\}\).*/\1/p')"
|
|
echo "this test job is task ${my_id:-<unknown, not serializing>}"
|
|
# ~25 min cap (75 * 20s): comfortably covers one earlier run
|
|
# (the realistic overlap is two runs); a rarer 3-way pile-up whose
|
|
# wait exceeds this just fails and gets an empty-commit re-run.
|
|
for attempt in $(seq 1 75); do
|
|
earlier="$(docker ps --format '{{.Names}}' 2>/dev/null \
|
|
| sed -n 's/.*TASK-\([0-9]\{1,\}\)_.*_JOB-test$/\1/p' \
|
|
| awk -v me="${my_id:-0}" 'NF && ($1 + 0) < (me + 0)')"
|
|
if [ -z "$earlier" ]; then
|
|
[ "$attempt" -gt 1 ] && echo "earlier test job(s) finished — proceeding"
|
|
exit 0
|
|
fi
|
|
echo "waiting for earlier test job(s): $(printf '%s ' $earlier)(attempt ${attempt})"
|
|
sleep 20
|
|
done
|
|
echo "still blocked by an earlier test job after ~25 min — giving up; a re-run should get through" >&2
|
|
exit 1
|
|
|
|
# Postgres as a plain background container instead of a `services:`
|
|
# block — see this workflow's top-level `env.DATABASE_URL` comment for
|
|
# why. Reachable as `postgres` from this job (the runner mounts the
|
|
# docker socket and runs jobs on a shared user-defined network, so a
|
|
# `--name postgres` sibling resolves by name); the network is
|
|
# discovered from the job container itself rather than hardcoded. The
|
|
# `docker rm -f postgres` below also clears a container left behind by
|
|
# the previous run (this runner has no reliable post-job cleanup), and
|
|
# the serialization step above means no other run is using it now.
|
|
- name: Start Postgres in the background
|
|
run: |
|
|
net="$(docker inspect "$(hostname)" \
|
|
--format '{{range $k, $_ := .NetworkSettings.Networks}}{{$k}} {{end}}' \
|
|
| awk '{print $1}')"
|
|
echo "job network: ${net:?could not determine the job container network}"
|
|
docker rm -f postgres >/dev/null 2>&1 || true
|
|
# Also drop any `postgres-<suffix>` container. A since-reverted
|
|
# attempt at per-run Postgres naming on this branch left some
|
|
# running on this persistent runner (their `if: always()` teardown
|
|
# never fired — this runner skips trailing steps after a failure),
|
|
# and nothing else removes them. A second idle Postgres here starves
|
|
# the runner's RAM and slows every query ~5x → `resetDatabase()`
|
|
# blows Mocha's 10s hook timeout. Nothing creates `postgres-*` any
|
|
# more, so this only ever clears that debris.
|
|
docker ps -aq --filter 'name=^postgres-' | xargs -r docker rm -f >/dev/null 2>&1 || true
|
|
docker run -d --name postgres --network "$net" \
|
|
-e POSTGRES_USER=ci -e POSTGRES_PASSWORD=ci -e POSTGRES_DB=batchcooking_ci \
|
|
postgres:16-alpine
|
|
for _ in $(seq 1 60); do
|
|
docker exec postgres pg_isready -U ci -d batchcooking_ci && exit 0
|
|
sleep 2
|
|
done
|
|
echo "Postgres did not become ready in time" >&2
|
|
docker logs postgres || true
|
|
exit 1
|
|
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- uses: https://github.com/astral-sh/setup-uv@v5
|
|
with:
|
|
python-version: "3.12"
|
|
enable-cache: true
|
|
|
|
# A `services:` block can only pull an already-published image — it
|
|
# can't build services/tech-step-intent-service/Dockerfile from this
|
|
# checkout. Running `uvicorn` as a plain background step instead: it
|
|
# keeps running for the rest of this job (steps in one job share the
|
|
# same process tree), and `pnpm --filter api test` below needs a real
|
|
# instance to talk to per this repo's "never mock an internal service"
|
|
# test convention — same "real dependency, started by the job itself"
|
|
# posture as the Postgres step above, not a mock HTTP server.
|
|
- name: Install services/tech-step-intent-service
|
|
working-directory: services/tech-step-intent-service
|
|
run: uv sync --frozen
|
|
- name: Start services/tech-step-intent-service in the background
|
|
working-directory: services/tech-step-intent-service
|
|
run: |
|
|
uv run uvicorn intent_service.main:app --host 0.0.0.0 --port 8000 &
|
|
# `/health` only returns 200 once this service has finished
|
|
# training itself from scratch (no model ever persisted to disk —
|
|
# see its own README) — measured at ~3min combined (fr+en) against
|
|
# the current ~74-technique corpus, down from ~930s once
|
|
# `spacy.TextCatBOW.v3`'s `length` stopped being left at its
|
|
# 262144 default (see `_TEXTCAT_MODEL_LENGTH` in
|
|
# `locale_pipeline.py`, and docker-compose.yml's healthcheck for
|
|
# the same reasoning). The timeout stays well above that: it's a
|
|
# ceiling for a slow/contended runner, not a target.
|
|
timeout 600 bash -c 'until curl -sf http://localhost:8000/health > /dev/null; do sleep 2; done'
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm --filter api exec prisma migrate deploy
|
|
- run: pnpm --filter api test
|
|
|
|
intent-service-test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: https://github.com/astral-sh/setup-uv@v5
|
|
with:
|
|
python-version: "3.12"
|
|
enable-cache: true
|
|
|
|
- name: Install services/tech-step-intent-service
|
|
working-directory: services/tech-step-intent-service
|
|
run: uv sync --frozen
|
|
- name: Run pytest
|
|
working-directory: services/tech-step-intent-service
|
|
run: uv run pytest -q
|
|
|
|
build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm build
|
|
|
|
e2e:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- name: Install Xvfb and Cypress dependencies
|
|
run: |
|
|
apt-get update
|
|
apt-get install -y xvfb libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2 libxtst6 xauth dbus-x11
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- name: Cache Cypress binary
|
|
uses: https://github.com/actions/cache@v4
|
|
with:
|
|
path: ~/.cache/Cypress
|
|
key: cypress-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
# pnpm install doesn't reliably trigger Cypress's postinstall binary
|
|
# download (see apps/web's cypress caveat in the README) — install it
|
|
# explicitly so `cypress run` finds it.
|
|
- run: pnpm --filter web exec cypress install
|
|
- name: Run E2E tests
|
|
env:
|
|
HOST: "0.0.0.0"
|
|
run: pnpm --filter web e2e
|
|
# No dev server needed here — Cypress spins up its own Vite dev
|
|
# server internally for component testing (see cypress.config.ts's
|
|
# `component.devServer`), unlike `e2e` above which needs the real app
|
|
# running first.
|
|
- run: pnpm --filter web cy:run:component
|