ci: isole la base Postgres du job test par run
Deux jobs `test` qui se chevauchent (push de branche peu après un merge sur `main`) se connectaient au *même* conteneur `--name postgres` sur le réseau partagé du runner Forgejo : leurs `prisma migrate deploy` + `seedReferenceData` (beforeAll racine Mocha) se télescopaient — P2002 sur `key`, « No pending migrations » à tort, décalage des ids de `tech_step`/`diet` cassant des assertions sans rapport. Vu 3× en une session (PR #47, #53, #54), à chaque fois re-déclenché par un commit vide sur un runner au repos. `concurrency:` serait le correctif naturel mais ce Forgejo (9.0.3) est antérieur à son support côté Actions (arrivé en Forgejo v14 / Gitea 1.26, et workflow-level uniquement même là) : la clé est ignorée silencieusement. À la place, le conteneur prend un nom unique par run (`postgres-${GITHUB_RUN_ID}`), poussé dans `DATABASE_URL` via `$GITHUB_ENV` pour les étapes suivantes — deux runs ne partagent plus jamais de base. Nettoyage : étape de teardown `if: always()` + purge des conteneurs abandonnés (bornée par l'âge) au démarrage, pour un job tué net avant son teardown. `timeout-minutes: 40` borne aussi le cas « job zombie ~1h » observé cette session (Mocha sans `--exit`). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
713d6c520d
commit
721a1ecf81
1 changed files with 81 additions and 15 deletions
96
.github/workflows/ci.yml
vendored
96
.github/workflows/ci.yml
vendored
|
|
@ -12,13 +12,18 @@ on:
|
|||
push:
|
||||
|
||||
env:
|
||||
# Hostname `postgres` = the container the `test` job starts itself (see
|
||||
# that job's "Start Postgres" step) — a plain `docker run`, not a
|
||||
# `services:` block, for the same reason the intent-service below can't be
|
||||
# one: this repo's Forgejo `act` runner does not keep a `services:`
|
||||
# container as the *same* instance across a job's steps, so `migrate
|
||||
# deploy` would land the schema in one Postgres and `pnpm --filter api
|
||||
# test`'s `beforeAll` would then hit an empty one ("table … does not
|
||||
# Only the `test` job ever opens a database connection, and it *overwrites*
|
||||
# this value at runtime (see its "Start Postgres" step) with a per-run
|
||||
# hostname. This entry stays for one reason: `build`'s `pnpm install` runs
|
||||
# `prisma generate` (apps/api `postinstall`), which wants DATABASE_URL to
|
||||
# be present and syntactically valid — it never dials the host, so the
|
||||
# literal `postgres` here need not resolve anywhere.
|
||||
#
|
||||
# Why the `test` job runs Postgres as a plain `docker run` and not a
|
||||
# `services:` block: this repo's Forgejo `act` runner does not keep a
|
||||
# `services:` container as the *same* instance across a job's steps, so
|
||||
# `migrate deploy` would land the schema in one Postgres and `pnpm --filter
|
||||
# api test`'s `beforeAll` would then hit an empty one ("table … does not
|
||||
# exist" right after "migrations successfully applied"). One long-lived
|
||||
# container in the job, shared by every step, removes that failure mode.
|
||||
DATABASE_URL: "postgresql://ci:ci@postgres:5432/batchcooking_ci?schema=public"
|
||||
|
|
@ -61,6 +66,13 @@ jobs:
|
|||
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
# Runaway ceiling, not a target: the job takes ~16-19 min normally (runs
|
||||
# 117-121). Mocha runs with no `--exit` (`apps/api/.mocharc.json`), so a
|
||||
# single leaked handle would otherwise hang this job until the runner's
|
||||
# global timeout — a `test` job zombied for ~1h once before being
|
||||
# cancelled by hand. 40 min leaves roughly 2x headroom for a slow /
|
||||
# contended runner while still capping a hang at well under that hour.
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- name: Install curl and the docker CLI
|
||||
run: |
|
||||
|
|
@ -68,26 +80,70 @@ jobs:
|
|||
|
||||
# Postgres as a plain background container instead of a `services:`
|
||||
# block — see this workflow's top-level `env.DATABASE_URL` comment for
|
||||
# why. Reachable as `postgres` from this job (the runner mounts the
|
||||
# docker socket and runs jobs on a shared user-defined network, so a
|
||||
# `--name postgres` sibling resolves by name); the network is
|
||||
# discovered from the job container itself rather than hardcoded.
|
||||
# why. The runner mounts the docker socket and runs every job on one
|
||||
# shared user-defined network, so a `--name` sibling resolves by name;
|
||||
# the network is discovered from the job container itself rather than
|
||||
# hardcoded.
|
||||
#
|
||||
# The container name carries the run id (`postgres-${GITHUB_RUN_ID}`)
|
||||
# rather than a bare `postgres` *because* that network is shared by
|
||||
# every job on this runner, across all branches. With a fixed name,
|
||||
# two `test` jobs overlapping — a feature push landing within minutes
|
||||
# of a `main` merge — both connected to the *same* container, and
|
||||
# their `prisma migrate deploy` + `seedReferenceData` (root Mocha
|
||||
# `beforeAll`) raced: `Unique constraint failed on the fields: (key)`
|
||||
# in `seedReferenceData`, "No pending migrations to apply" when there
|
||||
# should be some, `tech_step`/`diet` reference-row id shifts (+5)
|
||||
# breaking unrelated assertions. Hit three times in one session (PRs
|
||||
# #47, #53, #54), each needing an empty-commit re-trigger once the
|
||||
# runner was idle. A GitHub-style `concurrency:` group on this job is
|
||||
# the natural fix, but this Forgejo (9.0.3) predates Actions
|
||||
# `concurrency` support entirely — it arrived in Forgejo v14 / Gitea
|
||||
# 1.26 and the key is silently ignored on anything older. A per-run
|
||||
# container name reaches the same end (no two runs share a database)
|
||||
# with zero runner support required; the `if: always()` teardown step
|
||||
# at the end of the job, plus the age-gated reap below, keep these
|
||||
# per-run containers from piling up on the runner host.
|
||||
- name: Start Postgres in the background
|
||||
run: |
|
||||
pg_host="postgres-${GITHUB_RUN_ID}"
|
||||
|
||||
# Best-effort reap of per-run Postgres containers left behind by a
|
||||
# `test` job that was force-killed before its teardown step could
|
||||
# run (SIGKILL skips `if: always()` steps). Age-gated well above
|
||||
# this job's `timeout-minutes`, so a slower *concurrent* `test`
|
||||
# job's container is never in range.
|
||||
reap_before=$(( $(date +%s) - 7200 ))
|
||||
for c in $(docker ps -aq --filter "name=postgres-"); do
|
||||
started="$(docker inspect -f '{{.State.StartedAt}}' "$c" 2>/dev/null)" || continue
|
||||
started_epoch="$(date -d "$started" +%s 2>/dev/null)" || continue
|
||||
if [ "$started_epoch" -lt "$reap_before" ]; then
|
||||
echo "reaping abandoned Postgres container $c (started $started)"
|
||||
docker rm -f "$c" >/dev/null 2>&1 || true
|
||||
fi
|
||||
done
|
||||
|
||||
net="$(docker inspect "$(hostname)" \
|
||||
--format '{{range $k, $_ := .NetworkSettings.Networks}}{{$k}} {{end}}' \
|
||||
| awk '{print $1}')"
|
||||
echo "job network: ${net:?could not determine the job container network}"
|
||||
docker rm -f postgres >/dev/null 2>&1 || true
|
||||
docker run -d --name postgres --network "$net" \
|
||||
docker rm -f "$pg_host" >/dev/null 2>&1 || true
|
||||
docker run -d --name "$pg_host" --network "$net" \
|
||||
-e POSTGRES_USER=ci -e POSTGRES_PASSWORD=ci -e POSTGRES_DB=batchcooking_ci \
|
||||
postgres:16-alpine
|
||||
|
||||
# Hand this run's hostname to every later step (prisma migrate
|
||||
# deploy, pnpm --filter api test) — this line overrides the
|
||||
# workflow-level env.DATABASE_URL, whose `postgres` host no longer
|
||||
# resolves.
|
||||
echo "DATABASE_URL=postgresql://ci:ci@${pg_host}:5432/batchcooking_ci?schema=public" >> "$GITHUB_ENV"
|
||||
|
||||
for _ in $(seq 1 60); do
|
||||
docker exec postgres pg_isready -U ci -d batchcooking_ci && exit 0
|
||||
docker exec "$pg_host" pg_isready -U ci -d batchcooking_ci && exit 0
|
||||
sleep 2
|
||||
done
|
||||
echo "Postgres did not become ready in time" >&2
|
||||
docker logs postgres || true
|
||||
docker logs "$pg_host" || true
|
||||
exit 1
|
||||
|
||||
- uses: https://github.com/actions/checkout@v4
|
||||
|
|
@ -134,6 +190,16 @@ jobs:
|
|||
- run: pnpm --filter api exec prisma migrate deploy
|
||||
- run: pnpm --filter api test
|
||||
|
||||
# Remove this run's Postgres even when a step above failed or the job
|
||||
# hit its `timeout-minutes` — with a per-run name nothing else ever
|
||||
# reuses or cleans it, and a bare `docker run` sibling outlives the job
|
||||
# container. `|| true`: teardown must never itself fail the job. A job
|
||||
# killed outright (SIGKILL, no `always()` step) is caught by the
|
||||
# age-gated reap in "Start Postgres" on a later run.
|
||||
- name: Stop this run's Postgres
|
||||
if: always()
|
||||
run: docker rm -f "postgres-${GITHUB_RUN_ID}" >/dev/null 2>&1 || true
|
||||
|
||||
intent-service-test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
|
|
|
|||
Loading…
Reference in a new issue