recipe-maintainers/cc-ci
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(2): ghost healthcheck start_period overlay — fixes fresh-migration lock deadlock

Browse Source 
Root cause: Ghost's fresh-DB first boot runs a ~6-9min schema migration (round-trip-bound, not CPU);
the recipe healthcheck start_period:1m (~6min grace) kills the still-migrating task, leaving a stale
migrations_lock → every later task deadlocks (MigrationsAreLockedError). Hit on both 2- and 4-vCPU.
Fix (cc-ci deploy overlay, NOT a recipe/test change): compose.ccci-health.yml raises app healthcheck
start_period to 900s, wired via recipe_meta COMPOSE_FILE + install_steps.sh (+ CHAOS_BASE_DEPLOY for
the untracked overlay). No assertion weakened. Budget 1200s = migration + convergence. Only the
install tier needs it (upgrade redeploys on the populated DB → fast boot).
This commit is contained in:
autonomic-bot
2026-05-30 05:23:47 +01:00
parent 9771b6e16a
commit 13da216f8d
3 changed files with 60 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
tests/ghost/install_steps.sh Executable file
View File

@ -0,0 +1,26 @@
#!/usr/bin/env bash
# ghost — INSTALL-TIME hook (Phase 2 Q4.4). Runs during the install tier AFTER `abra app new` +
# EXTRA_ENV + `abra app secret generate` and BEFORE the single `abra app deploy`
# (lifecycle.py::_run_install_steps), with CCCI_RECIPE / CCCI_APP_DOMAIN / CCCI_APP_ENV in env.
#
# Purpose: provide the cc-ci deploy overlay `compose.ccci-health.yml` (app healthcheck start_period
# bump) into the recipe checkout so recipe_meta's COMPOSE_FILE (compose.yml:compose.ccci-health.yml)
# resolves. Without the larger start_period, Ghost's ~6-9min fresh-DB migration is killed mid-flight
# by the recipe's 1m-start_period healthcheck, leaving a stale migrations_lock → deadlock (see the
# overlay file header). The overlay is an UNTRACKED file in the recipe repo, so `git checkout -f`
# (the upgrade tier's re-checkout to PR head) preserves it — COMPOSE_FILE keeps resolving across
# install AND upgrade deploys. CHAOS_BASE_DEPLOY=True (recipe_meta) lets the pinned base deploy
# proceed despite this untracked file (abra's clean-tree check would otherwise FATA).
set -euo pipefail
: "${CCCI_RECIPE:?missing CCCI_RECIPE}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
RECIPE_DIR="${HOME}/.abra/recipes/${CCCI_RECIPE}"
if [ ! -d "$RECIPE_DIR" ]; then
echo " ghost install_steps: recipe dir $RECIPE_DIR missing — cannot provide health overlay" >&2
exit 1
fi
cp "$SCRIPT_DIR/compose.ccci-health.yml" "$RECIPE_DIR/compose.ccci-health.yml"
echo " ghost install_steps: provided compose.ccci-health.yml (healthcheck start_period bump) to ${CCCI_RECIPE}"