Run #2 base deploy: fresh mysql:8.0 init on the loaded cc-ci host (load ~8) took >6min
(InnoDB ~90s + system-tables + root-pw apply, starved by the app crash-loop churn), exceeding
the recipe's 1m db start_period (+6min retry grace) → swarm killed mysql mid-init (exit 137
unhealthy) → corrupt InnoDB redo logs → permanent deadlock (same signature as run #1's stale
vol). Widen db healthcheck start_period to 15m (matches app) so the slow first-boot finishes
before the healthcheck can fail it. Grace-only, masks no defect; bites base+head (published
recipe ships db start_period 1m everywhere) so overlay covers both. Torn down corrupt vol.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Course correction (REVIEW-2 bdef282) mandates upgrade-to-latest; harness base-deploys
prev published version 1.1.1+6-alpine which predates the recipe-PR 15m start_period bump
(ships 1m) → would deadlock on the ~6-9min fresh-DB migration (swarm kill mid-migration →
held migrations_lock). Policy-blessed minimal base overlay: compose.ccci.yml re-applies the
15m app-healthcheck start_period grace to the BASE so the from-version is deployable;
install_steps.sh provides it; CHAOS_BASE_DEPLOY skips clean-tree on the untracked overlay;
persists across head checkout (idempotent — PR head ships 15m). Grace-only, no test weakened.
Prior corrupt mysql vol (stale, interrupted init) torn down. Next: full run incl upgrade.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
abra rejects env-interpolation in healthcheck start_period (FATA 'Does not match
format duration' for both ${VAR} and quoted forms — validates the literal compose
duration before .env substitution). So §9 pt1's env-var route is impossible for
this field; the §9-compliant fix is a LITERAL start_period:20m bump in the
recipe-PR (recipe everyone runs, not a cc-ci overlay; strictly safer). Remove
APP_START_PERIOD from recipe_meta EXTRA_ENV; record the finding in DECISIONS
(ghost E1 must use the same approach); STATUS-2 → new PR head 7a2e0e0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrate discourse off the cc-ci compose overlay per plan §9 / plan-prefer-env-over-compose-overlay.md:
- recipe_meta: drop UPGRADE_BASE_VERSION + COMPOSE_FILE + CHAOS_BASE_DEPLOY; set APP_START_PERIOD=1200s
via EXTRA_ENV (the recipe-PR exposes start_period: ${APP_START_PERIOD:-5m}); declare upgrade tier N/A
(both published prev bases pin removed bitnami images; Adversary §7.1 granted, REVIEW-2 efe3790).
- delete tests/discourse/compose.ccci-health.yml + install_steps.sh (existed only to copy the overlay).
- DECISIONS.md + STATUS-2 record the §9 guardrail + discourse shape (upgrade N/A, env start_period,
pg_backup restore-hook recipe-PR = 5th data-loss recipe cc-ci caught).
recipe-PR head now 8b8df17 (start_period env var added). Not a claim — run STAGES=install,backup,restore,custom next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The override added in a750937 had no effect: _load_meta only copies a fixed
key whitelist into the meta dict, and UPGRADE_BASE_VERSION wasn't in it, so
meta.get(...) returned None and the upgrade base fell back to previous_version()
= recipe_versions[-2] (0.6.3+3.1.2). Add it to the whitelist so discourse's
honest 0.7.0 base is selected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements the real 0.7.0+3.3.1 -> 0.8.0+3.3.1 upgrade crossover instead of a
§7.1 skip-with-sign-off (Adversary leans DENY on the deferral; agreed):
- recipe_meta UPGRADE_BASE_VERSION=0.7.0+3.3.1 + generic support in
run_recipe_ci (prev = meta override or previous_version). Harness default
[-2]=0.6.3+3.1.2 is a hollow base (img 3.1.2 != head 3.3.1); [-1]=0.7.0+3.3.1
is the PR's true predecessor and shares head's servable 3.3.1 image.
- compose.ccci-health.yml re-pins services.{app,sidekiq}.image to
bitnamilegacy/discourse:3.3.1 so the 0.7.0 base (compose pins 404 bitnami:3.3.1)
is servable; idempotent on the head (PR already bitnamilegacy).
Consumes Adversary BUILDER-INBOX (deleted), leaves ADVERSARY-INBOX ack; STATUS-2
discourse section updated. Full lifecycle run launching next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_discourse.py: bootstrap an admin (recipe seeds none) + mint an ApiKey via rails runner in the app
container (class-B run-scoped). test_create_topic.py: POST /posts.json (unique marker) -> GET
/t/<id>.json title+cooked round-trip. test_site_basic.py: GET /site.json asserts discourse categories
config. Meets P3 (>=2 functional beyond health).
Install timed out at 1800s: discourse's 15-25min Rails cold boot overran both the deploy timeout and
the recipe healthcheck start_period:5m (swarm killed the booting app). Add compose.ccci-health.yml
(app healthcheck start_period 1200s) via install_steps.sh + recipe_meta COMPOSE_FILE + CHAOS_BASE_DEPLOY,
bump DEPLOY_TIMEOUT/TIMEOUT to 2400. Image re-pin (bitnamilegacy) already proven working. NO test weakened.
Correcting my own error. Real Adversary verdict (REVIEW-2 e850281): plausible Q4.7-full env-block claim
REFUTED but it is a RECIPE DEFECT (entrypoint.clickhouse.sh silent-wget restart-storm → ClickHouse never
starts), §7.1 sign-off leaning-DENY → fix via recipe-PR Q4.7b (cache tarball/wget retry+backoff/un-silence).
discourse Q4.6 sign-off DENIED — bitnamilegacy/discourse:3.3.1 served → 1-line re-pin recipe-PR. drone
Q4.10 §7.1 GRANTED. STATUS/DECISIONS/DEFERRED corrected to match. No fabricated refs.
A cc-ci deploy overlay sitting in the recipe checkout as an untracked file (ghost's
compose.ccci-health.yml via install_steps) makes abra stamp chaos-version='<commit>+U' (U=untracked).
The commit still equals head_ref (HC1 satisfied) but the '+U' broke the exact-prefix match → spurious
upgrade-tier FAIL. Strip the working-tree-state marker before the commit match; HC1 preserved (commit
must still equal head_ref — a stale checkout's commit would not match even after stripping). General:
benefits every future cc-ci overlay recipe.
Root cause: Ghost's fresh-DB first boot runs a ~6-9min schema migration (round-trip-bound, not CPU);
the recipe healthcheck start_period:1m (~6min grace) kills the still-migrating task, leaving a stale
migrations_lock → every later task deadlocks (MigrationsAreLockedError). Hit on both 2- and 4-vCPU.
Fix (cc-ci deploy overlay, NOT a recipe/test change): compose.ccci-health.yml raises app healthcheck
start_period to 900s, wired via recipe_meta COMPOSE_FILE + install_steps.sh (+ CHAOS_BASE_DEPLOY for
the untracked overlay). No assertion weakened. Budget 1200s = migration + convergence. Only the
install tier needs it (upgrade redeploys on the populated DB → fast boot).
- ops.py + test_{upgrade,backup,restore}.py: seed ci_marker into the MySQL `ghost` DB (db service)
via the mysql CLI; rides the recipe's mysqldump --tab backup. recipe is MySQL not sqlite (stale
comment fixed). Expect restore RED -> recipe-PR (no backupbot.restore hook; immich/mattermost class).
- functional/_ghost.py: cookie-aware Ghost Admin API client (stdlib http.cookiejar; Origin CSRF hdr).
- functional/test_post_roundtrip.py: §4.3 create published post + read back (unique marker, non-vacuous);
closes the DEFERRED ghost create-post item.
- PARITY.md + recipe_meta.py updated. Authored node-free; full-lifecycle run next, NOT yet claimed.
Root cause of the 2 failing custom tests: TLS_FLAVOR=notls → dovecot refuses plaintext auth over
network 143, so host-side IMAP login/auth isn't a meaningful signal. Smoke2 PROVED the in-container
path: sendmail (postfix container) local-injects a marker mail → doveadm search (imap container) finds
it in INBOX. test_mail_flow now exercises the real postfix→rspamd→dovecot deliver/store/fetch via
exec_in_app(service=smtp/imap). Dropped test_imap_login (network plaintext-auth disallowed under notls).
test_mailbox (create+config-export read-back) unchanged. PARITY.md updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Diagnostic (RECIPE=mumble STAGES=install,backup,restore,custom, no upgrade) PROVED backup+restore green
on a stable 1.0.0 deploy incl. ci_marker survival (P4). The full-run backup 409 ('container not
running') was the chaos UPGRADE redeploy: host-mode 64738 must be released by the old task + rebound by
the new, and HEALTH_PATH '/' only proves the mumble-web sidecar (not the voice server), so wait_healthy
passed while the app churned → backup-bot execed a not-running container. Fix: extend
lifecycle.wait_ready_probes to support a TCP probe ({tcp_host,tcp_port,stable=N consecutive connects});
mumble recipe_meta READY_PROBE returns 64738 (stable=3) so the harness waits for the voice server up
after install AND upgrade before backup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
git checkout <head_ref> aborted on the untracked install_steps-provided compose.host-ports.yml (which
head_ref tracks). Force-checkout yields the exact ref tree. Also fixes the mumble restore tier: backup
labels exist only in 1.0.0+, so backup/restore are meaningful only after the (now-working) upgrade moves
the app to head_ref. DECISIONS.md updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
autonomic-botandClaude Opus 4.8