resolve_upgrade_base now reads the head's published version (abra.head_compose_version,
the coop-cloud.<stack>.version label) and, when the last-green warm-canonical version
equals it, steps back to the newest published version strictly older than head instead
of deploying a same-version no-op. warm_reconcile gains version_key + newest_older_version
(single coop-cloud ordering source; sort_versions refactored onto version_key, no behavior
change). Skip only when no older published predecessor exists. Step-back returns kind=version
so it inherits F1d-2 pinned-tag checkout. Extends tests/unit/test_upgrade_base.py (13 pass).
21/21 recipes GREEN post-prevb. 0 prevb regressions. A-regall-2 closed
(plausible backup_restore=fail was recipe bug in 3.0.1+v2.0.0, NOT prevb;
run 758 / PR#3 / 3.1.0+v2.0.0 confirms L5 pass with fixed backup mechanism).
All batches 1-6 complete. M1+M2 both claimed 2026-06-17T04:45Z.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Builder diagnosis (a3d115d) accepted:
- backupbot.backup.path in 3.0.1+v2.0.0 places dump in writable layer (not restic volume)
- PR#4 (trivial regall trigger at 3.0.1+v2.0.0) exposes the bug; PR#3 (3.1.0+v2.0.0) fixes it
- Baseline run 658 used PR#3 (d77adba4698b) — same passing ref as run 758
Cold-verified: run 758 (PR#3, d77adba4698b) → level=5, backup_restore=pass ✓
Plausible regall result = L5 GREEN. Sweep now 21/21 complete.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
backupbot.backup.path: "/postgres.dump.gz" places dump in container writable
layer (not a volume), so restic never captures it. Restore post-hook fails
with "No such file or directory". PR#3 (3.1.0+v2.0.0) fixes this with
backupbot.backup.volumes.db-data.path. Baseline run 658 tested PR#3 (working
mechanism), not 3.0.1+v2.0.0 (broken). Re-opened PR#3 + !testme triggered
(comment 14651) to demonstrate backup_restore=pass. BUILDER-INBOX consumed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Runs 750 and 754 both fail: ci_marker absent after restore.
No-op upgrade (3.0.1+v2.0.0→3.0.1+v2.0.0) via UPGRADE_BASE_VERSION path is prevb-specific.
Baseline run 658 had genuine git-ref upgrade and passed L5.
Builder-INBOX written. M1 blocked pending plausible fix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The custom tier runs on the PR head — now genuinely the official discourse/discourse image (prevb
stopped the overlay reverting it to bitnamilegacy). mint_admin hardcoded /opt/bitnami/discourse (404 on
official) → create-topic roundtrip failed. Detect /var/www/discourse, re-export DISCOURSE_DB_PASSWORD
from /run/secrets (entrypoint exports it only for boot), run bin/rails; keep bitnami fallback.
docker stack deploy doesn't prune services the head compose dropped (discourse PR#4 drops sidekiq),
leaving them orphaned on the base image. perform_upgrade now reconciles the live stack to the head
compose service set (lifecycle.prune_orphan_services). Makes the deployed stack faithfully reflect
the head — no test weakened. No-op when service sets match / compose unresolvable.
Build #695 (RECIPE=gitea PR=1 REF=357926f26e69): level=5/5, test_lfs_roundtrip PASS (18s).
Build #692 (RECIPE=drone REF=main): level=5/5, dep path confirmed.
All 6 M2 DoD conditions met per Adversary REVIEW-gtea.md @2026-06-15T22:10Z.
Phase gtea complete. Gitea enrolled as a fully-tested recipe with LFS PR verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Build #695 (gitea PR=1 REF=357926f26e69): level=5, all stages PASS, test_lfs_roundtrip
PASS (18s) — LFS roundtrip verified in real CI on lfs-plain-gitea PR #1.
Build #692 (drone dep path PR=0 REF=main): level=5, drone recipe unaffected.
Build #684 (gitea main PR=0): level=5 (verified in prior round).
cc-ci self-test lint green. Unit tests 53/53. no_secret_leak in all runs.
Also records build #691 FAIL finding: STACK_NAME not in .env (fixed in ad53b5a).
Gate M2: ADVERSARY PASS.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
abra does NOT write STACK_NAME to the app's .env file — it derives it at runtime
by replacing dots with underscores (e.g. gite-e1cb78.ci.commoninternet.net →
gite-e1cb78_ci_commoninternet_net). Build #691 failed with 'STACK_NAME not found'
because the env file read was looking for a key that doesn't exist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Blocker 4 fix: abra `secret generate --all` uses .env.sample for length hints; the
lfs-plain-gitea PR has SECRET_LFS_JWT_SECRET_VERSION=v1 COMMENTED OUT, so abra produces
a wrong-length secret. gitea requires exactly 43 chars (32 bytes base64 URL-safe); wrong
length → gitea fatals trying to save the JWT secret to the read-only Docker Config
app.ini → health check fails → swarm rolls back.
Fix: new UPGRADE_SECRET_PREP hook (meta.py) called before `abra secret generate --all`
in the upgrade path. abra's `--all` is idempotent (skips existing secrets), so the
correctly pre-inserted secret survives. gitea's recipe_meta.py implements the hook using
`docker secret create` directly to guarantee correct format regardless of .env.sample.
Also consumes machine-docs/BUILDER-INBOX.md (Adversary Blocker 4 digest).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Build #684 (RECIPE=gitea REF=main PR=0): PASS level=5 — all tiers pass, LFS correctly
SKIP on main, HC1 SHA match (e6a1cc79=e6a1cc79). M2 main-branch DoD MET.
Build #685 (RECIPE=gitea PR=1 REF=357926f26e69): FAIL level=1 — new critical blocker:
upgrade chaos redeploy to PR head with compose.lfs.yml fails with rollback_completed.
Root cause: lfs_jwt_secret generated by abra --all with wrong length/format because
.env.sample in PR #1 has `SECRET_LFS_JWT_SECRET_VERSION=v1 # length=43` COMMENTED OUT.
Gitea starts but fails health check on bad JWT secret → Docker swarm rolls back.
Also filed: cc-ci self-test lint failures (9 ruff format violations in gtea files),
drone dep path not re-verified via live CI since a121d2c.
M2 still NOT claimable — Builder must fix lfs_jwt_secret generation and re-trigger #685.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Blocker 1 (LFS roundtrip fails on PR #1):
- Add UPGRADE_EXTRA_ENV to gitea recipe_meta.py — after PR-head checkout
(compose.lfs.yml now in ABRA_DIR), add compose.lfs.yml to COMPOSE_FILE
and set SECRET_LFS_JWT_SECRET_VERSION=v1 so the upgrade chaos redeploy
actually runs with LFS enabled. Without this, the base install checks out
the 3.5.x tag (compose.lfs.yml removed), EXTRA_ENV sees no LFS, and the
upgrade chaos redeploy inherits the no-LFS .env — so the LFS test runs
(compose.lfs.yml is restored by recipe_checkout_ref) but LFS is off.
- Add abra.secret_generate(domain) in generic.perform_upgrade when
upgrade_env is non-empty — generates lfs_jwt_secret before chaos redeploy.
Blocker 2 (REF=main upgrade fails HC1):
- Always use recipe_head_commit (git rev-parse HEAD) for head_ref instead
of using ref directly. When ref="main" (a branch name), the HC1 commit
check "head_ref.startswith(chaos_commit)" always fails since "main" ≠ SHA.
recipe_head_commit returns the actual SHA after the fetch/checkout.
Side-fix (stale creds — build #675):
- ops.py pre_install: delete the per-domain creds file before calling
_ensure_admin. A fresh install wipes gitea's DB; any creds file from a
prior run on the same domain is stale and causes 401s in all API calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two critical issues prevent M2: (1) lfs_jwt_secret not generated via disk .env → LFS disabled in
container; (2) upgrade tier fails when REF=main. Details + fix hints in BUILDER-INBOX.md.
Run 674 (main): upgrade FAIL ("not intended PR-head"); run 676 (PR#1 LFS): test_lfs_roundtrip
fails at git-push batch endpoint (LFS not enabled in deployed container). Builder must fix before M2.
level=5/5 verified; 53/53 unit tests PASS (Adversary cold run from adv-clone);
code review: all test hooks have teeth; dep path correct; LFS skip correct.
One non-blocking finding: stale screenshot (pre-existing harness bug, manual run_id reuse).
Prerequisite for M2: enables the bridge to pick up !testme comments
on gitea recipe PRs (PR #1 lfs-plain-gitea) and post results back.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LIVENESS PROTOCOL: declared per 10-min rule. Adversary pre-checks done
at 950ab8b, ready to verify. Claim posted at bac3662 (~20:13Z).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Required by test_lfs_roundtrip.py for the M2 LFS capstone run on the
lfs-plain-gitea PR branch. Also revert the same change from the Incus
host (cc-ci/configuration.nix) where it was mistakenly added.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gitea 1.22+ (including 1.24.2 on cc-ci) requires explicit scopes
when creating API tokens. Add read:user + read:organization to satisfy
the token creation endpoint and the read-back assertions that follow.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Empty-repo HTTPS push with git clone exits 0 but silently fails (remote
branch creation on an empty clone is unreliable). Fix:
- Create repo with auto_init=True + default_branch=main (initial commit present)
- Clone into a non-existing subdir (git clone must target non-existing path)
- Push via explicit cred_url (bypasses remote config; no tracking needed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_git_push.py + test_lfs_roundtrip.py: use cred_url (https://user:pass@host/...)
instead of GIT_CONFIG_COUNT insteadOf rewriting, which silently failed to
propagate credentials to the push step (repo remained empty after push exit 0).
Also add GIT_SSL_NO_VERIFY=true and GIT_TERMINAL_PROMPT=0.
- test_lfs_roundtrip.py: fix restart health-poll path /api/v1/version → /version
(_api() already prepends /api/v1; double prefix produced 404 and a 120s timeout).
- nix/hosts/cc-ci/configuration.nix: add git-lfs to systemPackages (required for
the LFS capstone test on the lfs-plain-gitea PR branch).
Adversary pre-M1 findings: Issue 1 (git-lfs absent) + Issue 2 (double path) both fixed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_csrf is a hidden field; wait_for_selector defaults to state=visible
and times out. Switch to the visible username input which proves the
login form rendered.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
autonomic-bot
