recipe-maintainers/cc-ci
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(2w): W0.9 WC1.1 hardening (proven live: healthy upgrade + marquee rollback)

Browse Source 
Bugs found by the live proof, fixed:
- warmsnap: snapshot now swaps a <recipe>/snapshot/ SUBDIR, not the whole
  <recipe>/ dir — so the reconciler's sibling last_good file survives a
  snapshot swap (was being clobbered).
- warm_reconcile: deploy_version captures abra's stdout (it writes FATA to
  stdout) in the error; add wait_undeployed() after every undeploy so
  snapshot/restore/redeploy don't race a half-removed swarm stack; the upgrade
  deploy is wrapped so a deploy FAILURE (not just unhealthy) also triggers
  rollback. (57 unit pass.)

LIVE PROOF on warm keycloak (annotated fake tags via CCCI_SKIP_FETCH):
(a) healthy upgrade 10.7.1->10.7.9: snapshot+deploy+health-pass, last_good
    committed=10.7.9, marker realm preserved.
(b) MARQUEE rollback: broken latest 10.7.10 (lint-fail) -> rollback to 10.7.9,
    HEALTHY, marker realm INTACT (data preserved through broken-upgrade+restore),
    last_good NOT advanced, rollback alert written (attempted=10.7.10,
    last_good=10.7.9, recovered=True). keycloak recovered to canonical
    10.7.1+26.6.2 healthy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
autonomic-bot
2026-05-29 01:21:05 +01:00
parent 07ea951f31
commit 32f00717ac
3 changed files with 59 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
tests/unit/test_warmsnap.py
View File

@ -18,8 +18,10 @@ def test_warm_root_env_override(monkeypatch):
monkeypatch.setenv("CCCI_WARM_ROOT", "/tmp/ci-warm-test")
assert warmsnap.warm_root() == "/tmp/ci-warm-test"
assert warmsnap.app_dir("keycloak") == "/tmp/ci-warm-test/keycloak"
assert warmsnap.meta_path("keycloak") == "/tmp/ci-warm-test/keycloak/meta.json"
assert warmsnap.volumes_dir("keycloak") == "/tmp/ci-warm-test/keycloak/volumes"
# snapshot lives in a subdir so sibling state (last_good) survives the atomic swap.
assert warmsnap.snap_dir("keycloak") == "/tmp/ci-warm-test/keycloak/snapshot"
assert warmsnap.meta_path("keycloak") == "/tmp/ci-warm-test/keycloak/snapshot/meta.json"
assert warmsnap.volumes_dir("keycloak") == "/tmp/ci-warm-test/keycloak/snapshot/volumes"
def test_warm_root_default(monkeypatch):
@ -34,11 +36,11 @@ def test_read_meta_absent(monkeypatch, tmp_path):
def _write_snapshot(tmp_path, recipe, volumes):
appdir = tmp_path / recipe
(appdir / "volumes").mkdir(parents=True)
(appdir / "meta.json").write_text(json.dumps({"recipe": recipe, "volumes": volumes}))
snapdir = tmp_path / recipe / "snapshot"
(snapdir / "volumes").mkdir(parents=True)
(snapdir / "meta.json").write_text(json.dumps({"recipe": recipe, "volumes": volumes}))
for v in volumes:
(appdir / "volumes" / f"{v}.tar").write_bytes(b"fake")
(snapdir / "volumes" / f"{v}.tar").write_bytes(b"fake")
def test_has_snapshot_complete(monkeypatch, tmp_path):
@ -49,11 +51,20 @@ def test_has_snapshot_complete(monkeypatch, tmp_path):
assert meta["volumes"] == ["a_mariadb", "a_providers"]
def test_last_good_survives_sibling_of_snapshot(monkeypatch, tmp_path):
# The reconciler's last_good lives in <recipe>/ (sibling of snapshot/) — a test that the layout
# keeps them separate so a snapshot swap can't clobber last_good.
monkeypatch.setenv("CCCI_WARM_ROOT", str(tmp_path))
appdir = tmp_path / "keycloak"
assert warmsnap.snap_dir("keycloak").startswith(str(appdir))
assert os.path.dirname(warmsnap.snap_dir("keycloak")) == str(appdir)
def test_has_snapshot_incomplete_missing_tar(monkeypatch, tmp_path):
monkeypatch.setenv("CCCI_WARM_ROOT", str(tmp_path))
# meta lists two volumes but only one tar present -> incomplete -> not usable.
appdir = tmp_path / "keycloak"
(appdir / "volumes").mkdir(parents=True)
(appdir / "meta.json").write_text(json.dumps({"recipe": "keycloak", "volumes": ["a", "b"]}))
(appdir / "volumes" / "a.tar").write_bytes(b"fake")
snapdir = tmp_path / "keycloak" / "snapshot"
(snapdir / "volumes").mkdir(parents=True)
(snapdir / "meta.json").write_text(json.dumps({"recipe": "keycloak", "volumes": ["a", "b"]}))
(snapdir / "volumes" / "a.tar").write_bytes(b"fake")
assert warmsnap.has_snapshot("keycloak") is False