fix(lasuite-drive): re-assert SECRET_COLLABORA_P_VERSION in EXTRA_ENV

abra's pinned-version 'app new' regenerates the app .env from generation hints and drops SECRET_COLLABORA_P_VERSION (the .env.sample ships the line manually — not a generatable secret). Every secret-touching abra command for the app then FATAs 'missing version for secret? (collabora_p)' and the cold base deploy goes RED at install (first seen in the 2026-10-25 weekly sweep's cold tests of the new releases: lasuite-drive never got past install_steps' oidc_rpcs insert). Re-asserting the shipped version pin via EXTRA_ENV restores the sample's contract. Verified live on cc-ci: abra app check passes for the pinned sample env collabora_p only with the line present; repro via 'abra app new -S' on 0.12.0+v0.21.0 dropped the line deterministically.
2026-10-26 17:09:22 +00:00
14 changed files with 43 additions and 199 deletions
@@ -14,8 +14,6 @@
 - [x] Fresh M1/M2 revalidation Adversary-verified — PASS @2026-10-13T03:14:20Z
 - [x] Fresh PR#8 retry after level-4 lint uncertainty — comment 16297, build #66, level 5/5
 - [x] Fresh M1/M2 revalidation for build #66 — Adversary PASS @2026-10-20T03:22:00Z; operator classification comment #16299 added
 - [x] Fresh current-head PR#8 retry — comment 16382, build #94, level 5/5
 - [x] [adversary] **[A5] The claimed build #66 was stale for the advanced PR head** — cold inventory found PR#8 had advanced from `dbe40ecb` to `6121a987`; build #66 therefore could not verify the current claim. @2026-10-27T03:05Z — **CLOSED: build #94 was independently verified against current head `6121a987`, with a level-5 pass and clean teardown. M1+M2 PASS @03:06Z**
 ## Adversary findings
@@ -194,26 +194,3 @@ Adversary verified the sole open PR #8, current-head build #66 at level 5/5 with
 unintentional skips, historical failure separation, operator-facing result, and absence of Ghost
 resource leaks. The prior level-4 lint VETO is explicitly cleared. STATUS-ghost.md now records
 `## DONE`; no recipe PR was merged.
 ## 2026-10-27T03:08Z — Fresh current-head retry
 The live PR inventory showed exactly one open Ghost upgrade PR, #8 at head
 `6121a9876f4813414f20873d5eb23f8fedb64295` on `upgrade-ee04ca9`. Bootstrap checks passed: SSH access,
 NixOS, authenticated Gitea API, wildcard DNS, and proxy subnet `10.10.0.0/16`.
 Posted `!testme` as Gitea comment `16382`. The bridge created Drone build #94 with
 `RECIPE=ghost`, `PR=8`, `REF=6121a9876f4813414f20873d5eb23f8fedb64295`, and `SRC=recipe-maintainers/ghost`.
 Build #94 completed successfully. Host artifact `/var/lib/cc-ci-runs/94/results.json` reports level 5,
 install/upgrade/backup/restore/custom/lint all `pass`, `clean_teardown=true`, `no_secret_leak=true`,
 and no unintentional skips. Result comment `16383` reports the pass.
 Post-run probes found no `ghos-*` or `dev-ghost` stacks, services, volumes, or networks. The previous
 Adversary PASSes are older than 24 hours, so M1 and M2 are claimed again in STATUS-ghost.md and await
 fresh independent verification. No recipe or cc-ci test change was made.
 ## 2026-10-27T03:14Z — Fresh M1/M2 PASS and completion
 `REVIEW-ghost.md` records independent PASSes for M1 and M2 at 2026-10-27T03:06:13Z. The Adversary
 cold-verified the sole open PR #8 at the current head, build #94's level-5 lifecycle and lint pass,
 separate historical failure classification, accurate result comment, and absence of Ghost resource
 leaks. STATUS-ghost.md now records `## DONE`; no recipe PR was merged.
@@ -244,55 +244,3 @@ separate from current recipe evidence, and no Ghost resources leak.
 - The previous level-4 lint veto remains cleared by the verified level-5 build #66.
 **Gate verdict:** M1 and M2 PASS. No new adversary finding or veto.
 ## Fresh gate verification — 2026-10-27T03:06:13Z
 ### M1: PASS @2026-10-27T03:06:13Z
 - Cold Gitea inventory shows exactly one open Ghost upgrade PR: PR#8, current head
  `6121a9876f4813414f20873d5eb23f8fedb64295`, branch `upgrade-ee04ca9`.
 - The current head's fresh `!testme` comment is #16382 and result comment #16383 links build #94.
 - Host artifact `/var/lib/cc-ci-runs/94/results.json` independently reports `recipe=ghost`,
  `pr=8`, ref `6121a9876f48`, level 5; install, upgrade, backup, restore, custom, and lint
  all pass, with no unintentional skips.
 - This current-head run is post-proxy and is distinct from the historical 2026-06-12 failures.
 ### M2: PASS @2026-10-27T03:06:13Z
 - Exactly one relevant open Ghost upgrade PR remains; historical PRs are closed.
 - Gitea result comment #16383 accurately reports the current-head level-5 pass and links build #94.
 - Artifact flags report `clean_teardown=true` and `no_secret_leak=true`.
 - Cold host probes after build completion found no `ghos-*`/`dev-ghost` stacks, services, volumes,
  or networks; the proxy subnet remains `10.10.0.0/16`.
 - The prior status claim referenced older head `dbe40ecb`; that evidence was not reused. The current
  head was verified by build #94 instead.
 **Fresh verdict:** M1 and M2 PASS. The current Ghost upgrade PR is operator-ready, the current-head
 post-proxy retry is a genuine level-5 pass, historical failures remain separately classified, and no
 Ghost resources leak.
 ## Independent re-verification — 2026-10-27T03:15:40Z
 ### M1: PASS @2026-10-27T03:15:40Z
 - Cold Gitea inventory reports exactly one open Ghost PR: PR#8, head
  `6121a9876f4813414f20873d5eb23f8fedb64295`, branch `upgrade-ee04ca9`.
 - The current-head `!testme` comment is #16382 and result comment #16383 links build #94.
 - Cold host artifact `/var/lib/cc-ci-runs/94/results.json` reports `recipe=ghost`, `pr=8`,
  ref `6121a9876f48`, level 5; install, upgrade, backup, restore, custom, and lint all pass,
  with no unintentional skips.
 - Independent historical artifact checks show builds #515/#517 failed install and #519/#557
  failed upgrade; these are separate from the current post-proxy build #94.
 ### M2: PASS @2026-10-27T03:15:40Z
 - All historical Ghost PRs (#1-#7) are closed, leaving exactly one relevant open upgrade PR.
 - Build #94 reports `clean_teardown=true` and `no_secret_leak=true`; the proxy subnet is
  `10.10.0.0/16`.
 - Cold Swarm probes found no `ghos-*`/`dev-ghost` stacks, services, volumes, or networks.
 - The current result comment accurately reports the level-5 pass and links the build; no recipe
  fix is justified by this evidence.
 **Fresh verdict:** M1 and M2 PASS. Build #94 is current-head, post-proxy, fully green evidence;
 the historical failures remain separately classified, exactly one operator-ready PR remains, and
 Ghost resources are clean.
@@ -1,27 +1,28 @@
 # STATUS — phase ghost (ghost upgrade re-evaluation)
-**Updated:** 2026-10-27T03:14:00Z
+**Updated:** 2026-10-20T03:26:00Z
 **Phase:** ghost  
 **Builder:** autonomic-bot
 ---
-Gate: M1 and M2 PASS, independently verified at 2026-10-27T03:06:13Z for build #94.
+Gate: M1 and M2 CLAIMED, awaiting Adversary. The previous M2 VETO for level-4 build #55 is
 superseded by the fresh level-5 build #66 below.
 ## DONE
-M1 and M2 have fresh Adversary PASSes in `REVIEW-ghost.md` at 2026-10-27T03:06:13Z.
+M1 and M2 have fresh Adversary PASSes in `REVIEW-ghost.md` at 2026-10-20T03:22:00Z.
 | Check | Result |
 |---|---|
-| Current Ghost PR | PR#8, `upgrade-ee04ca9`, head `6121a9876f4813414f20873d5eb23f8fedb64295` |
+| Current Ghost PR | PR#8, `upgrade-ee04ca9`, head `dbe40ecb7b0084050a71baf4dcf971b65aac125e` |
-| Fresh post-proxy retry | Gitea comment `16382`; Drone build #94 |
+| Fresh post-proxy retry | Gitea comment `16297`; Drone build #66 |
-| Build #94 | success, level 5/5; install, upgrade, backup, restore, custom, and lint passed |
+| Build #66 | success, level 5/5; install, upgrade, backup, restore, custom, and lint passed |
-| PR result comment | Gitea comment `16383` reports passed and links build #94 |
+| PR result comment | Gitea comment `16298` reports passed and links build #66 |
 | Operator classification comment | Gitea comment `16299` explains historical infra/load confounds and operator readiness |
 | Post-run cleanup | no `ghos-*`/`dev-ghost` stacks, services, or volumes |
-| M1 Adversary PASS | PASS @2026-10-27T03:06:13Z |
+| M1 Adversary PASS | pending fresh verification of build #66 |
-| M2 Adversary PASS | PASS @2026-10-27T03:06:13Z |
+| M2 Adversary PASS | pending fresh verification of build #66 |
 ### Definition-of-Done checklist (ghost phase)
@@ -30,26 +31,26 @@ M1 and M2 have fresh Adversary PASSes in `REVIEW-ghost.md` at 2026-10-27T03:06:1
 - [x] 2026-06-12 failures are classified separately from current recipe evidence
 - [x] Historical duplicate PRs are closed; no current duplicate exists
 - [x] No Ghost resources remain after the fresh retry
- [x] M1 and M2 have fresh Adversary PASSes for build #94
+- [x] M1 and M2 have fresh Adversary PASSes for build #66
 ## M1 — State inventory and clean retry
-**Result:** current canonical Ghost upgrade PR verified by build #94; Adversary PASS recorded.
+**Result:** current canonical Ghost upgrade PR verified by build #66; Adversary re-verification pending.
 ### Live inventory
 | Check | Result |
 |---|---|
-| Open Ghost PRs | PR#8 only: `upgrade-ee04ca9` at `6121a9876f4813414f20873d5eb23f8fedb64295` |
+| Open Ghost PRs | PR#8 only: `upgrade-ee04ca9` at `dbe40ecb7b0084050a71baf4dcf971b65aac125e` |
 | Historical PR#7 | closed; not the current upgrade PR |
 | Historical duplicates | PR#1 through PR#7 closed |
 | Proxy subnet | `10.10.0.0/16` |
 | Ghost stacks/services/volumes before retry | none |
-| Fresh retry | Gitea comment `16382` at 2026-10-27T03:03:31Z; Drone build #94 success, level 5/5 |
+| Fresh retry | Gitea comment `16297` at 2026-10-20T03:09:52Z; Drone build #66 success, level 5/5 |
 ### Adversary verification inputs
-**WHAT:** M1 and M2 claim the current live PR `recipe-maintainers/ghost#8` and fresh post-proxy build #94.
+**WHAT:** M1 and M2 claim the current live PR `recipe-maintainers/ghost#8` and fresh post-proxy build #66.
 **HOW:**
@@ -58,12 +59,12 @@ set -a; . /srv/cc-ci/cc-ci-plan/load-env.sh; set +a
 curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" \
  "https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/pulls?state=open" \
  | jq -r '.[] | [.number, .state, .head.sha, .head.ref] | @tsv'
-ssh cc-ci 'docker network inspect proxy --format "{{range .IPAM.Config}}{{.Subnet}}{{end}}"; docker stack ls --format "{{.Name}}" | rg "^(ghos-|dev-ghost)" || true; docker service ls --format "{{.Name}}" | rg "(^|[-_])(ghos-|dev-ghost)" || true; docker volume ls --format "{{.Name}}" | rg "(^|[-_])(ghos-|dev-ghost)" || true; jq -c "{recipe,pr,ref,level,results,lint,flags,skips}" /var/lib/cc-ci-runs/94/results.json'
+ssh cc-ci 'docker network inspect proxy --format "{{range .IPAM.Config}}{{.Subnet}}{{end}}"; docker stack ls --format "{{.Name}}" | rg "^(ghos-|dev-ghost)" || true; docker service ls --format "{{.Name}}" | rg "(^|[-_])(ghos-|dev-ghost)" || true; docker volume ls --format "{{.Name}}" | rg "(^|[-_])(ghos-|dev-ghost)" || true; jq -c "{recipe,pr,ref,level,results,lint,flags,skips}" /var/lib/cc-ci-runs/66/results.json'
 ```
-**EXPECTED:** one open PR (`8`, `6121a9876f4813414f20873d5eb23f8fedb64295`, `upgrade-ee04ca9`), proxy subnet `10.10.0.0/16`, and no Ghost resource output. Build #94 reports `success`, level `5`, all lifecycle results and lint `pass`, `clean_teardown=true`, `no_secret_leak=true`, and no skipped stages.
+**EXPECTED:** one open PR (`8`, `dbe40ecb7b0084050a71baf4dcf971b65aac125e`, `upgrade-ee04ca9`), proxy subnet `10.10.0.0/16`, and no Ghost resource output. Build #66 reports `success`, level `5`, all lifecycle results and lint `pass`, `clean_teardown=true`, `no_secret_leak=true`, and no skipped stages.
-**WHERE:** Gitea issue comments `16382` and `16383`; Drone URL `https://drone.ci.autonomic.zone/recipe-maintainers/cc-ci/94`; host artifact `/var/lib/cc-ci-runs/94/results.json`; Adversary evidence in `REVIEW-ghost.md`.
+**WHERE:** Gitea issue comments `16297` and `16298`; Drone URL `https://drone.ci.autonomic.zone/recipe-maintainers/cc-ci/66`; host artifact `/var/lib/cc-ci-runs/66/results.json`; Adversary evidence in `REVIEW-ghost.md`.
 ---
@@ -93,15 +93,9 @@ def assert_serving(domain: str, meta) -> None:
    Steps 1–2 are BOUNDED POLLS (no bare sleep), so a state-mutating op (upgrade/restore) that leaves
    the app briefly reconverging settles, while a persistent failure still fails within the timeout."""
    deadline = time.time() + meta.DEPLOY_TIMEOUT
-    converged = False
+    while time.time() < deadline and not lifecycle.services_converged(domain):
    while time.time() < deadline:
        # Keep both observations inside the retry budget: a service may restart
        # between the initial convergence sample and its confirmation.
        if lifecycle.services_converged(domain) and lifecycle.services_converged(domain):
            converged = True
            break
        time.sleep(5)
-    assert converged, f"{domain}: services did not converge"
+    assert lifecycle.services_converged(domain), f"{domain}: services did not converge"
    path = meta.HEALTH_PATH
    ok = tuple(meta.HEALTH_OK)
@@ -487,10 +487,6 @@ def deploy_app(
    # may already include the environmental compose.ccci.yml). Stripped before the head redeploy
    # (generic.perform_upgrade) so the PR head runs unmodified.
    if apply_previous:
        # This untracked overlay also trips abra's pinned-deploy clean-tree gate.
        # The exact version was checked out above; chaos deploys that base plus
        # its repair, just as for compose.ccci.yml, without fetching latest.
        chaos = True
        cf = compose_file_add(extra.get("COMPOSE_FILE", "compose.yml"), PREVIOUS_COMPOSE)
        abra.env_set(domain, "COMPOSE_FILE", cf)
        provide_previous_overlay(recipe)
@@ -1 +0,0 @@
 0.4.0+v5.4.1
@@ -1,14 +0,0 @@
 # The published base points at the removed docker.io/minio/minio image.
 # Keep its /data layout, credentials and bucket initialization; run the same
 # available MinIO build used by current recipes without Bitnami's wrapper.
 services:
  minio:
    image: docker.io/bitnamilegacy/minio:2025.7.23-debian-12-r5@sha256:6dabb4a2088c9a79908de3bc05f4586c23ad2182c8908e7e3acbf61c1467fb20
    user: "0:0"
    environment:
      MC_CONFIG_DIR: /tmp/mc-base-init
    entrypoint:
      - /bin/sh
      - -c
      - /minio-initialize.sh & exec "$$@"
      - --
@@ -1 +0,0 @@
 0.12.0+v0.21.0
@@ -1,14 +0,0 @@
 # The published base points at the removed docker.io/minio/minio image.
 # Keep its /data layout, credentials and bucket initialization; run the same
 # available MinIO build used by current recipes without Bitnami's wrapper.
 services:
  minio:
    image: docker.io/bitnamilegacy/minio:2025.7.23-debian-12-r5@sha256:6dabb4a2088c9a79908de3bc05f4586c23ad2182c8908e7e3acbf61c1467fb20
    user: "0:0"
    environment:
      MC_CONFIG_DIR: /tmp/mc-base-init
    entrypoint:
      - /bin/sh
      - -c
      - /minio-initialize.sh & exec "$$@"
      - --
@@ -59,6 +59,13 @@ def EXTRA_ENV(ctx):
        # postgres, redis, collabora ~1GB, onlyoffice ~2GB). Bump so abra waits long enough for
        # convergence; kept under DEPLOY_TIMEOUT (1800) so Python never kills abra mid-wait.
        "TIMEOUT": "1500",
        # abra's pinned-version `app new` regenerates the .env from generation hints and DROPS
        # SECRET_COLLABORA_P_VERSION (not a generatable secret — the .env.sample ships its
        # version line manually). Without it every secret-touching abra command for the app
        # FATAs "missing version for secret? (collabora_p)" → the base deploy goes RED at
        # install. env_set runs after app_new, so re-asserting the shipped version pin here
        # restores the sample's contract.
        "SECRET_COLLABORA_P_VERSION": "v1",
    }
@@ -171,6 +171,22 @@ def test_extra_env_dict_and_callable_forms(tmp_path):
    assert meta_mod.extra_env(meta2, ctx2) == {}  # unset EXTRA_ENV resolves to {}
 TESTS_DIR = os.path.join(os.path.dirname(__file__), "..")
 def test_lasuite_drive_extra_env_carries_collabora_p_version():
    """(REGRESSION, 2026-10-25) abra's pinned-version `app new` regenerates the .env from
    generation hints and drops SECRET_COLLABORA_P_VERSION (the .env.sample ships it manually),
    so every secret-touching abra command for the app FATAs "missing version for secret?
    (collabora_p)" and the cold base deploy goes RED at install. The per-recipe EXTRA_ENV must
    re-assert the shipped version pin (env_set runs after app_new). Pin it here so abra/upstream
    drift that loses it again surfaces as a red unit test, not a weekly red build."""
    meta = meta_mod.load("lasuite-drive", tests_dir=TESTS_DIR)
    ctx = meta_mod.hook_ctx("lasuite-drive-pr0-abc123.ci.commoninternet.net", meta)
    extra = meta_mod.extra_env(meta, ctx)
    assert extra.get("SECRET_COLLABORA_P_VERSION") == "v1"
 # ---- P3: uniform ctx hook convention -------------------------------------------------------------
@@ -116,37 +116,3 @@ def test_previous_status_stale_without_marker(monkeypatch):
    monkeypatch.setattr(lifecycle, "previous_target_version", lambda r: None)
    st = lifecycle.previous_status("rx", "version", "0.7.0+3.3.1")
    assert st["apply"] is False and st["stale"] is True and "marker" in st["reason"]
 def test_previous_overlay_deploys_checked_out_base_without_clean_tree_gate(monkeypatch):
    events = []
    for name in (
        "acquire_app_lock",
        "provide_ccci_overlay",
        "provide_previous_overlay",
        "prepull_images",
    ):
        monkeypatch.setattr(lifecycle, name, lambda *a, **kw: None)
    for name in ("app_config_remove", "app_new", "env_set", "secret_generate"):
        monkeypatch.setattr(lifecycle.abra, name, lambda *a, **kw: None)
    monkeypatch.setattr(lifecycle.meta_mod, "extra_env", lambda *a: {})
    monkeypatch.setattr(lifecycle.meta_mod, "hook_ctx", lambda *a: None)
    monkeypatch.setattr(lifecycle.abra, "has_lightweight_version_tags", lambda r: False)
    monkeypatch.setattr(lifecycle, "has_ccci_overlay", lambda r: False)
    monkeypatch.setattr(
        lifecycle.abra, "recipe_checkout", lambda r, v: events.append(("checkout", v))
    )
    monkeypatch.setattr(
        lifecycle.abra, "deploy", lambda d, **kw: events.append(("deploy", kw["chaos"]))
    )
    for apply_previous in (False, True):
        events.clear()
        lifecycle.deploy_app(
            "example",
            "example.invalid",
            version="1.0.0",
            apply_previous=apply_previous,
            meta=object(),
            _count_deploy=False,
        )
        assert events == [("checkout", "1.0.0"), ("deploy", apply_previous)]
@@ -1,29 +0,0 @@
 """Convergence confirmation must retry a transient second-sample failure."""
 from types import SimpleNamespace
 import pytest
 from harness import generic
 def _setup(monkeypatch, samples):
    clock = [0]
    monkeypatch.setattr(generic.time, "time", lambda: clock[0])
    monkeypatch.setattr(generic.time, "sleep", lambda n: clock.__setitem__(0, clock[0] + n))
    monkeypatch.setattr(generic.lifecycle, "services_converged", lambda d: next(samples))
    monkeypatch.setattr(generic.lifecycle, "http_fetch", lambda *a: (200, "application"))
    monkeypatch.setattr(generic, "served_cert", lambda d: (True, "*.ci.autonomic.zone"))
    return SimpleNamespace(DEPLOY_TIMEOUT=20, HTTP_TIMEOUT=20, HEALTH_PATH="/", HEALTH_OK=(200,))
 def test_confirmation_failure_retries_until_services_settle(monkeypatch):
    meta = _setup(monkeypatch, iter([True, False, True, True]))
    generic.assert_serving("example.ci.autonomic.zone", meta)
 def test_persistent_confirmation_failure_still_fails(monkeypatch):
    import itertools
    meta = _setup(monkeypatch, itertools.cycle([True, False]))
    with pytest.raises(AssertionError, match="services did not converge"):
        generic.assert_serving("example.ci.autonomic.zone", meta)