7123d8288e
Per-recipe deploy budget confirmed minimal (1 base + N_cold_deps, upgrade shares
the base in place) and enforced (DG4.1); no redundant deploy existed. All four DoD
items PASS in REVIEW-2b (edf34e3). Phase 2b complete.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
114 lines
6.7 KiB
Markdown
114 lines
6.7 KiB
Markdown
# STATUS — Phase 2b (confirm the test sequence minimizes deploys)
|
||
|
||
**Phase plan (SSOT):** `/srv/cc-ci/cc-ci-plan/plan-phase2b-test-performance.md`
|
||
**Loop state for THIS phase:** STATUS-2b / BACKLOG-2b / REVIEW-2b / JOURNAL-2b (DECISIONS.md shared).
|
||
Phase 1/1*/2/2* STATUS/BACKLOG/REVIEW files are HISTORY — not this phase's state.
|
||
|
||
## Phase
|
||
NARROWED scope (operator 2026-05-30): the only task is to **confirm the per-recipe test sequence
|
||
already uses the minimum number of deploys** (and fix it if not) **without weakening any test**.
|
||
The broad empirical-perf program is parked in IDEAS. Likely outcome (operator's expectation):
|
||
already minimal via the deploy-once / deploy-sharing design.
|
||
|
||
## Definition of Done (Phase 2b) — B1–B4, each Adversary cold-verified in REVIEW-2b
|
||
- [x] **B1 — Deploy budget documented and minimal.** PASS (REVIEW-2b @2026-05-31T05:38Z, `edf34e3`).
|
||
- [x] **B2 — Enforced, not just claimed** (deploy-count guard + RUN SUMMARY, expected reflects budget).
|
||
PASS (REVIEW-2b @2026-05-31T05:38Z).
|
||
- [x] **B3 — No test weakened to save a deploy** (coverage/isolation/teardown unchanged).
|
||
PASS (REVIEW-2b @2026-05-31T05:38Z; claim is doc-only, harness byte-identical).
|
||
- [x] **B4 — Recorded** (`docs/perf/deploys.md`). PASS (REVIEW-2b @2026-05-31T05:38Z).
|
||
|
||
## DONE
|
||
|
||
All four DoD items (B1–B4) Adversary cold-verified **PASS** in REVIEW-2b @2026-05-31T05:38Z (commit
|
||
`edf34e3`); no Phase-2b VETO. Outcome: the per-recipe test-sequence deploy budget was **already
|
||
minimal** (`1 base + N_cold_deps`, upgrade shares the base in place) and **enforced** (DG4.1); no
|
||
redundant deploy existed, so none was removed. Recorded in `docs/perf/deploys.md` + DECISIONS.md.
|
||
|
||
**Sequencing note (operator):** Phase 2b ran as a manually-kicked-off parallel loop; Phase 2 is not
|
||
yet `## DONE` (plausible Q4.7b / drone Q4.10 / Q5 remain — standing Phase-2 DONE VETO in REVIEW-2.md).
|
||
Phase-2b's DoD is independent of Phase-2 completion and is fully verified. Whether Phase-2b DONE is
|
||
acknowledged before Phase-2 DONE is an operator sequencing call, not a verification gap.
|
||
|
||
---
|
||
|
||
## Gate: 2b CLAIMED, awaiting Adversary (@2026-05-31, commit on origin/main)
|
||
|
||
**Outcome: the per-recipe deploy budget is ALREADY MINIMAL and ENFORCED. No redundant deploy found;
|
||
none removed because none existed.** This is a confirm-and-document result (no harness behavior
|
||
change). Deliverable: `docs/perf/deploys.md`.
|
||
|
||
### WHAT is claimed (the budget)
|
||
Per cold `run_recipe_ci.py` run of a recipe:
|
||
```
|
||
deploys == 1 (base) + N_cold_deps # enforced as a hard failure
|
||
```
|
||
- **1 base deploy** shared by ALL five tiers: install → upgrade → backup → restore → custom.
|
||
- **+1 per COLD declared dep**, deployed once and reused; a **live-warm** dep contributes **0**.
|
||
- The **upgrade tier adds NO deploy**: the base is deployed at the **previous published version**
|
||
when upgrade runs (`base = prev or target`), and the upgrade is an **in-place chaos redeploy** of
|
||
PR-head onto that same app — NOT counted, and the real HC1 upgrade under test.
|
||
- **backup/restore add NO deploy** (operate on the same running app).
|
||
- This is **tighter** than plan B1's nominal `1 + 1(upgrade) + N` because the base deploy *is* the
|
||
prior-version deploy — the prior-version and base deploy are the same deploy.
|
||
|
||
### HOW the Adversary can verify (from a fresh clone)
|
||
|
||
**(a) Static — only `deploy_app` increments the count, and it's called in exactly 3 sites:**
|
||
```
|
||
grep -n "_record_deploy" runner/harness/lifecycle.py # called ONLY inside deploy_app (:107, :211)
|
||
grep -rn "deploy_app(" runner/ | grep -v "def deploy_app" # 3 callers: :699 :819 (+ deps.py:100)
|
||
```
|
||
- `lifecycle.py:211` — `deploy_app` is the sole caller of `_record_deploy`.
|
||
- `run_recipe_ci.py:819` — the single base deploy (cold main path).
|
||
- `runner/harness/deps.py:100` — one per declared dep.
|
||
- `run_recipe_ci.py:699` — `promote_canonical` (WC5), which **pops** `CCCI_DEPLOY_COUNT_FILE` first
|
||
(`:697`) so it is OUTSIDE the per-run budget (post-green warm-cache maintenance, not a test deploy).
|
||
- `lifecycle.chaos_redeploy` (the upgrade, `lifecycle.py:418-435`) does **NOT** call `deploy_app`
|
||
→ not counted (docstring states this explicitly).
|
||
- `generic.perform_backup`/`perform_restore` → `backup_app`/`restore_app`: no `deploy_app` → not counted.
|
||
- Base-version selection that makes upgrade share the base deploy: `run_recipe_ci.py:746-754`
|
||
(`want_upgrade`; `prev = UPGRADE_BASE_VERSION or previous_version`; `base = prev or target`).
|
||
|
||
**(b) Enforcement — DG4.1 guard hard-fails on mismatch:**
|
||
```
|
||
sed -n '958,1010p' runner/run_recipe_ci.py
|
||
```
|
||
- `expected_deploy_count = 1 + deps_deployed_count` (`:984`); warm deps excluded (`:982-983`).
|
||
- RUN SUMMARY prints `deploy-count = N (expect M)` (`:986`).
|
||
- `if deploy_count != expected_deploy_count: … overall = 1` → non-zero exit (`:1005-1010`).
|
||
⇒ every GREEN run proves the recipe stayed within budget; a redundant redeploy turns it RED.
|
||
|
||
**(c) Dynamic (optional, cold) — re-run a no-dep and a cold-dep recipe:**
|
||
```
|
||
RECIPE=ghost STAGES=install,upgrade,backup,restore,custom cc-ci-run runner/run_recipe_ci.py
|
||
RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py
|
||
```
|
||
|
||
**(d) B3 — coverage unchanged:** confirm all five tiers still run their real generic+overlay
|
||
assertions against the shared app (`run_lifecycle_tier`, `ALL_STAGES` `run_recipe_ci.py:56`), the
|
||
upgrade is a real prev→PR-head crossover (`assert_upgraded`), and P4 backup→restore is real
|
||
data-integrity (seed→backup→mutate→restore→assert). Nothing is skipped/softened to share the deploy.
|
||
|
||
**(e) B4 — the record:** `docs/perf/deploys.md` (this deliverable).
|
||
|
||
### EXPECTED outcomes
|
||
- (a) `_record_deploy` appears only inside `deploy_app`; exactly the 3 `deploy_app` callers above.
|
||
- (b) guard present and hard-failing as quoted; `expected = 1 + cold_deps`.
|
||
- (c) ghost: `deploy-count = 1 (expect 1)`, all tiers `pass`.
|
||
lasuite-docs + cold keycloak: `deploy-count = 2 (expect 2)`, `deps deployed: ['keycloak']`,
|
||
all tiers `pass`, `DEPS teardown` clean.
|
||
- Historical corroboration (Phase 2 runs, recorded in STATUS-2/REVIEW-2): every recipe ran at
|
||
`deploy-count = 1` (no/warm dep) or `deploy-count = 2 (expect 2)` (one cold dep, lasuite-docs
|
||
Q2.4 — REVIEW-2 `:114`). No run ever exceeded `1 + N_cold_deps`.
|
||
|
||
### WHERE the inputs live
|
||
- Deliverable doc: `docs/perf/deploys.md`.
|
||
- Code: `runner/run_recipe_ci.py` (`:56`, `:746-754`, `:819`, `:958-1010`),
|
||
`runner/harness/lifecycle.py` (`:107-211`, `:418-435`), `runner/harness/deps.py` (`:81-120`),
|
||
`runner/harness/generic.py` (`perform_upgrade`/`perform_backup`/`perform_restore`).
|
||
- Commit: see `git log origin/main` for the `claim(2b)` commit.
|
||
|
||
## Gates
|
||
- Gate 2b — CLAIMED, awaiting Adversary PASS in REVIEW-2b.
|