From b17b6f12321b3e65cc4945dbcd056f8a729c2073 Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Thu, 11 Jun 2026 21:03:51 +0000 Subject: [PATCH] =?UTF-8?q?claim(mailu):=20M2=20=E2=80=94=20DEFERRED=20clo?= =?UTF-8?q?sed;=20PARITY.md=20updated=20with=20dual-volume=20evidence;=20o?= =?UTF-8?q?perator=20summary=20written;=20PR#3=20open=20for=20merge;=20awa?= =?UTF-8?q?iting=20Adversary=20fresh=20re-trigger?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- machine-docs/DEFERRED.md | 7 ++ machine-docs/STATUS-mailu.md | 149 +++++++++++++++++++---------------- tests/mailu/PARITY.md | 18 ++++- 3 files changed, 100 insertions(+), 74 deletions(-) diff --git a/machine-docs/DEFERRED.md b/machine-docs/DEFERRED.md index 1f61955..b2d1748 100644 --- a/machine-docs/DEFERRED.md +++ b/machine-docs/DEFERRED.md @@ -287,6 +287,13 @@ before the build is called done) — but does **not** force closure. - **Linked IDEA / BACKLOG:** Q4.6. ### 2026-05-29 — mailu: no backup config (P4 N/A) — recipe-PR to add backupbot +- [x] **CLOSED @2026-06-11 (phase mailu, Builder):** Mirror PR#3 (`add-backupbot-labels`, head + `edc0201a79d3`) on `git.autonomic.zone/recipe-maintainers/mailu` adds backupbot v2 labels to + `admin` service (`/data` SQLite) and `imap` service (`/mail` Maildir). Full lifecycle at PR head + = LEVEL 5 (drone build #477): install/upgrade/backup/restore/functional all PASS; both + `/data` (SQLite) and `/mail` (Maildir) seeded + wiped + verified restored. Adversary M1 PASS + @2026-06-11T21:00Z. PR left open for operator merge. mailu's backup rung is now earned + (`backup_capable=True`), not skipped. Phase mailu M1 PASS; M2 claim in progress. - [x] **RE-ENTERED @2026-06-11:** operator approved the backupbot recipe-PR route — executing as phase `mailu` (cc-ci-plan/plan-phase-mailu-backup.md). - [ ] **What:** mailu (Q4.9) ships **no `backupbot.backup` label** on any service, so cc-ci's backup/restore tiers cleanly SKIP (`backup_capable=False`) — P4 (backup data-integrity) is N/A diff --git a/machine-docs/STATUS-mailu.md b/machine-docs/STATUS-mailu.md index 8caf531..5c05c7e 100644 --- a/machine-docs/STATUS-mailu.md +++ b/machine-docs/STATUS-mailu.md @@ -8,21 +8,16 @@ ## Current state -**Gate M1: CLAIMED (re-claim after ADV-mailu-01 fix) — awaiting Adversary** +**Gate M1: PASS** (Adversary verified @2026-06-11T21:00Z — see REVIEW-mailu.md) -Drone build #477: LEVEL 5 PASS at PR#3 head (edc0201a79d3), all rungs green including -backup/restore on real seeded mail data — BOTH `/data` (SQLite) AND `/mail` (Maildir) now -specifically tested and verified. Claimed 2026-06-11T21:XX. +**Gate M2: CLAIMED — awaiting Adversary** -ADV-mailu-01 fix: extended `ops.py::pre_backup` to inject a real mail message; extended -`ops.py::pre_restore` to also wipe the Maildir; added `test_backup_captures_mail_message` and -`test_restore_returns_mail_message` to verify both volumes survive the cycle. - -**Gate M2:** NOT YET CLAIMED +All M2 prep complete: DEFERRED closed, PARITY.md updated with full dual-volume coverage, +operator summary below. Adversary to fresh-re-trigger `!testme` on PR#3 for M2 cold pass. --- -## DoD tracker (M1) +## DoD tracker (M1) — COMPLETE - [x] Data-layout research documented (which volumes hold durable state, justification in PR desc) - [x] Recipe-mirror PR open with backupbot v2 labels (admin `/data` + imap `/mail`) @@ -31,82 +26,96 @@ ADV-mailu-01 fix: extended `ops.py::pre_backup` to inject a real mail message; e - Version bump: `3.0.1+2024.06.52` → `3.0.2+2024.06.52` - Adds `deploy.labels: {backupbot.backup: "true", backupbot.backup.path: "/data"}` to `admin` - Adds `deploy.labels: {backupbot.backup: "true", backupbot.backup.path: "/mail"}` to `imap` -- [x] Version label bumped in compose.yml (3.0.1 → 3.0.2+2024.06.52) -- [x] cc-ci: `tests/mailu/ops.py` with pre_backup (seed mailbox + inject mail message) + pre_restore (delete mailbox + wipe Maildir) -- [x] cc-ci: `tests/mailu/test_backup.py` asserting mailbox AND mail message present at backup time -- [x] cc-ci: `tests/mailu/test_restore.py` asserting mailbox AND mail message restored after restore -- [x] cc-ci: `tests/mailu/PARITY.md` updated (P4 now covered, not N/A) -- [x] Full lifecycle green at PR head (L5) including backup/restore on BOTH volumes — via drone `!testme` - - **Drone build #477**: LEVEL 5 of 5 — all rungs PASS (install/upgrade/backup/restore/custom/lint) - - `test_backup_captures_mailbox` PASS — `citest@` present in config-export at backup time - - `test_backup_captures_mail_message` PASS — message `ccci-backup-probe` in INBOX at backup time - - `test_restore_returns_mailbox` PASS — `citest@` restored after pre_restore deletion - - `test_restore_returns_mail_message` PASS — `ccci-backup-probe` message back after Maildir wipe+restore +- [x] cc-ci: `tests/mailu/ops.py` — pre_backup seeds account + injects mail message; pre_restore wipes both sqlite record AND Maildir +- [x] cc-ci: `tests/mailu/test_backup.py` — two tests: mailbox + mail message present at backup time +- [x] cc-ci: `tests/mailu/test_restore.py` — two tests: mailbox + mail message restored after restore +- [x] cc-ci: `tests/mailu/PARITY.md` updated (P4 COVERED with dual-volume evidence) +- [x] Drone build #477: LEVEL 5 PASS at PR head — all rungs including backup/restore on both volumes + - `test_backup_captures_mailbox` PASS — SQLite `/data` covered + - `test_backup_captures_mail_message` PASS — Maildir `/mail` covered + - `test_restore_returns_mailbox` PASS — SQLite `/data` restored + - `test_restore_returns_mail_message` PASS — Maildir `/mail` restored - `clean_teardown: true`, `no_secret_leak: true` - - No mailu stacks on host post-run (`docker stack ls` confirms) -- [x] ADV-mailu-01 fix applied: seed now covers both volumes (`/data` SQLite + `/mail` Maildir) -- [x] Before/after level recorded - - **BEFORE** (main, no labels): `backup_capable=False` → backup rung = intentional-skip → max **L4** - - **AFTER** (PR#3 head edc0201a): `backup_capable=True` (auto-detected from labels) → backup rung = PASS → **L5** +- [x] Before/after: BEFORE = L4 (backup intentional-skip); AFTER = L5 (earned) +- [x] M1 Adversary PASS @2026-06-11T21:00Z; ADV-mailu-01 closed -## DoD tracker (M2) +## DoD tracker (M2) — IN PROGRESS -- [ ] Fresh Adversary cold pass (independent re-trigger at PR#3 head) -- [ ] Levels reconciled -- [ ] DEFERRED entry closed -- [ ] STATUS-mailu.md operator summary -- [ ] REVIEW-mailu.md shows PASS for M1 + M2 (within 24h) +- [x] DEFERRED entry closed (DEFERRED.md — mailu entry marked CLOSED @2026-06-11 with PR+run pointers) +- [x] Levels reconciled (PARITY.md updated; before=L4-skip, after=L5-earned, proven in builds #473/#477) +- [x] Operator summary written (this STATUS-mailu.md — see below) +- [ ] Fresh Adversary cold pass (independent re-trigger at PR#3 head, restore integrity re-checked) +- [ ] REVIEW-mailu.md shows M2 PASS (within 24h of M1) --- -## Verification recipe (for Adversary M1 re-check) +## Verification recipe (for Adversary M2 check) ```bash -# 1. Verify backupbot v2 labels in PR#3 compose.yml (branch: add-backupbot-labels) +# 1. Verify PR#3 is still open and unmerged, head commit unchanged GITEA_PASSWORD=$(grep GITEA_PASSWORD /srv/cc-ci/.testenv | cut -d= -f2-) -curl -s "https://git.autonomic.zone/api/v1/repos/recipe-maintainers/mailu/contents/compose.yml?ref=add-backupbot-labels" \ - -u "autonomic-bot:${GITEA_PASSWORD}" \ - | python3 -c "import sys,json,base64; print(base64.b64decode(json.load(sys.stdin)['content']).decode())" \ - | grep -A4 backupbot -# Expected: -# admin service → backupbot.backup: "true" + backupbot.backup.path: "/data" -# imap service → backupbot.backup: "true" + backupbot.backup.path: "/mail" - -# 2. Verify PR#3 head commit -# Expected: edc0201a79d36bc87696b0f93f1ee88ad7bd10ed - -# 3. Verify drone build #477 level 5 -DRONE_TOKEN=$(ssh cc-ci 'cat /run/secrets/bridge_drone_token') -curl -s "https://drone.ci.commoninternet.net/api/repos/recipe-maintainers/cc-ci/builds/477" \ - -H "Authorization: Bearer ${DRONE_TOKEN}" | python3 -c " -import sys,json; b=json.load(sys.stdin) -print('status:', b['status']) -print('steps:', [(s['name'], s['status']) for st in b['stages'] for s in st['steps']]) +curl -s "https://git.autonomic.zone/api/v1/repos/recipe-maintainers/mailu/pulls/3" \ + -u "autonomic-bot:${GITEA_PASSWORD}" | python3 -c " +import sys,json; pr=json.load(sys.stdin) +print('state:', pr['state']) +print('head sha:', pr['head']['sha']) +print('merged:', pr.get('merged', False)) " -# Expected: success, clone+ci both success +# Expected: state=open, head sha=edc0201a79d36bc87696b0f93f1ee88ad7bd10ed, merged=False -# 4. Verify full results.json — including NEW maildir tests -ssh cc-ci 'cat /var/lib/cc-ci-runs/477/results.json' -# Expected: -# level=5, all rungs pass (backup_restore, functional, install, lint, upgrade) -# backup stage tests include: -# test_backup_captures_mailbox PASS (SQLite /data) -# test_backup_captures_mail_message PASS (Maildir /mail) -# restore stage tests include: -# test_restore_returns_mailbox PASS (SQLite /data) -# test_restore_returns_mail_message PASS (Maildir /mail) -# flags: clean_teardown=true, no_secret_leak=true +# 2. Re-trigger via !testme on PR#3 (Adversary does this independently) +# Expected: new drone build reaches LEVEL 5, all backup/restore tests PASS -# 5. Verify clean teardown on host -ssh cc-ci 'docker stack ls | grep -i mailu || echo "no mailu stacks (clean)"' -# Expected: no mailu stacks +# 3. Verify DEFERRED.md mailu entry is closed +grep -A3 "2026-05-29 — mailu" /srv/cc-ci/cc-ci-adv/machine-docs/DEFERRED.md +# Expected: [x] CLOSED @2026-06-11 with PR#3 + build #477 pointer -# 6. Re-trigger to verify at current PR head (M2 requirement): -# Comment !testme on PR#3 as the Adversary and observe level 5 again +# 4. Verify PARITY.md updated with full dual-volume coverage +cat /srv/cc-ci/cc-ci-adv/tests/mailu/PARITY.md | grep -A20 "Backup data-integrity" +# Expected: mentions both /data (SQLite) and /mail (Maildir), both volumes seeded+wiped+verified + +# 5. Confirm levels: before=L4, after=L5 +# BEFORE: git.autonomic.zone/recipe-maintainers/mailu main — no backupbot labels → backup_capable=False → skip → L4 +# AFTER: PR#3 head edc0201a79d3 — backupbot labels present → backup_capable=True → L5 (all rungs earned) ``` --- +## Operator summary (for handoff) + +### What this phase delivered + +**PR#3 on `git.autonomic.zone/recipe-maintainers/mailu`** (branch `add-backupbot-labels`, +head `edc0201a79d36bc87696b0f93f1ee88ad7bd10ed`) — **open, awaiting operator merge.** + +**What the PR adds:** +- Backupbot v2 labels on `admin` service: `backupbot.backup: "true"` + `backupbot.backup.path: "/data"` + — backs up the SQLite database at `/data` (all accounts, mailboxes, domains, DKIM config) +- Backupbot v2 labels on `imap` service: `backupbot.backup: "true"` + `backupbot.backup.path: "/mail"` + — backs up the Maildir at `/mail` (all stored messages for all users) +- Version bump: `3.0.1+2024.06.52` → `3.0.2+2024.06.52` (recipe version convention) +- No other compose changes; minimal diff + +**What CI proved at PR head (drone build #477):** +- Install ✅ — fresh deploy of mailu at PR version +- Upgrade ✅ — previous published version → PR head, reconverges +- Backup ✅ — creates a mailbox + injects a real mail message; backup snapshot taken; both present at backup time +- Restore ✅ — wipes both the sqlite account record AND the Maildir; restore brings back both the account AND the stored message +- Functional ✅ — health check, mail flow (send/receive via postfix→dovecot), mailbox create+read +- Lint ✅ — abra recipe lint passes +- Clean teardown, no secret leak + +**Before/after:** +- BEFORE (main, no labels): `backup_capable=False` → backup rung = intentional skip → max **L4** +- AFTER (PR#3 head): `backup_capable=True` (auto-detected from backupbot labels) → backup rung earned → **L5** + +**To act:** merge PR#3 on `recipe-maintainers/mailu`. After merge, mailu will earn L5 on main +(`!testme` against main should hit L5 once the recipe is published with the new version). + +No cc-ci config changes are needed post-merge — the harness auto-detects `backup_capable` from the labels. + +--- + ## Blocked items (none) @@ -115,4 +124,4 @@ ssh cc-ci 'docker stack ls | grep -i mailu || echo "no mailu stacks (clean)"' ## DONE -Not yet. Written here only when all DoD items have M1+M2 Adversary PASS in REVIEW-mailu.md. +Not yet. Written here only when M1+M2 Adversary PASS appear in REVIEW-mailu.md. diff --git a/tests/mailu/PARITY.md b/tests/mailu/PARITY.md index dfb8a84..63de667 100644 --- a/tests/mailu/PARITY.md +++ b/tests/mailu/PARITY.md @@ -36,10 +36,20 @@ service (`/data` — sqlite DB) and `imap` service (`/mail` — Maildir), making at the PR head. cc-ci tests (all in `tests/mailu/`): -- `ops.py` — `pre_backup`: seeds `citest@` via `flask mailu user`; `pre_restore`: deletes - it via sqlite3 Python to simulate data loss. -- `test_backup.py` — asserts `citest@` is in `config-export` at backup time. -- `test_restore.py` — asserts `citest@` is back in `config-export` after restore. +- `ops.py` — `pre_backup`: seeds `citest@` via `flask mailu user` + injects a uniquely-tagged + message (`ccci-backup-probe`) into the INBOX via `sendmail` in the smtp container → verified with + `doveadm search` in the imap container. `pre_restore`: (1) deletes the account from sqlite via + Python; (2) `rm -rf /mail//citest` in the imap container to wipe the Maildir — both + volumes are wiped independently so each must be restored to pass. +- `test_backup.py` — two tests: `test_backup_captures_mailbox` asserts `citest@` is in + `config-export` at backup time (SQLite `/data`); `test_backup_captures_mail_message` asserts the + `ccci-backup-probe` message is in INBOX (Maildir `/mail`). +- `test_restore.py` — two tests: `test_restore_returns_mailbox` asserts the account is back in + `config-export` after restore (SQLite `/data`); `test_restore_returns_mail_message` asserts the + `ccci-backup-probe` message is back in INBOX (Maildir `/mail`). + +Both volumes (`/data` SQLite + `/mail` Maildir) are specifically seeded, independently wiped, and +verified restored — proven in drone build #477 (all four backup/restore tests PASS, L5). Auto-detected: `generic.backup_capable()` scans the compose.yml for `backupbot.backup.*true` and returns `True` at the PR head — no `BACKUP_CAPABLE` override needed in `recipe_meta.py`.