diff --git a/machine-docs/JOURNAL-cfold.md b/machine-docs/JOURNAL-cfold.md index da59884..5348215 100644 --- a/machine-docs/JOURNAL-cfold.md +++ b/machine-docs/JOURNAL-cfold.md @@ -248,4 +248,98 @@ Conclusion: unchanged, the recipe artifact still carries the expected `compose.ccci.yml` file, and the failure remains in the live upgrade path rather than discovery/custom-test coverage. - Net: cfold remains blocked on a cfold-neutral Ghost upgrade regression / flake. No repo-local code - change was justified by this audit alone. + change was justified by that audit alone. + +## 2026-06-13 — Ghost PR #3 fresh probe after reopen: same upgrade-only failure, plus duplicate trigger signal + +I looked for the smallest allowed M2 step that did not touch recipe code: reuse an existing Ghost PR head +that had historically gone green and rerun it through the live `!testme` path. + +Actions taken: + +```bash +$ set -a && . /srv/cc-ci/.testenv && set +a +$ curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" -X PATCH \ + -H 'Content-Type: application/json' \ + -d '{"state":"open"}' \ + "https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/pulls/3" +# PR #3 reopened; head remains 720faa0bebc46a34857b2933df1924ccabbd4087 + +$ curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" -X POST \ + -H 'Content-Type: application/json' \ + -d '{"body":"!testme"}' \ + "https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/issues/3/comments" +# comment 14497 created at 2026-06-13T00:07:50Z +``` + +Fresh live outcomes: + +```bash +$ ssh cc-ci 'jq "{run_id, pr, recipe, ref, level, results, stages: (.stages | map({name,status,summary}))}" /var/lib/cc-ci-runs/568/results.json' +{ + "run_id": "568", + "pr": "3", + "recipe": "ghost", + "ref": "720faa0bebc4", + "level": 1, + "results": { + "backup": "pass", + "custom": "pass", + "install": "pass", + "restore": "pass", + "upgrade": "fail" + }, + "stages": [ + {"name": "install", "status": "pass", "summary": null}, + {"name": "backup", "status": "pass", "summary": null}, + {"name": "restore", "status": "pass", "summary": null}, + {"name": "custom", "status": "pass", "summary": null}, + {"name": "lint", "status": "pass", "summary": null} + ] +} + +$ ssh cc-ci 'jq "{run_id, pr, recipe, ref, level, finished, results, stages: (.stages | map({name,status}))}" /var/lib/cc-ci-runs/569/results.json' +{ + "run_id": "569", + "pr": "3", + "recipe": "ghost", + "ref": "720faa0bebc4", + "level": 1, + "finished": 1781309502.5494862, + "results": { + "backup": "pass", + "custom": "pass", + "install": "pass", + "restore": "pass", + "upgrade": "fail" + }, + "stages": [ + {"name": "install", "status": "pass"}, + {"name": "backup", "status": "pass"}, + {"name": "restore", "status": "pass"}, + {"name": "custom", "status": "pass"}, + {"name": "lint", "status": "pass"} + ] +} +``` + +Comment-stream evidence for duplicate triggers from one `!testme`: + +```bash +$ curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" \ + "https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/issues/3/comments?limit=20" +# ... +# 14497: !testme (2026-06-13T00:07:50Z) +# 14498: cc-ci failure comment for run 568 (2026-06-13T00:08:05Z) +# 14499: cc-ci in-progress comment for run 569 (2026-06-13T00:08:05Z) +# 14500: cc-ci in-progress comment for run 570 (2026-06-13T00:08:05Z) +``` + +Takeaways: + +- Ghost is now freshly red post-cfold on three distinct PR heads (`720faa0b`, `d88f5801`, `d42d0f7c`), all + with the same upgrade-only failure shape while custom discovery stays green. +- That further weakens any cfold-local explanation; the blocker remains in Ghost's live upgrade path. +- There is also likely a separate trigger dedupe problem: one `!testme` comment spawned runs `568`, `569`, + and `570`. I did not broaden into a D1 investigation in this loop step because cfold M2 is already + hard-blocked by Ghost's repeated upgrade failures, but the evidence is now recorded. diff --git a/machine-docs/STATUS-cfold.md b/machine-docs/STATUS-cfold.md index 1a74af4..f328104 100644 --- a/machine-docs/STATUS-cfold.md +++ b/machine-docs/STATUS-cfold.md @@ -58,8 +58,10 @@ Adversary verdict: Current work item: - full real-CI `!testme` sweep evidence is mostly assembled; one recipe (`ghost`) remains non-green for a cfold-neutral upgrade regression on the recipe/environment side -- fresh artifact audit confirms historical build `185` passed on the same Ghost ref now failing in - probe build `559`, so no cfold-local code fix has been identified in this repo yet +- fresh follow-up probes now show the Ghost upgrade failure is not confined to PR #4 / PR #5: a reopened + PR #3 at ref `720faa0b` also re-failed twice post-cfold (`568`, `569`) with the same shape +- one fresh `!testme` comment on Ghost PR #3 also spawned multiple runs (`568`, `569`, `570`), so there + is likely a duplicate-trigger side issue to investigate separately from cfold itself ### M2 baseline matrix (built from live PR heads + fresh post-cfold evidence) @@ -71,7 +73,7 @@ Current work item: | custom-html-tiny | PR #7 `526502ba` | 5 | 1 | build `510` -> L5 | | discourse | PR #2 `b7d8a244` | 5 | 3 | build `521` -> L5 | | drone | PR #1 `049438e1` | 5 | 1 | build `506` -> L5 | -| ghost | PR #4 `d88f5801` | 5 | 4 | build `557` -> L1 (upgrade fail) | +| ghost | PR #3 `720faa0b` | 5 | 4 | build `568` -> L1 (upgrade fail) | | hedgedoc | PR #1 `441c411c` | 5 | 2 | build `555` -> L5 | | immich | PR #2 `17f1649c` | 5 | 3 | build `522` -> L5 | | keycloak | PR #3 `bfe0d16f` | 5 | 3 | build `553` -> L5 | @@ -90,17 +92,21 @@ Current work item: `ghost` is the only recipe still preventing an M2 claim. -- Current upgrade PR head `d88f5801` (PR #4) failed again in build `557`: install/backup/restore/custom all - passed, but the upgrade op paused and rolled back before any upgrade-tier assertions could run. -- To isolate cfold from recipe-head drift, a separate probe PR #5 was opened at Ghost's last known-green - PR head `d42d0f7c` (historical build `185` = L4/L5-equivalent on 2026-06-05). That probe re-failed in - build `559` with the same upgrade pause: install/backup/restore/custom all passed, upgrade failed. -- Focused artifact audit after build `559` confirmed the same-ref comparison explicitly: +- Current upgrade PR heads and fresh post-cfold outcomes are all red with the same stage shape: + - PR #3 `720faa0b`: builds `568` and `569` -> L1; install/backup/restore/custom/lint pass, upgrade fail + - PR #4 `d88f5801`: build `557` -> L1; install/backup/restore/custom pass, upgrade fail + - PR #5 `d42d0f7c`: build `559` -> L1; install/backup/restore/custom/lint pass, upgrade fail +- Focused artifact audit still confirms the strongest same-ref comparison explicitly: historical build `185` (`d42d0f7c7cf9`) had `upgrade=pass`, while fresh build `559` on that same ref has `upgrade=fail` with the canonical `custom` stage still green. +- The fresh PR #3 rerun adds a second previously-green Ghost upgrade head that now fails the same way, + so the blocker is broader than a single Ghost branch and still points away from cfold itself. +- Side observation from the PR #3 retrigger: a single `!testme` comment at `2026-06-13T00:07:50Z` spawned + three new Ghost runs (`568`, `569`, `570`). `568` and `569` are already red with the same upgrade-only + failure; `570` was still in flight at the time of this status update. - Conclusion so far: Ghost's current failure is not caused by the `custom/` folder migration; the custom - tier still discovers and passes all 4 canonical custom tests, and the regression reproduces on the - previously-green Ghost head as an upgrade convergence failure. + tier still discovers and passes all 4 canonical custom tests, and the regression reproduces across + multiple Ghost PR heads as an upgrade convergence failure. ### Fresh Adversary state