Compare commits

...
Author SHA1 Message Date
autonomic-bot 521cd01c1c fix(lasuite-drive): re-assert SECRET_COLLABORA_P_VERSION in EXTRA_ENV
continuous-integration/drone/push Build is passing
abra's pinned-version 'app new' regenerates the app .env from generation
hints and drops SECRET_COLLABORA_P_VERSION (the .env.sample ships the line
manually — not a generatable secret). Every secret-touching abra command
for the app then FATAs 'missing version for secret? (collabora_p)' and the
cold base deploy goes RED at install (first seen in the 2026-10-25 weekly
sweep's cold tests of the new releases: lasuite-drive never got past
install_steps' oidc_rpcs insert). Re-asserting the shipped version pin via
EXTRA_ENV restores the sample's contract.

Verified live on cc-ci: abra app check passes for the pinned sample env
collabora_p only with the line present; repro via 'abra app new -S' on
0.12.0+v0.21.0 dropped the line deterministically.
2026-10-26 17:09:22 +00:00
autonomic-bot d944ccac02 review(ghost): reverify claimed gate
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2026-10-20 03:23:31 +00:00
autonomic-bot c90c2e84c8 status(ghost): complete verified phase
continuous-integration/drone/push Build is passing
2026-10-20 03:23:02 +00:00
autonomic-bot 5ba6bf230f claim(ghost): revalidate current upgrade result
continuous-integration/drone/push Build is passing
2026-10-20 03:22:30 +00:00
autonomic-bot 13d1b33014 review(ghost): pass fresh level-five retry
continuous-integration/drone/push Build is passing
2026-10-20 03:22:11 +00:00
autonomic-bot 2977495851 claim(ghost): revalidate level-five upgrade run
continuous-integration/drone/push Build is passing
2026-10-20 03:16:08 +00:00
autonomic-bot eeec9b006f review(ghost): fail stale level-4 result
continuous-integration/drone/push Build is passing
2026-10-20 03:10:22 +00:00
autonomic-bot 6d592b9417 status(ghost): record fresh PR8 pass
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2026-10-13 03:16:42 +00:00
autonomic-bot 820ded47ef review(ghost): revalidate current upgrade PR PASS
continuous-integration/drone/push Build is passing
2026-10-13 03:14:41 +00:00
autonomic-bot 35ba99768c runner: never embed a fossil screenshot from a collided run dir (#45)
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2026-10-05 17:46:25 +00:00
autonomic-bot 5cec3e0e31 gitea: fix LFS round-trip post-restart wait (stale on 28.0.0) (#44)
continuous-integration/drone/push Build is passing
2026-10-05 17:37:27 +00:00
autonomic-bot f675941c34 Merge pull request 'nix: sync root authorizedKeys with live cc-ci keys' (#43) from fix/root-authorized-keys-sync into main
continuous-integration/drone/push Build is failing
continuous-integration/drone Build is passing
2026-09-28 19:43:59 +00:00
autonomic-bot 69a64572fe nix: sync root authorizedKeys with live cc-ci keys
continuous-integration/drone/push Build is failing
The live /root/.ssh/authorized_keys on cc-ci had drifted from this
declarative list: several keys were added manually over time and would
have been wiped by the next nixos-rebuild switch.

- sync the declared list to the live file (10 keys, verified blob-for-blob)
- add the new notplants-orchestrator and nptest keys (added live first)
- drop claude-sandbox keys at operator request (incl. one unnamed live
  key identified as a sandbox key, fingerprint SHA256:Wlhj5g4IjCId1wvR
  HnxEiwVVu2N+aYa4gRsY1XfOkts) and the never-live claude@claude-vm key
- live file updated in the same change, so state is converged now
2026-09-28 19:43:45 +00:00
autonomic-bot 2f6787f043 Merge pull request 'drone: escape nix interpolation in have_secret' (#42) from fix-drone-secret-scope2 into main
continuous-integration/drone/push Build is pending
continuous-integration/drone Build is failing
2026-09-21 17:00:51 +00:00
autonomic-bot 0bf49537b8 drone: escape nix interpolation in have_secret
continuous-integration/drone/push Build is pending
2026-09-21 17:00:49 +00:00
autonomic-bot 46ad39145a Merge pull request 'drone: scope have_secret to the stack' (#41) from fix-drone-secret-scope into main
continuous-integration/drone/push Build is pending
2026-09-21 17:00:30 +00:00
autonomic-bot afe7a10758 drone: scope have_secret to the new stack — old stack's secrets satisfied the check post-rename
continuous-integration/drone/push Build is pending
2026-09-21 17:00:29 +00:00
autonomic-bot c5106bc1eb Merge pull request 'acme-dns: fix storage-seed unit nesting' (#40) from fix-seed-nesting into main
continuous-integration/drone/push Build is failing
2026-09-21 16:56:02 +00:00
autonomic-bot 8366843d89 acme-dns: fix storage-seed nesting (services. inside the module's systemd wrapper)
continuous-integration/drone/push Build is failing
2026-09-21 16:56:00 +00:00
autonomic-bot 93b86e0cc1 Merge pull request 'domain cutover: ci.autonomic.zone — cert, front doors, drone, harness naming' (#39) from domain-cutover into main
continuous-integration/drone/push Build is failing
2026-09-21 16:53:34 +00:00
autonomic-bot 542e8cfb11 domain cutover: front doors + drone + per-run naming to ci.autonomic.zone
continuous-integration/drone/push Build is failing
- acme-dns.nix: ONE dual-zone SAN cert (ci+*.ci.autonomic.zone AND ci+*.ci.commoninternet.net)
  via the same acmedns account — storage re-keyed by cc-ci-acme-storage-seed.service; handoff
  reads the new cert dir. Single secret pair => zero changes to the traefik reconciler.
- dashboard/bridge/reports: dual Host rules during the bake window (bridge gets explicit
  parentheses so && does not shadow the dashboard on the new host).
- drone abra app renamed to drone.ci.autonomic.zone (fresh DB supported: DRONE_USER_CREATE
  re-injects the sops bridge token); runner RPC + bootstrap-drone-oauth.sh follow.
- harness: app_domain() issues *.ci.autonomic.zone run domains; RUN_APP_RE / stack-name
  regexes / docker-prune accept BOTH zones during the bake. Warm stacks deliberately stay
  on the legacy zone (data-warm volumes; post-bake migration).
- URLs in bridge/dashboard defaults + recipe-report.py follow the new names.
2026-09-21 16:52:53 +00:00
autonomic-bot eb5fb82611 Merge pull request 'reports.nix: pin non-browser UA on /pr/ proxy' (#38) from fix/anubis-ua-reports-proxy into main
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is failing
2026-09-14 16:45:37 +00:00
autonomic-bot c1835c40f0 reports.nix: pin non-browser UA on /pr/ proxy — Gitea's Anubis 307-challenges browser UAs, turning every report STATUS cell into '?'
continuous-integration/drone/push Build is passing
2026-09-14 16:43:27 +00:00
autonomic-bot 0d8c6fba79 Merge pull request 'subagent model: opencode-go/deepseek-v4-flash' (#37) from config/subagents-flash into main
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
subagent model: opencode-go/deepseek-v4-flash (#37)
2026-09-08 17:28:04 +00:00
notplantsandClaude Opus 5 3acf6a86f6 subagent model: opencode-go/deepseek-v4-flash
continuous-integration/drone/push Build is passing
The operator enabled China-hosted models on the workspace, so deepseek-v4-flash
is available on the Go tier again (it was the model the weekly run used before
the host moved off ZEN, and it is the cheap/fast one for the per-recipe
subagents). Verified on the host: `opencode run --model
opencode-go/deepseek-v4-flash` answers.

The weekly run's MAIN agent moves to opencode-go/glm-5.3-flash separately, in
upgrader.env on the host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz
2026-09-08 17:28:02 +00:00
autonomic-bot d5acc945a2 journal(ghost): record current-state recheck
continuous-integration/drone/push Build is passing
2026-09-08 16:53:18 +00:00
autonomic-bot 8d29ed10d1 Merge pull request 'subagent model: opencode-go/glm-5.2 (the cc-ci host is on the Go subscription)' (#36) from config/opencode-go into main
continuous-integration/drone/push Build is passing
subagent model: opencode-go/glm-5.2 (#36)
2026-09-08 16:50:42 +00:00
30 changed files with 613 additions and 127 deletions
+2 -2
View File
@@ -41,11 +41,11 @@ from datetime import UTC, datetime
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
GITEA_API = os.environ.get("GITEA_API", "https://git.autonomic.zone/api/v1") GITEA_API = os.environ.get("GITEA_API", "https://git.autonomic.zone/api/v1")
DRONE_URL = os.environ.get("DRONE_URL", "https://drone.ci.commoninternet.net") DRONE_URL = os.environ.get("DRONE_URL", "https://drone.ci.autonomic.zone")
# Dashboard base URL — where per-run artifacts (summary card PNG, level badge SVG) are served # Dashboard base URL — where per-run artifacts (summary card PNG, level badge SVG) are served
# (Phase 3 U2.3: /runs/<run_id>/...). The PR comment (U3) embeds the card + badge from here. The # (Phase 3 U2.3: /runs/<run_id>/...). The PR comment (U3) embeds the card + badge from here. The
# run_id is the Drone build number (== `num`), so the URLs are /runs/<num>/{summary.png,badge.svg}. # run_id is the Drone build number (== `num`), so the URLs are /runs/<num>/{summary.png,badge.svg}.
DASH_URL = os.environ.get("DASH_URL", "https://ci.commoninternet.net") DASH_URL = os.environ.get("DASH_URL", "https://ci.autonomic.zone")
CI_REPO = os.environ.get("CI_REPO", "recipe-maintainers/cc-ci") CI_REPO = os.environ.get("CI_REPO", "recipe-maintainers/cc-ci")
TRIGGER = "!testme" TRIGGER = "!testme"
# Hidden HTML-comment marker embedded in the bot's PR comment so a re-`!testme` UPDATES the same # Hidden HTML-comment marker embedded in the bot's PR comment so a re-`!testme` UPDATES the same
+1 -1
View File
@@ -22,7 +22,7 @@ import urllib.error
import urllib.request import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
DRONE_URL = os.environ.get("DRONE_URL", "https://drone.ci.commoninternet.net") DRONE_URL = os.environ.get("DRONE_URL", "https://drone.ci.autonomic.zone")
CI_REPO = os.environ.get("CI_REPO", "recipe-maintainers/cc-ci") CI_REPO = os.environ.get("CI_REPO", "recipe-maintainers/cc-ci")
CACHE_TTL = int(os.environ.get("CACHE_TTL", "30")) CACHE_TTL = int(os.environ.get("CACHE_TTL", "30"))
# Per-recipe history display cap (phase dash): a long-lived recipe (plausible/custom-html have 30+ # Per-recipe history display cap (phase dash): a long-lived recipe (plausible/custom-html have 30+
+6
View File
@@ -9,9 +9,15 @@
- [x] Verify no Ghost resources leak after the retry — no `ghos-*`/`dev-ghost` stacks, services, or volumes - [x] Verify no Ghost resources leak after the retry — no `ghos-*`/`dev-ghost` stacks, services, or volumes
- [x] M1 current evidence Adversary-verified — PASS @2026-09-07T21:29:58Z - [x] M1 current evidence Adversary-verified — PASS @2026-09-07T21:29:58Z
- [x] M2 current operator-ready outcome Adversary-verified — PASS @2026-09-07T21:29:58Z - [x] M2 current operator-ready outcome Adversary-verified — PASS @2026-09-07T21:29:58Z
- [x] Fresh current PR#8 post-proxy retry — comment 16237, build #50, level 5/5
- [x] Fresh PR#8 cleanup and duplicate inventory — exactly one open PR, no Ghost resources
- [x] Fresh M1/M2 revalidation Adversary-verified — PASS @2026-10-13T03:14:20Z
- [x] Fresh PR#8 retry after level-4 lint uncertainty — comment 16297, build #66, level 5/5
- [x] Fresh M1/M2 revalidation for build #66 — Adversary PASS @2026-10-20T03:22:00Z; operator classification comment #16299 added
## Adversary findings ## Adversary findings
- [x] [adversary] **[A1] Build #585 must NOT be used as the "clean post-proxy pass"** — it ran pre-proxy (03:59Z vs proxy fix at 05:38Z) and tested PR#5 (cfold probe), not PR#4. A genuine post-proxy !testme on PR#4 is required for M1. @2026-06-13T06:22Z — **CLOSED: Builder used build #612 (post-proxy, 06:13Z), not #585. M1 PASS @06:38Z** - [x] [adversary] **[A1] Build #585 must NOT be used as the "clean post-proxy pass"** — it ran pre-proxy (03:59Z vs proxy fix at 05:38Z) and tested PR#5 (cfold probe), not PR#4. A genuine post-proxy !testme on PR#4 is required for M1. @2026-06-13T06:22Z — **CLOSED: Builder used build #612 (post-proxy, 06:13Z), not #585. M1 PASS @06:38Z**
- [x] [adversary] **[A2] `update_config.monitor` is likely the root cause of upgrade timing failures** — builds #557 and #578 both failed with `UpdateStatus=paused`, NOT VIP exhaustion. @2026-06-13T06:22Z — **CLOSED: Build #612 passed post-proxy confirming infra-confound. Operator comment explains MySQL timing under load. M1+M2 PASS @06:38Z** - [x] [adversary] **[A2] `update_config.monitor` is likely the root cause of upgrade timing failures** — builds #557 and #578 both failed with `UpdateStatus=paused`, NOT VIP exhaustion. @2026-06-13T06:22Z — **CLOSED: Build #612 passed post-proxy confirming infra-confound. Operator comment explains MySQL timing under load. M1+M2 PASS @06:38Z**
- [x] [adversary] **[A3] PR#5 (cfold probe) should be closed once PR#4 has its verdict** — not the canonical upgrade. @2026-06-13T06:22Z — **CLOSED: PR#5 closed (verified). M2 PASS @06:38Z** - [x] [adversary] **[A3] PR#5 (cfold probe) should be closed once PR#4 has its verdict** — not the canonical upgrade. @2026-06-13T06:22Z — **CLOSED: PR#5 closed (verified). M2 PASS @06:38Z**
- [x] [adversary] **[A4] Current run #55 is not fully green** — cold artifact for current PR#8 head `dbe40ecb` is level 4: lifecycle/custom stages pass, but lint is `unver` (`abra recipe lint exited 2 with no table`) and is listed as an unintentional skip. Result comment #16249 says only `passed` without explaining the blocker. Repro: `ssh cc-ci 'jq -c "{recipe,pr,ref,level,results,flags,skips,lint}" /var/lib/cc-ci-runs/55/results.json'`. @2026-10-20T03:10:02Z — **CLOSED: fresh build #66 for the same current head reports level 5 with lint pass and no unintentional skips; M2 PASS @2026-10-20T03:22Z**
+80
View File
@@ -114,3 +114,83 @@ The artifact also reports the `lint` stage as pass. Gitea comment `15815` was up
result with links to build #1332. A post-run host probe produced no matching `ghos-*` or `dev-ghost` result with links to build #1332. A post-run host probe produced no matching `ghos-*` or `dev-ghost`
stack, service, or volume. `REVIEW-ghost.md` records fresh M1 and M2 PASS verdicts at stack, service, or volume. `REVIEW-ghost.md` records fresh M1 and M2 PASS verdicts at
`2026-09-07T21:29:58Z` after independent current-state verification. `2026-09-07T21:29:58Z` after independent current-state verification.
## 2026-09-08T16:53Z — Bootstrap and current-state recheck
Bootstrap checks passed: `ssh cc-ci 'hostname && whoami && nixos-version'` returned `cc-ci`,
`root`, and `26.05.20260906.c257840`; the authenticated Gitea API returned version `1.27.3`; and
the wildcard probe resolved to `195.201.88.249`. `git pull --rebase` reported an up-to-date clone.
Current Ghost inventory remains exactly one open PR: #7, head
`14575de6209ab2c4fb29d95fe454b761070cd351`, branch `upgrade-808ac05`. Closed PR inventory contains
PRs #1–#6, with PR#4 merged and PR#5/PR#6 closed. Gitea comments #15814 and #15815 contain the
fresh `!testme` and passed result for build #1332.
The authoritative host artifact `/var/lib/cc-ci-runs/1332/results.json` reports level 5 and
`install`, `upgrade`, `backup`, `restore`, `custom`, and `lint` all `pass`. The host probe reports
proxy subnet `10.10.0.0/16` and no matching Ghost stacks, services, volumes, or networks. The
unauthenticated Drone API endpoint returned HTTP 401, so no claim is based on that endpoint; the
host artifact and Gitea result comment remain the verification sources.
## 2026-10-13T03:14Z — Fresh PR#8 revalidation
Live inventory found exactly one open Ghost upgrade PR: PR#8, branch `upgrade-ee04ca9`, head
`ee04ca9eb4f1b00ab6719e6858f7876088e0ee3d`; PRs #1 through #7 are closed. The proxy network is
`10.10.0.0/16`.
Triggered `!testme` as Gitea comment `16237` at `2026-10-13T03:11:52Z`. The bridge created build
#50 and result comment `16238`. The cold artifact `/var/lib/cc-ci-runs/50/results.json` reports
level 5, `install`, `upgrade`, `backup`, `restore`, `custom`, and `lint` all passing, with
`clean_teardown=true`, `no_secret_leak=true`, and no skipped stages. The upgrade stage included
`tests.ghost.test_upgrade.test_upgrade_preserves_state`.
Host probes after completion found no `ghos-*` or `dev-ghost` stacks, services, volumes, or
networks. The Adversary independently recorded M1 and M2 PASS at `2026-10-13T03:14:20Z` in
`REVIEW-ghost.md`.
## 2026-10-20T03:12Z — Fresh retry resolved lint uncertainty
The live PR remains #8 at head `dbe40ecb7b0084050a71baf4dcf971b65aac125e`. The prior current-head
run #55 passed all lifecycle stages but had an unintentional `lint` unverifiable result because Abra
panicked while sorting a catalogue version. One fresh retry was triggered with comment #16297. It
created build #66, which completed successfully at level 5/5.
Verification commands and results:
```sh
curl -fsS https://drone.ci.autonomic.zone/api/repos/recipe-maintainers/cc-ci/builds/66
# status=success, RECIPE=ghost, PR=8, REF=dbe40ecb7b0084050a71baf4dcf971b65aac125e
ssh cc-ci 'jq -c "{recipe,pr,ref,level,results,lint,flags,skips}" /var/lib/cc-ci-runs/66/results.json'
# level=5; install/upgrade/backup/restore/custom/lint=pass; clean_teardown=true; no_secret_leak=true
ssh cc-ci 'docker stack ls; docker service ls; docker volume ls; docker network ls'
# no ghos-* or dev-ghost resources
```
The result comment #16298 links build #66 and reports passed. M1 and M2 are re-claimed in STATUS for
fresh independent verification; no recipe change was needed.
## 2026-10-20T03:24Z — Live recheck and operator classification
Bootstrap checks passed: `ssh cc-ci 'hostname && whoami'` returned `cc-ci` and `root`,
`ssh cc-ci 'nixos-version'` returned `26.05.20260906.c257840 (Yarara)`, authenticated Gitea API
version is `1.27.3`, and a random `probe-*.ci.commoninternet.net` resolved to gateway address
`195.201.88.249`.
Live Gitea inventory shows exactly one open Ghost PR: #8, `upgrade-ee04ca9`, head
`dbe40ecb7b0084050a71baf4dcf971b65aac125e`. Closed PRs include the historical upgrade/probe PRs;
none is an open duplicate. The clean post-proxy retry is comment #16297 / build #66. The host
artifact reports level 5, all lifecycle stages and lint `pass`, `clean_teardown=true`,
`no_secret_leak=true`, and no skipped stages. The proxy subnet is `10.10.0.0/16`; stack, service,
volume, and network probes produced no `ghos-*` or `dev-ghost` resources.
Posted operator-facing classification comment #16299 on PR #8. It records build #66 as the current
evidence, identifies the 2026-06-12 proxy/IPAM and concurrent-load failures as historical, and
states that the PR is operator-ready and unmerged. No recipe or test change was needed.
## 2026-10-20T03:26Z — M1/M2 passed and phase completed
`REVIEW-ghost.md` records fresh independent PASSes for M1 and M2 at 2026-10-20T03:22:00Z. The
Adversary verified the sole open PR #8, current-head build #66 at level 5/5 with lint pass and no
unintentional skips, historical failure separation, operator-facing result, and absence of Ghost
resource leaks. The prior level-4 lint VETO is explicitly cleared. STATUS-ghost.md now records
`## DONE`; no recipe PR was merged.
+99
View File
@@ -145,3 +145,102 @@ Builder may write `## DONE` to STATUS-ghost.md.
- Current Swarm cleanup probe found no Ghost-named resources, and the phase's current PR head matches the level-5 result artifact. - Current Swarm cleanup probe found no Ghost-named resources, and the phase's current PR head matches the level-5 result artifact.
**Fresh verdict:** M1 and M2 PASS. The ghost phase is operator-ready; the 2026-06-12 failures remain historical, pre-proxy/load-confounded evidence and are not evidence against the current PR. **Fresh verdict:** M1 and M2 PASS. The ghost phase is operator-ready; the 2026-06-12 failures remain historical, pre-proxy/load-confounded evidence and are not evidence against the current PR.
## Fresh revalidation
**M1: PASS @2026-10-13T03:14:20Z**
- Cold Gitea inventory shows exactly one open Ghost upgrade PR: #8, `chore: upgrade ghost to 6.69.0-alpine`, head `ee04ca9eb4f1b00ab6719e6858f7876088e0ee3d`, branch `upgrade-ee04ca9`.
- The fresh `!testme` comment is Gitea comment #16237 at 2026-10-13T03:11:52Z. Its result comment #16238 links Drone build #50 and reports pass.
- Cold Drone API check for build #50 reports `success` with parameters `RECIPE=ghost`, `PR=8`, `REF=ee04ca9eb4f1b00ab6719e6858f7876088e0ee3d`; the exec pipeline and both steps also report success.
- Cold host artifact `/var/lib/cc-ci-runs/50/results.json` matches PR #8 and the full level-5 matrix: install, upgrade, backup, restore, custom, and lint are all `pass`; `clean_teardown=true`, `no_secret_leak=true`, and no stages are skipped.
- This evidence is post-proxy and current; the 2026-06-12 failures remain historical and were not used as current recipe evidence.
**M2: PASS @2026-10-13T03:14:20Z**
- Exactly one relevant open Ghost upgrade PR remains: #8. The prior PR #7 is no longer open, and no duplicate open upgrade PR is present.
- The operator-facing result comment #16238 is green and links build #50.
- Cold Swarm probes found no `ghos-*` or `dev-ghost` stacks, services, volumes, or networks after build #50; the proxy subnet remains `10.10.0.0/16`.
**Fresh verdict:** M1 and M2 PASS. The current Ghost upgrade PR is operator-ready, the fresh post-proxy retry is green, the historical failure classification remains valid, and no Ghost resources leak.
## Fresh revalidation — 2026-10-20
### M1: PASS @2026-10-20T03:10:02Z
- Cold Gitea inventory shows exactly one open Ghost upgrade PR: #8, head
`dbe40ecb7b0084050a71baf4dcf971b65aac125e` on `upgrade-ee04ca9`.
- The current head has a fresh post-proxy `!testme` comment #16248 and result comment #16249,
linked to run #55. The run artifact identifies `recipe=ghost`, `pr=8`, and ref
`dbe40ecb7b00`.
- Run #55 independently reports install, upgrade, backup, restore, and custom as `pass`, with
`clean_teardown=true` and `no_secret_leak=true`. No Ghost stacks, services, volumes, or
networks remain; proxy subnet is `10.10.0.0/16`.
### M2: FAIL @2026-10-20T03:10:02Z
- The current run artifact is level 4, not level 5. Its lint stage is `unver`, with
`skips.unintentional=["lint"]` and detail `abra recipe lint exited 2 with no table`.
- Result comment #16249 says only `passed` and does not disclose the unverified lint stage.
The lifecycle evidence is strong, but the operator-facing outcome is not accurately explained
as required by the phase gate. The Builder must either obtain a fresh fully verified result or
add a precise operator-facing explanation of the lint blocker and its impact before M2 can pass.
## VETO CLEARED @2026-10-20T03:22:00Z
The prior veto applied to run #55. Fresh build #66 for the same PR head has verified lint
pass and an accurate result comment; see the fresh M1/M2 verdict below.
## Fresh gate verification — 2026-10-20T03:22:00Z
### M1: PASS @2026-10-20T03:22:00Z
- Cold Gitea inventory shows exactly one open Ghost upgrade PR: PR#8, head
`dbe40ecb7b0084050a71baf4dcf971b65aac125e` on `upgrade-ee04ca9`; PRs #1-#7 are closed.
- Gitea comments #16297 and #16298 show the fresh `!testme` and its result for build #66.
- Drone API build #66 independently reports `status=success`, `RECIPE=ghost`, `PR=8`,
`REF=dbe40ecb7b0084050a71baf4dcf971b65aac125e`, and `SRC=recipe-maintainers/ghost`.
- Host artifact `/var/lib/cc-ci-runs/66/results.json` reports `level=5`, with install,
upgrade, backup, restore, custom, and lint all `pass`; no unintentional skips are present.
- The current evidence is post-proxy and uses the current PR head. The historical failed
artifacts remain separate: builds #515/#517 failed at install and #519/#557 failed at
upgrade, while the proxy was subsequently fixed to `10.10.0.0/16`; they were not used as
evidence against the current retry.
### M2: PASS @2026-10-20T03:22:00Z
- Exactly one relevant open Ghost upgrade PR remains, PR#8; all historical PRs #1-#7 are closed.
- The operator-facing result comment #16298 links build #66 and reports the successful result.
- Artifact flags are `clean_teardown=true` and `no_secret_leak=true`; `docker stack ls`,
`docker service ls`, `docker volume ls`, and `docker network ls` probes found no
`ghos-*` or `dev-ghost` resources. The proxy subnet is `10.10.0.0/16`.
- The prior M2 veto is cleared: build #66 has a verified lint `pass`, unlike build #55's
unintentional lint `unver` result.
**Fresh verdict:** M1 and M2 PASS. The current Ghost upgrade PR is operator-ready, the
post-proxy retry is a genuine level-5 pass for the current head, historical failures remain
separate from current recipe evidence, and no Ghost resources leak.
## Gate re-verification — 2026-10-20T03:23:01Z
### M1: PASS @2026-10-20T03:23:01Z
- Cold live inventory still shows exactly one open Ghost PR: PR#8 at
`dbe40ecb7b0084050a71baf4dcf971b65aac125e`, branch `upgrade-ee04ca9`; PRs #1-#7 are closed.
- Drone build #66 independently reports success with `RECIPE=ghost`, `PR=8`, and the exact
current head SHA in `REF`.
- `/var/lib/cc-ci-runs/66/results.json` independently reports level 5, install/upgrade/
backup/restore/custom/lint all pass, no unintentional skips, `clean_teardown=true`, and
`no_secret_leak=true`.
- Proxy is `10.10.0.0/16`; no Ghost stacks, services, volumes, or networks are present.
- Comments #16297, #16298, and operator classification comment #16299 are present; #16299
explicitly separates historical 2026-06-12 failures from the current post-proxy evidence.
### M2: PASS @2026-10-20T03:23:01Z
- The live PR inventory and build artifact confirm exactly one operator-ready Ghost upgrade PR.
- Comment #16299 accurately reports the level-5 result, MySQL upgrade success, historical
failure classification, and no leaked Ghost resources; it states the PR was not merged.
- The previous level-4 lint veto remains cleared by the verified level-5 build #66.
**Gate verdict:** M1 and M2 PASS. No new adversary finding or veto.
+24 -19
View File
@@ -1,25 +1,28 @@
# STATUS — phase ghost (ghost upgrade re-evaluation) # STATUS — phase ghost (ghost upgrade re-evaluation)
**Updated:** 2026-09-07T21:35Z **Updated:** 2026-10-20T03:26:00Z
**Phase:** ghost **Phase:** ghost
**Builder:** autonomic-bot **Builder:** autonomic-bot
--- ---
Gate: M1 and M2 CLAIMED, awaiting Adversary. The previous M2 VETO for level-4 build #55 is
superseded by the fresh level-5 build #66 below.
## DONE ## DONE
Both M1 and M2 have fresh Adversary PASSes dated 2026-09-07T21:29:58Z. The current canonical M1 and M2 have fresh Adversary PASSes in `REVIEW-ghost.md` at 2026-10-20T03:22:00Z.
Ghost upgrade PR is #7; historical PR#4 was merged after the June completion.
| Check | Result | | Check | Result |
|---|---| |---|---|
| Current Ghost PR | PR#7, `upgrade-808ac05`, head `14575de6209ab2c4fb29d95fe454b761070cd351` | | Current Ghost PR | PR#8, `upgrade-ee04ca9`, head `dbe40ecb7b0084050a71baf4dcf971b65aac125e` |
| Fresh post-proxy retry | Gitea comment `15814`; Drone build #1332 | | Fresh post-proxy retry | Gitea comment `16297`; Drone build #66 |
| Build #1332 | success, level 5/5; install, upgrade, backup, restore, custom, and lint passed | | Build #66 | success, level 5/5; install, upgrade, backup, restore, custom, and lint passed |
| PR result comment | Gitea comment `15815` reports passed and links build #1332 | | PR result comment | Gitea comment `16298` reports passed and links build #66 |
| Operator classification comment | Gitea comment `16299` explains historical infra/load confounds and operator readiness |
| Post-run cleanup | no `ghos-*`/`dev-ghost` stacks, services, or volumes | | Post-run cleanup | no `ghos-*`/`dev-ghost` stacks, services, or volumes |
| M1 Adversary PASS | `REVIEW-ghost.md`, 2026-09-07T21:29:58Z | | M1 Adversary PASS | pending fresh verification of build #66 |
| M2 Adversary PASS | `REVIEW-ghost.md`, 2026-09-07T21:29:58Z | | M2 Adversary PASS | pending fresh verification of build #66 |
### Definition-of-Done checklist (ghost phase) ### Definition-of-Done checklist (ghost phase)
@@ -28,26 +31,26 @@ Ghost upgrade PR is #7; historical PR#4 was merged after the June completion.
- [x] 2026-06-12 failures are classified separately from current recipe evidence - [x] 2026-06-12 failures are classified separately from current recipe evidence
- [x] Historical duplicate PRs are closed; no current duplicate exists - [x] Historical duplicate PRs are closed; no current duplicate exists
- [x] No Ghost resources remain after the fresh retry - [x] No Ghost resources remain after the fresh retry
- [x] M1 and M2 have fresh Adversary PASSes - [x] M1 and M2 have fresh Adversary PASSes for build #66
## M1 — State inventory and clean retry ## M1 — State inventory and clean retry
**Result:** current canonical Ghost upgrade PR verified by build #1332. **Result:** current canonical Ghost upgrade PR verified by build #66; Adversary re-verification pending.
### Live inventory ### Live inventory
| Check | Result | | Check | Result |
|---|---| |---|---|
| Open Ghost PRs | PR#7 only: `upgrade-808ac05` at `14575de6209ab2c4fb29d95fe454b761070cd351` | | Open Ghost PRs | PR#8 only: `upgrade-ee04ca9` at `dbe40ecb7b0084050a71baf4dcf971b65aac125e` |
| Historical PR#4 | closed 2026-06-15; not the current upgrade PR | | Historical PR#7 | closed; not the current upgrade PR |
| Historical duplicates | PR#3, PR#5, and PR#6 closed | | Historical duplicates | PR#1 through PR#7 closed |
| Proxy subnet | `10.10.0.0/16` | | Proxy subnet | `10.10.0.0/16` |
| Ghost stacks/services/volumes before retry | none | | Ghost stacks/services/volumes before retry | none |
| Fresh retry | Gitea comment `15814` at 2026-09-07T21:29:04Z; Drone build #1332 success, level 5/5 | | Fresh retry | Gitea comment `16297` at 2026-10-20T03:09:52Z; Drone build #66 success, level 5/5 |
### Adversary verification inputs ### Adversary verification inputs
**WHAT:** M1 is in progress. The correct live PR is `recipe-maintainers/ghost#7`, not historical PR#4. Build #1332 is the fresh post-proxy retry for its current head. **WHAT:** M1 and M2 claim the current live PR `recipe-maintainers/ghost#8` and fresh post-proxy build #66.
**HOW:** **HOW:**
@@ -56,12 +59,12 @@ set -a; . /srv/cc-ci/cc-ci-plan/load-env.sh; set +a
curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" \ curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" \
"https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/pulls?state=open" \ "https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/pulls?state=open" \
| jq -r '.[] | [.number, .state, .head.sha, .head.ref] | @tsv' | jq -r '.[] | [.number, .state, .head.sha, .head.ref] | @tsv'
ssh cc-ci 'docker network inspect proxy --format "{{range .IPAM.Config}}{{.Subnet}}{{end}}"; docker stack ls --format "{{.Name}}" | rg "^(ghos-|dev-ghost)" || true' ssh cc-ci 'docker network inspect proxy --format "{{range .IPAM.Config}}{{.Subnet}}{{end}}"; docker stack ls --format "{{.Name}}" | rg "^(ghos-|dev-ghost)" || true; docker service ls --format "{{.Name}}" | rg "(^|[-_])(ghos-|dev-ghost)" || true; docker volume ls --format "{{.Name}}" | rg "(^|[-_])(ghos-|dev-ghost)" || true; jq -c "{recipe,pr,ref,level,results,lint,flags,skips}" /var/lib/cc-ci-runs/66/results.json'
``` ```
**EXPECTED:** one open PR (`7`, `14575de6209ab2c4fb29d95fe454b761070cd351`, `upgrade-808ac05`), proxy subnet `10.10.0.0/16`, and no Ghost stack output. Build #1332 reports `success`, level `5`, and all lifecycle results `pass`. **EXPECTED:** one open PR (`8`, `dbe40ecb7b0084050a71baf4dcf971b65aac125e`, `upgrade-ee04ca9`), proxy subnet `10.10.0.0/16`, and no Ghost resource output. Build #66 reports `success`, level `5`, all lifecycle results and lint `pass`, `clean_teardown=true`, `no_secret_leak=true`, and no skipped stages.
**WHERE:** Gitea issue comment `15814`; Drone URL `https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1332`. **WHERE:** Gitea issue comments `16297` and `16298`; Drone URL `https://drone.ci.autonomic.zone/recipe-maintainers/cc-ci/66`; host artifact `/var/lib/cc-ci-runs/66/results.json`; Adversary evidence in `REVIEW-ghost.md`.
--- ---
@@ -74,5 +77,7 @@ ssh cc-ci 'docker network inspect proxy --format "{{range .IPAM.Config}}{{.Subne
| 519 | 2026-06-12T03:03Z | d88f5801 | ❌ FAIL | pre-proxy-fix, MySQL timing under load | | 519 | 2026-06-12T03:03Z | d88f5801 | ❌ FAIL | pre-proxy-fix, MySQL timing under load |
| 557 | 2026-06-12T21:51Z | d88f5801 | ❌ FAIL | pre-proxy-fix | | 557 | 2026-06-12T21:51Z | d88f5801 | ❌ FAIL | pre-proxy-fix |
| **612** | **2026-06-13T06:13Z** | **d88f5801** | **✅ PASS level 5/5** | **post-proxy-fix** | | **612** | **2026-06-13T06:13Z** | **d88f5801** | **✅ PASS level 5/5** | **post-proxy-fix** |
| **50** | **2026-10-13T03:12Z** | **ee04ca9e** | **✅ PASS level 5/5** | historical PR#8 retry |
| **66** | **2026-10-20T03:20Z** | **dbe40ecb** | **✅ PASS level 5/5** | fresh current PR#8 retry; lint verified |
Proxy /16 fix: 2026-06-13T05:38Z (pvfix phase). Proxy /16 fix: 2026-06-13T05:38Z (pvfix phase).
+13 -2
View File
@@ -34,10 +34,21 @@
}; };
# Root SSH authorized keys — preserved across nixos-rebuild switches. # Root SSH authorized keys — preserved across nixos-rebuild switches.
# 2026-09-28: synced to the live /root/.ssh/authorized_keys on cc-ci (which had drifted:
# several keys had been added manually and would have been wiped by the next rebuild).
# Claude-sandbox keys removed at operator request; new `nptest` and
# `notplants-orchestrator` keys added.
users.users.root.openssh.authorizedKeys.keys = [ users.users.root.openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOk8NaeBdPbS2gfUvbny8h0AkZlVjGYHzx4QPXSJ38gd claude@claude-vm" "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGZGp/DQTFuD1GvsyTzCVBUTmoWqcb5T+Z7zZo5nYLXO"
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhgo41nt8/L+Cr0PKd8jQK45mw/A+h041j6LQ8JWZisEVaQOzr6s9rxPL8VT5ML4P3/4bMblzdDiXWlJxymcb+yk5S5TnVrMavzHEDhWHwEvTRMe6xNTmsU6cmmhRw7PJqqQ+0GTlQalu3I4jkC0kTF7kuPwduUOgUuSpJqxvDTwYiXoyVnOQHAIygh+BmQvYUz0PBfQgIhgcbYmGZ++T0DnMzdGFzW2UB/iy5mymnpmbaZCgLy0w8AoDE+0YLtUc4gwTXc183nvqO1i7LQr+3jBYkv5ZthCCc52vXFHDSw9xZ5ohsOrBvoi5foRbqinmU5/t0aTK7SSrat7xXm/odIOyS+S7PJyeEcsXN6d5zdxbabAy5vLfodEaKGZd4rqQeDCxOTPAS/BlrBV/EV714n4E+fSOAllAuMBO4IibJM/gLJrh2Dql3co50QW9HEDeSC7iqp2lxRBDxvUs3rIEzy7o4HSN8chqBUK1bbBY6B17fuNHIpBAw4akRVVvPnVM= trav@trav480sweet"
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6jrKj7iZUNRLBTZG0vZM1D/BXtARhhB4+GrvpyuqmPb9iw2ifT9YqRUwgyGrOW9U6nIAR9yFnfp9+FkyhEKWByqEBbe/zYKlGLRGjfsIdDdW29QQ3hvmqNyboCkXLxZGat93poYhnoomqicmGD/xST4s0OUhcK9E494lUmenlD9dcMZW1aKpJ+9O4Dq6A7nk2z1e4KFcZdrZDI2Hgg+gfEdsKZQqd/R3Mls/eVKpzhfv3Y8BiNoHssUChVf8IGESqTOBOR7Dk7FsU5Z2ZcnQ1coxY7VlBn4fPjTWmz/Ac0jLqgcpCLpNyQzFPDVMYZKYrPVoqBeKVhN5YnfwR5OVP8YsakT/obLwC43sx/esXfjhVGcsRoGpiLOfazzNw/eC8s6FlS8cesOubEM37a7F25z4UEG3d487oM7EjQ39gBCCj/KRgUimCKMWsm6yIas4OSctBWEAo/NhZp0gwulSRxleW6eJCNNwzOmWjdzYIVWoVP0EIeM95Tq8PVUN7gpc= aadil@t480"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMyHSi12R0+HCVBz7+d9fyOBnoJi8Nsj5D7vQ9UQO8a5"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJVlfoLBPseQ9fA9534KmRg2KWcksKZGzAJIpHJ2JpsI mfowler.email@protonmail.com" "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJVlfoLBPseQ9fA9534KmRg2KWcksKZGzAJIpHJ2JpsI mfowler.email@protonmail.com"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAcyTGb/wVgdhg5oBCZZvBaR1RuUQRY/3WHnOQpNDCsp claude-cc-ci-sandbox@20260526" "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHOcLo0YBa0UYi7i/l8K/Y/7cF2OclmDqSTlAsHM0dOS notplants-orchestrator-ed25519"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMniNzAzuI527bfk/EipqFILFayUCwYXDoZ3R7+QgYq6"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKmGDZC6wrOQNJAW5PPDpxgEXXrcsnIU4b3QJLtq05RQ cc-ci-loops-to-root@cc-ci"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIONhAh8/jjH8v8AOZIlzL8yyNyb5VihefIEkaHschoJy nptest"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHIa8iunWtA4mqLKV6MuiTo3RkVj2ucsk3gLL4ArMEPO notplants-orchestrator"
]; ];
# Firewall — Hetzner has a public IP, so open 80+443 for Traefik. # Firewall — Hetzner has a public IP, so open 80+443 for Traefik.
+128 -66
View File
@@ -1,8 +1,13 @@
# Restricted DNS-01 certificate issuance for ci.commoninternet.net. # Restricted DNS-01 certificate issuance for ci.autonomic.zone (+ legacy ci.commoninternet.net).
# #
# This host is authoritative only for acme.commoninternet.net. Gandi continues # This host is authoritative only for acme.commoninternet.net. Gandi continues
# to own commoninternet.net; it delegates this narrow zone and one permanent # to own commoninternet.net AND autonomic.zone; both zones delegate their narrow
# _acme-challenge CNAME manually. No Gandi credential is present here. # _acme-challenge CNAME to the same acme-dns account. No Gandi credential is present here.
#
# Domain cutover 2026-09: ONE cert now covers BOTH zones (SANs: ci/*.ci.autonomic.zone +
# ci/*.ci.commoninternet.net) — a single secret pair flows through the unchanged
# warm_reconcile._traefik_setup ssl_cert/ssl_key flow, avoiding dual-cert SNI entirely.
# The legacy commoninternet.net SANs are dropped in Phase 4 (post-bake reissue).
{ config, pkgs, ... }: { config, pkgs, ... }:
let let
publicIPv4 = config.cc-ci.publicIPv4; publicIPv4 = config.cc-ci.publicIPv4;
@@ -42,11 +47,40 @@ let
# These are wiring values only. The acme-dns account JSON is generated by # These are wiring values only. The acme-dns account JSON is generated by
# Lego below /var/lib/acme and never enters Nix, git, or /etc. # Lego below /var/lib/acme and never enters Nix, git, or /etc.
# Storage path points at the NEW cert's dir; cc-ci-acme-storage-seed.service
# (below) clones the legacy account entry into it under the new zone key so
# Lego can write the new zone's TXT via the same (registration-disabled)
# acme-dns account — the CNAME for _acme-challenge.ci.autonomic.zone already
# delegates to that account's subdomain.
legoEnvironment = pkgs.writeText "cc-ci-acme-dns-lego.env" '' legoEnvironment = pkgs.writeText "cc-ci-acme-dns-lego.env" ''
ACME_DNS_API_BASE=http://127.0.0.1:8080 ACME_DNS_API_BASE=http://127.0.0.1:8080
ACME_DNS_STORAGE_PATH=/var/lib/acme/ci.commoninternet.net/acme-dns-accounts.json ACME_DNS_STORAGE_PATH=/var/lib/acme/ci.autonomic.zone/acme-dns-accounts.json
ACME_DNS_ALLOWLIST=127.0.0.1/32 ACME_DNS_ALLOWLIST=127.0.0.1/32
''; '';
# Seed the new cert's acmedns storage from the legacy one: same acme-dns
# account (same subdomain/credentials — the only thing the DNS CNAME points
# at), re-keyed for the new zone. Idempotent; runs before the ACME unit.
acmeStorageSeed = pkgs.writeShellApplication {
name = "cc-ci-acme-storage-seed";
runtimeInputs = with pkgs; [ jq coreutils ];
text = ''
src=/var/lib/acme/ci.commoninternet.net/acme-dns-accounts.json
dst=/var/lib/acme/ci.autonomic.zone/acme-dns-accounts.json
install -d -m 0700 /var/lib/acme/ci.autonomic.zone
if [ ! -s "$src" ]; then
echo "storage seed: no legacy accounts.json — skipping (fresh host?)"
exit 0
fi
jq 'if has("ci.autonomic.zone") then .
elif has("ci.commoninternet.net") then
. + { "ci.autonomic.zone": .["ci.commoninternet.net"] }
else . end' "$src" > "$dst.tmp"
mv "$dst.tmp" "$dst"
chmod 0600 "$dst"
echo "storage seed: ok"
'';
};
in in
{ {
imports = [ ./options.nix ]; imports = [ ./options.nix ];
@@ -77,78 +111,106 @@ in
"f /var/lib/ci-certs/acme-production-enabled 0600 root root -" "f /var/lib/ci-certs/acme-production-enabled 0600 root root -"
]; ];
services.acme-dns = { # Three systemd units (W201 statix): one attrset, distinct unit names — avoids
description = "Restricted authoritative DNS for cc-ci ACME DNS-01"; # `services = { ... }` attribute sets repeating the `services` key.
wantedBy = [ "multi-user.target" ]; services = {
after = [ "network-online.target" ]; acme-dns = {
wants = [ "network-online.target" ]; description = "Restricted authoritative DNS for cc-ci ACME DNS-01";
serviceConfig = { wantedBy = [ "multi-user.target" ];
User = "acme-dns"; after = [ "network-online.target" ];
Group = "acme-dns"; wants = [ "network-online.target" ];
StateDirectory = "acme-dns"; serviceConfig = {
StateDirectoryMode = "0700"; User = "acme-dns";
WorkingDirectory = "/var/lib/acme-dns"; Group = "acme-dns";
ExecStart = "${pkgs.acme-dns}/bin/acme-dns -c ${acmeDnsConfig}"; StateDirectory = "acme-dns";
Restart = "on-failure"; StateDirectoryMode = "0700";
RestartSec = "5s"; WorkingDirectory = "/var/lib/acme-dns";
AmbientCapabilities = [ "CAP_NET_BIND_SERVICE" ]; ExecStart = "${pkgs.acme-dns}/bin/acme-dns -c ${acmeDnsConfig}";
CapabilityBoundingSet = [ "CAP_NET_BIND_SERVICE" ]; Restart = "on-failure";
NoNewPrivileges = true; RestartSec = "5s";
PrivateTmp = true; AmbientCapabilities = [ "CAP_NET_BIND_SERVICE" ];
PrivateDevices = true; CapabilityBoundingSet = [ "CAP_NET_BIND_SERVICE" ];
ProtectHome = true; NoNewPrivileges = true;
ProtectSystem = "strict"; PrivateTmp = true;
ReadWritePaths = [ "/var/lib/acme-dns" ]; PrivateDevices = true;
RestrictAddressFamilies = [ "AF_INET" "AF_UNIX" ]; ProtectHome = true;
ProtectSystem = "strict";
ReadWritePaths = [ "/var/lib/acme-dns" ];
RestrictAddressFamilies = [ "AF_INET" "AF_UNIX" ];
};
}; };
};
# Traefik consumes its wildcard as immutable Swarm secrets, so a renewed # Seed the new cert's acmedns storage before the ACME unit first runs (see
# host certificate must be copied and reconciled rather than merely reloaded. # acmeStorageSeed above). Ordering via the generated acme unit name.
# This service is started only by the production-mode ACME postRun hook. cc-ci-acme-storage-seed = {
services.cc-ci-acme-traefik-handoff = { description = "Seed ci.autonomic.zone acme-dns storage from the legacy account";
description = "Install renewed cc-ci wildcard into Traefik Swarm secrets"; after = [ "acme-dns.service" ];
after = [ "docker.service" "deploy-proxy.service" ]; wantedBy = [ "acme-ci.autonomic.zone.service" ];
requires = [ "docker.service" ]; before = [ "acme-ci.autonomic.zone.service" ];
path = [ pkgs.coreutils pkgs.docker pkgs.systemd pkgs.gnugrep ]; serviceConfig = {
serviceConfig = { Type = "oneshot";
Type = "oneshot"; UMask = "0077";
UMask = "0077"; };
script = "${acmeStorageSeed}/bin/cc-ci-acme-storage-seed";
}; };
script = ''
src=/var/lib/acme/ci.commoninternet.net
dst=/var/lib/ci-certs/live
test -s "$src/fullchain.pem"
test -s "$src/key.pem"
install -d -m 0700 "$dst"
install -m 0444 "$src/fullchain.pem" "$dst/fullchain.pem.new"
install -m 0400 "$src/key.pem" "$dst/privkey.pem.new"
mv -f "$dst/fullchain.pem.new" "$dst/fullchain.pem"
mv -f "$dst/privkey.pem.new" "$dst/privkey.pem"
# deploy-proxy performs the health-gated Swarm rollout. Its reconciler # Traefik consumes its wildcard as immutable Swarm secrets, so a renewed
# derives a fresh version from the public certificate chain and inserts # host certificate must be copied and reconciled rather than merely reloaded.
# the matching ssl_cert/ssl_key secrets before deploying Traefik. # This service is started only by the production-mode ACME postRun hook.
systemctl restart deploy-proxy.service # Traefik consumes its wildcard as immutable Swarm secrets, so a renewed
# host certificate must be copied and reconciled rather than merely reloaded.
# This service is started only by the production-mode ACME postRun hook.
cc-ci-acme-traefik-handoff = {
description = "Install renewed cc-ci wildcard into Traefik Swarm secrets";
after = [ "docker.service" "deploy-proxy.service" ];
requires = [ "docker.service" ];
path = [ pkgs.coreutils pkgs.docker pkgs.systemd pkgs.gnugrep ];
serviceConfig = {
Type = "oneshot";
UMask = "0077";
};
script = ''
src=/var/lib/acme/ci.autonomic.zone
dst=/var/lib/ci-certs/live
test -s "$src/fullchain.pem"
test -s "$src/key.pem"
install -d -m 0700 "$dst"
install -m 0444 "$src/fullchain.pem" "$dst/fullchain.pem.new"
install -m 0400 "$src/key.pem" "$dst/privkey.pem.new"
mv -f "$dst/fullchain.pem.new" "$dst/fullchain.pem"
mv -f "$dst/privkey.pem.new" "$dst/privkey.pem"
# A successful rollout no longer references old wildcard versions. Best # deploy-proxy performs the health-gated Swarm rollout. Its reconciler
# effort removal retains any secret Docker still reports as in use. # derives a fresh version from the public certificate chain and inserts
keep="v$(sha256sum "$dst/fullchain.pem" | cut -c1-16)" # the matching ssl_cert/ssl_key secrets before deploying Traefik.
docker secret ls --format '{{.Name}}' | \ systemctl restart deploy-proxy.service
grep -E '^traefik_ci_commoninternet_net_ssl_(cert|key)_v' | \
grep -v -E "_(ssl_cert|ssl_key)_$keep\$" | \ # A successful rollout no longer references old wildcard versions. Best
while IFS= read -r stale; do docker secret rm "$stale" || true; done # effort removal retains any secret Docker still reports as in use.
''; keep="v$(sha256sum "$dst/fullchain.pem" | cut -c1-16)"
docker secret ls --format '{{.Name}}' | \
grep -E '^traefik_ci_commoninternet_net_ssl_(cert|key)_v' | \
grep -v -E "_(ssl_cert|ssl_key)_$keep\$" | \
while IFS= read -r stale; do docker secret rm "$stale" || true; done
'';
};
}; };
}; };
security.acme = { security.acme = {
acceptTerms = true; acceptTerms = true;
certs."ci.commoninternet.net" = { certs."ci.autonomic.zone" = {
domain = "ci.commoninternet.net"; domain = "ci.autonomic.zone";
extraDomainNames = [ "*.ci.commoninternet.net" ]; # Dual-zone SANs: the new zone + the legacy one. One cert = one ssl_cert/ssl_key
# Staging issuance succeeded using the permanent, narrowly delegated # secret pair = zero traefik reconciler changes. Phase 4 drops the legacy SANs.
# CNAME. Production uses the same restricted acme-dns account. extraDomainNames = [
"*.ci.autonomic.zone"
"ci.commoninternet.net"
"*.ci.commoninternet.net"
];
# Staging issuance proved the permanent, narrowly delegated CNAME and the
# restricted acme-dns account. Production uses the same account (re-keyed
# for the new zone by cc-ci-acme-storage-seed.service).
dnsProvider = "acmedns"; dnsProvider = "acmedns";
environmentFile = "/etc/acme-dns/lego.env"; environmentFile = "/etc/acme-dns/lego.env";
dnsResolver = "1.1.1.1:53"; dnsResolver = "1.1.1.1:53";
+3 -2
View File
@@ -33,7 +33,8 @@ let
image: cc-ci-bridge:${imageTag} image: cc-ci-bridge:${imageTag}
environment: environment:
- GITEA_API=https://git.autonomic.zone/api/v1 - GITEA_API=https://git.autonomic.zone/api/v1
- DRONE_URL=https://drone.ci.commoninternet.net - DRONE_URL=https://drone.ci.autonomic.zone
- DASH_URL=https://ci.autonomic.zone
- CI_REPO=recipe-maintainers/cc-ci - CI_REPO=recipe-maintainers/cc-ci
- BRIDGE_LISTEN=0.0.0.0:8080 - BRIDGE_LISTEN=0.0.0.0:8080
# Polling is PRIMARY (outbound, read-only, always on); the /hook webhook is an optional # Polling is PRIMARY (outbound, read-only, always on); the /hook webhook is an optional
@@ -57,7 +58,7 @@ let
labels: labels:
- "traefik.enable=true" - "traefik.enable=true"
- "traefik.http.services.ccci-bridge.loadbalancer.server.port=8080" - "traefik.http.services.ccci-bridge.loadbalancer.server.port=8080"
- "traefik.http.routers.ccci-bridge.rule=Host(`ci.commoninternet.net`) && PathPrefix(`/hook`)" - "traefik.http.routers.ccci-bridge.rule=(Host(`ci.autonomic.zone`) || Host(`ci.commoninternet.net`)) && PathPrefix(`/hook`)"
- "traefik.http.routers.ccci-bridge.entrypoints=web-secure" - "traefik.http.routers.ccci-bridge.entrypoints=web-secure"
- "traefik.http.routers.ccci-bridge.tls=true" - "traefik.http.routers.ccci-bridge.tls=true"
networks: networks:
+3 -3
View File
@@ -1,4 +1,4 @@
# Results dashboard (§4.5, D7): the YunoHost-CI-like overview at ci.commoninternet.net. Reads the # Results dashboard (§4.5, D7): the YunoHost-CI-like overview at ci.autonomic.zone. Reads the
# Drone API (read-only) and renders latest-run-per-recipe + SVG badges. Packaged as a Nix-built OCI # Drone API (read-only) and renders latest-run-per-recipe + SVG badges. Packaged as a Nix-built OCI
# image and run as a swarm service on `proxy`, routed by traefik at Host(ci.commoninternet.net) — the # image and run as a swarm service on `proxy`, routed by traefik at Host(ci.commoninternet.net) — the
# comment-bridge's Host && PathPrefix(`/hook`) rule is longer, so /hook still wins (priority by rule # comment-bridge's Host && PathPrefix(`/hook`) rule is longer, so /hook still wins (priority by rule
@@ -33,7 +33,7 @@ let
app: app:
image: cc-ci-dashboard:${imageTag} image: cc-ci-dashboard:${imageTag}
environment: environment:
- DRONE_URL=https://drone.ci.commoninternet.net - DRONE_URL=https://drone.ci.autonomic.zone
- CI_REPO=recipe-maintainers/cc-ci - CI_REPO=recipe-maintainers/cc-ci
- DASH_LISTEN=0.0.0.0:8080 - DASH_LISTEN=0.0.0.0:8080
- DRONE_TOKEN_FILE=/run/secrets/drone_token - DRONE_TOKEN_FILE=/run/secrets/drone_token
@@ -57,7 +57,7 @@ let
labels: labels:
- "traefik.enable=true" - "traefik.enable=true"
- "traefik.http.services.ccci-dashboard.loadbalancer.server.port=8080" - "traefik.http.services.ccci-dashboard.loadbalancer.server.port=8080"
- "traefik.http.routers.ccci-dashboard.rule=Host(`ci.commoninternet.net`)" - "traefik.http.routers.ccci-dashboard.rule=Host(`ci.autonomic.zone`) || Host(`ci.commoninternet.net`)"
- "traefik.http.routers.ccci-dashboard.entrypoints=web-secure" - "traefik.http.routers.ccci-dashboard.entrypoints=web-secure"
- "traefik.http.routers.ccci-dashboard.tls=true" - "traefik.http.routers.ccci-dashboard.tls=true"
networks: networks:
+4 -2
View File
@@ -30,9 +30,11 @@ let
exit 0 exit 0
fi fi
# NEVER prune mid-run: a live run-app stack means a deploy/test is in flight (mid-pull layers # NEVER prune mid-run: a live run-app stack means a deploy/test is in flight (mid-pull layers
# can look prunable). Run-app services: <=4char>-<6hex>_ci_commoninternet_net_* (lifecycle.py). # can look prunable). Run-app services: <=4char>-<6hex>_<zone-stem>_* (lifecycle.py).
# Zone stem matches BOTH ci_autonomic_zone (current) and ci_commoninternet_net
# (legacy stacks from before the 2026-09 domain cutover) during the bake window.
if docker service ls --format '{{.Name}}' \ if docker service ls --format '{{.Name}}' \
| grep -qE '^[a-z0-9]{1,4}-[0-9a-f]{6}_ci_commoninternet_net_'; then | grep -qE '^[a-z0-9]{1,4}-[0-9a-f]{6}_ci_(autonomic_zone|commoninternet_net)_'; then
echo "docker-prune: a run-app stack is live — skipping (never prune mid-run)" echo "docker-prune: a run-app stack is live — skipping (never prune mid-run)"
exit 0 exit 0
fi fi
+1 -1
View File
@@ -35,7 +35,7 @@ in
wantedBy = [ "multi-user.target" ]; wantedBy = [ "multi-user.target" ];
environment = { environment = {
DRONE_RPC_PROTO = "https"; DRONE_RPC_PROTO = "https";
DRONE_RPC_HOST = "drone.ci.commoninternet.net"; DRONE_RPC_HOST = "drone.ci.autonomic.zone";
DRONE_RUNNER_CAPACITY = maxTests; # MAX_TESTS concurrency cap (see let-binding above) DRONE_RUNNER_CAPACITY = maxTests; # MAX_TESTS concurrency cap (see let-binding above)
DRONE_RUNNER_NAME = "cc-ci-exec"; DRONE_RUNNER_NAME = "cc-ci-exec";
# exec runner needs a writable root for build workspaces # exec runner needs a writable root for build workspaces
+6 -3
View File
@@ -1,5 +1,5 @@
# Drone CI server = coop-cloud `drone` recipe via abra (swarm, traefik-routed at # Drone CI server = coop-cloud `drone` recipe via abra (swarm, traefik-routed at
# drone.ci.commoninternet.net, Gitea SSO, wildcard cert / no ACME). The exec *runner* is a # drone.ci.autonomic.zone, Gitea SSO, wildcard cert / no ACME). The exec *runner* is a
# separate host systemd service (modules/drone-runner.nix). See DECISIONS.md "CI engine"/"Drone # separate host systemd service (modules/drone-runner.nix). See DECISIONS.md "CI engine"/"Drone
# deployment shape". # deployment shape".
# #
@@ -17,7 +17,7 @@ let
name = "cc-ci-reconcile-drone"; name = "cc-ci-reconcile-drone";
runtimeInputs = with pkgs; [ abra docker jq gnused gnugrep coreutils git curl ]; runtimeInputs = with pkgs; [ abra docker jq gnused gnugrep coreutils git curl ];
text = '' text = ''
DRONE_DOMAIN="drone.ci.commoninternet.net" DRONE_DOMAIN="drone.ci.autonomic.zone"
ENV_FILE="$HOME/.abra/servers/default/$DRONE_DOMAIN.env" ENV_FILE="$HOME/.abra/servers/default/$DRONE_DOMAIN.env"
if [ ! -r /run/secrets/drone_rpc_secret ] || [ ! -r /run/secrets/drone_gitea_client_secret ]; then if [ ! -r /run/secrets/drone_rpc_secret ] || [ ! -r /run/secrets/drone_gitea_client_secret ]; then
@@ -52,7 +52,10 @@ let
set_env DRONE_ENV_VERSION "v1" set_env DRONE_ENV_VERSION "v1"
set_env COMPOSE_FILE '"compose.yml:compose.gitea.yml"' set_env COMPOSE_FILE '"compose.yml:compose.gitea.yml"'
have_secret() { docker secret ls --format '{{.Name}}' | grep -q "_$1_v1$"; } # Stack-scoped: secret names are prefixed with the app's stack name, so during the
# 2026-09 domain rename the OLD stack's *_rpc_secret_v1 must not satisfy this check.
DRONE_STACK="drone_ci_autonomic_zone"
have_secret() { docker secret ls --format '{{.Name}}' | grep -q "^''${DRONE_STACK}_$1_v1$"; }
have_secret rpc_secret || abra app secret insert "$DRONE_DOMAIN" rpc_secret v1 /run/secrets/drone_rpc_secret -f -n have_secret rpc_secret || abra app secret insert "$DRONE_DOMAIN" rpc_secret v1 /run/secrets/drone_rpc_secret -f -n
have_secret client_secret || abra app secret insert "$DRONE_DOMAIN" client_secret v1 /run/secrets/drone_gitea_client_secret -f -n have_secret client_secret || abra app secret insert "$DRONE_DOMAIN" client_secret v1 /run/secrets/drone_gitea_client_secret -f -n
+5 -1
View File
@@ -33,6 +33,10 @@ let
proxy_ssl_server_name on; proxy_ssl_server_name on;
proxy_set_header Host git.autonomic.zone; proxy_set_header Host git.autonomic.zone;
proxy_set_header Accept "application/json"; proxy_set_header Accept "application/json";
# Pin a non-browser UA: Gitea sits behind Anubis, which 307-challenges browser-like
# UAs to an un-CORS-able counter-domain — which would turn every STATUS cell into "?".
# A stable non-browser UA passes through (verified: curl 200, browser UA 307).
proxy_set_header User-Agent "ccci-reports-proxy/1.0";
proxy_pass https://git.autonomic.zone/api/v1/repos/recipe-maintainers/$1/pulls/$2; proxy_pass https://git.autonomic.zone/api/v1/repos/recipe-maintainers/$1/pulls/$2;
proxy_intercept_errors off; proxy_intercept_errors off;
proxy_connect_timeout 5s; proxy_connect_timeout 5s;
@@ -69,7 +73,7 @@ let
labels: labels:
- "traefik.enable=true" - "traefik.enable=true"
- "traefik.http.services.ccci-reports.loadbalancer.server.port=80" - "traefik.http.services.ccci-reports.loadbalancer.server.port=80"
- "traefik.http.routers.ccci-reports.rule=Host(`report.ci.commoninternet.net`)" - "traefik.http.routers.ccci-reports.rule=Host(`report.ci.autonomic.zone`) || Host(`report.ci.commoninternet.net`)"
- "traefik.http.routers.ccci-reports.entrypoints=web-secure" - "traefik.http.routers.ccci-reports.entrypoints=web-secure"
- "traefik.http.routers.ccci-reports.tls=true" - "traefik.http.routers.ccci-reports.tls=true"
networks: networks:
+1 -1
View File
@@ -2,7 +2,7 @@
"$schema": "https://opencode.ai/config.json", "$schema": "https://opencode.ai/config.json",
"agent": { "agent": {
"general": { "general": {
"model": "opencode-go/glm-5.2" "model": "opencode-go/deepseek-v4-flash"
} }
} }
} }
+4 -1
View File
@@ -117,7 +117,10 @@ def assert_serving(domain: str, meta) -> None:
# app is routed (Traefik serves the wildcard zone-wide). The serving proof is steps 1–2 above. # app is routed (Traefik serves the wildcard zone-wide). The serving proof is steps 1–2 above.
verified, detail = served_cert(domain) verified, detail = served_cert(domain)
assert verified, f"{domain}: served wildcard cert is not trusted/valid — {detail}" assert verified, f"{domain}: served wildcard cert is not trusted/valid — {detail}"
assert "commoninternet.net" in detail.lower(), f"{domain}: served cert unexpected — {detail}" # Dual-zone cert (2026-09 cutover): the SAN list covers both *.ci zones; a run
# domain in either zone must be covered by the one serving wildcard.
zone = "autonomic.zone" if domain.endswith(".autonomic.zone") else "commoninternet.net"
assert zone in detail.lower(), f"{domain}: served cert unexpected — {detail}"
# ---- Op/assertion split (Phase 1e HC3) ------------------------------------------------------- # ---- Op/assertion split (Phase 1e HC3) -------------------------------------------------------
+2 -2
View File
@@ -25,7 +25,7 @@ from . import meta as meta_mod
GATEWAY_IP = "143.244.213.108" # *.ci.commoninternet.net -> gateway (TLS passthrough to cc-ci) GATEWAY_IP = "143.244.213.108" # *.ci.commoninternet.net -> gateway (TLS passthrough to cc-ci)
# A run app domain is "<recipe[:4]>-<6hex>.ci.commoninternet.net" (see DECISIONS.md). Used by the # A run app domain is "<recipe[:4]>-<6hex>.ci.commoninternet.net" (see DECISIONS.md). Used by the
# janitor to recognise orphaned run apps (infra apps like traefik/drone/backups don't match). # janitor to recognise orphaned run apps (infra apps like traefik/drone/backups don't match).
RUN_APP_RE = re.compile(r"^[a-z0-9]{1,4}-[0-9a-f]{6}\.ci\.commoninternet\.net$") RUN_APP_RE = re.compile(r"^[a-z0-9]{1,4}-[0-9a-f]{6}\.ci\.(?:autonomic\.zone|commoninternet\.net)$")
class TeardownError(RuntimeError): class TeardownError(RuntimeError):
@@ -1190,7 +1190,7 @@ def janitor() -> None:
# also catch stacks whose .env was already deleted (abra ls won't list them) # also catch stacks whose .env was already deleted (abra ls won't list them)
for svc in _docker_names("service", ""): for svc in _docker_names("service", ""):
# svc like cust-c95a69_ci_commoninternet_net_app -> reconstruct domain # svc like cust-c95a69_ci_commoninternet_net_app -> reconstruct domain
m = re.match(r"^([a-z0-9]{1,4}-[0-9a-f]{6})_ci_commoninternet_net_", svc) m = re.match(r"^([a-z0-9]{1,4}-[0-9a-f]{6})_ci_(?:autonomic_zone|commoninternet_net)_", svc)
if m: if m:
seen.add(f"{m.group(1)}.ci.commoninternet.net") seen.add(f"{m.group(1)}.ci.commoninternet.net")
+4 -2
View File
@@ -1,7 +1,9 @@
"""Shared run-app domain naming (used by the conftest fixtures and the orchestrator). """Shared run-app domain naming (used by the conftest fixtures and the orchestrator).
Domain = "<recipe[:4]>-<6hex(recipe|pr|ref)>.ci.commoninternet.net" — short enough for Docker's Domain = "<recipe[:4]>-<6hex(recipe|pr|ref)>.ci.autonomic.zone" — short enough for Docker's
64-char swarm config/secret name limit, unique per run, collision-safe across recipes (DECISIONS.md). 64-char swarm config/secret name limit, unique per run, collision-safe across recipes (DECISIONS.md).
(Migrated from *.ci.commoninternet.net 2026-09; the legacy zone remains valid until the bake
window ends — lifecycle/warm name matching accepts both.)
""" """
from __future__ import annotations from __future__ import annotations
@@ -18,4 +20,4 @@ def app_domain(recipe: str, pr: str = "0", ref: str | None = None) -> str:
ref = ref or ("local" + str(int(time.time()))) ref = ref or ("local" + str(int(time.time())))
tag = _short(recipe, 4).lower() tag = _short(recipe, 4).lower()
h = hashlib.sha1(f"{recipe}|{pr}|{ref}".encode()).hexdigest()[:6] h = hashlib.sha1(f"{recipe}|{pr}|{ref}".encode()).hexdigest()[:6]
return f"{tag}-{h}.ci.commoninternet.net" return f"{tag}-{h}.ci.autonomic.zone"
+37
View File
@@ -27,6 +27,7 @@ from __future__ import annotations
import json import json
import os import os
import shutil
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
from . import level as level_mod from . import level as level_mod
@@ -35,6 +36,17 @@ from . import level as level_mod
# The dashboard serves these read-only at /runs/<run_id>/... (U0.4). Overridable for tests. # The dashboard serves these read-only at /runs/<run_id>/... (U0.4). Overridable for tests.
RUNS_DIR_DEFAULT = "/var/lib/cc-ci-runs" RUNS_DIR_DEFAULT = "/var/lib/cc-ci-runs"
# The artifact files/dirs a run OWNS in its run dir (written fresh every run).
_OWNED_ARTIFACTS = (
"results.json",
"summary.html",
"summary.png",
"badge.svg",
"lint.txt",
"screenshot.png",
"junit",
)
def runs_dir() -> str: def runs_dir() -> str:
return os.environ.get("CCCI_RUNS_DIR", RUNS_DIR_DEFAULT) return os.environ.get("CCCI_RUNS_DIR", RUNS_DIR_DEFAULT)
@@ -49,6 +61,31 @@ def run_id() -> str:
return os.environ.get("CCCI_APP_DOMAIN") or os.environ.get("CCCI_RUN_ID") or "manual" return os.environ.get("CCCI_APP_DOMAIN") or os.environ.get("CCCI_RUN_ID") or "manual"
def fresh_artifact_dir(path: str) -> str:
"""Ensure the run artifact dir holds no stale artifacts from a PREVIOUS run with the same id.
Drone build numbers restarted when its DB was re-created (2026-09-27) while /var/lib/cc-ci-runs
still holds old-era dirs numbered into the thousands — a new build reusing an old number would
otherwise mix old artifacts into the new run's card (observed 2026-10-05: !testme gitea build 35
collided with a May-31 run 35 and embedded custom-html-tiny's fossil screenshot.png, because
screenshot.capture() skips its snap when out_path already exists). Removes ONLY the files/dirs
this run owns (see _OWNED_ARTIFACTS); anything else in the dir is left untouched. Best-effort
(R7): never raises into the run."""
os.makedirs(path, exist_ok=True)
for name in _OWNED_ARTIFACTS:
p = os.path.join(path, name)
if os.path.isdir(p):
shutil.rmtree(p, ignore_errors=True)
else:
try:
os.remove(p)
except FileNotFoundError:
pass
except OSError:
pass # R7: a stuck file must never fail the run — the card is best-effort anyway
return path
def junit_file(junit_dir: str, tier: str, source: str, path: str) -> str: def junit_file(junit_dir: str, tier: str, source: str, path: str) -> str:
"""Deterministic per-(tier,source,file) JUnit XML path under junit_dir.""" """Deterministic per-(tier,source,file) JUnit XML path under junit_dir."""
base = os.path.splitext(os.path.basename(path))[0] base = os.path.splitext(os.path.basename(path))[0]
+6 -5
View File
@@ -142,12 +142,13 @@ def capture(domain: str, out_path: str, *, recipe_meta: dict | None = None) -> s
page = context.new_page() page = context.new_page()
if hook is not None: if hook is not None:
# Recipe-specific safe view (post-login etc.). The hook owns navigation + # Recipe-specific safe view (post-login etc.). The hook owns navigation +
# the no-secret-page guarantee; it should call page.screenshot itself, but if # the no-secret-page guarantee; we ALWAYS snap the page after it returns.
# it doesn't, we still snap the resulting page below. SCREENSHOT(page, ctx) — # Never trust a pre-existing out_path: a recycled run artifact dir (drone
# the uniform ctx convention (rcust P3). # build numbers restarted 2026-09-27 while old-era dirs linger) must not get
# its fossil screenshot embedded (observed 2026-10-05, !testme gitea #10
# build 35). SCREENSHOT(page, ctx) — the uniform ctx convention (rcust P3).
hook(page, meta_mod.hook_ctx(domain, recipe_meta)) hook(page, meta_mod.hook_ctx(domain, recipe_meta))
if not os.path.exists(out_path): _snap_with_blank_retry(page, out_path)
_snap_with_blank_retry(page, out_path)
else: else:
# Default: landing page. Accept any rendered status (200 or an auth redirect to a # Default: landing page. Accept any rendered status (200 or an auth redirect to a
# login form) — both are credential-free and representative of "the app is up". # login form) — both are credential-free and representative of "the app is up".
+7 -1
View File
@@ -25,6 +25,9 @@ import urllib.request
# Recipes that, when declared as a dep, are served from a shared live-warm instance at a stable # Recipes that, when declared as a dep, are served from a shared live-warm instance at a stable
# domain instead of being co-deployed per run. Maps dep-recipe -> stable domain. # domain instead of being co-deployed per run. Maps dep-recipe -> stable domain.
WARM_DOMAINS = { WARM_DOMAINS = {
# Warm stacks stay on the legacy zone through the 2026-09 domain cutover (data-warm
# volumes + stack names are tied to the abra app name; renaming re-seeds the CI bases).
# They move to *.ci.autonomic.zone in a post-bake phase together with their stacks.
"keycloak": "warm-keycloak.ci.commoninternet.net", "keycloak": "warm-keycloak.ci.commoninternet.net",
} }
@@ -38,11 +41,14 @@ _CTX.check_hostname = False
_CTX.verify_mode = ssl.CERT_NONE _CTX.verify_mode = ssl.CERT_NONE
# A cold per-run stack name looks like "<tag>-<6hex>_ci_commoninternet_net_<svc>"; extract the hex. # A cold per-run stack name looks like "<tag>-<6hex>_ci_commoninternet_net_<svc>"; extract the hex.
_STACK_HEX_RE = re.compile(r"^[a-z0-9]{1,4}-([0-9a-f]{6})_ci_commoninternet_net_") _STACK_HEX_RE = re.compile(
r"^[a-z0-9]{1,4}-([0-9a-f]{6})_ci_(?:autonomic_zone|commoninternet_net)_"
)
def stable_domain(recipe: str) -> str: def stable_domain(recipe: str) -> str:
"""The stable warm domain for a recipe: `warm-<recipe>.ci.commoninternet.net` — the canonical """The stable warm domain for a recipe: `warm-<recipe>.ci.commoninternet.net` — the canonical
base's hostname (legacy zone until the post-bake warm-stack migration; see WARM_DOMAINS note).
scheme for BOTH the live-warm keycloak and the data-warm canonicals (WC2), distinct from cold scheme for BOTH the live-warm keycloak and the data-warm canonicals (WC2), distinct from cold
per-run `<recipe[:4]>-<6hex>`. (WARM_DOMAINS['keycloak'] equals stable_domain('keycloak').)""" per-run `<recipe[:4]>-<6hex>`. (WARM_DOMAINS['keycloak'] equals stable_domain('keycloak').)"""
return f"warm-{recipe}.ci.commoninternet.net" return f"warm-{recipe}.ci.commoninternet.net"
+4
View File
@@ -1184,6 +1184,10 @@ def main() -> int:
# {tier,source,file,rc,junit} record; after the run we assemble results.json (per-stage/per-test + # {tier,source,file,rc,junit} record; after the run we assemble results.json (per-stage/per-test +
# level) into the artifact dir. Best-effort — never changes the verdict (R7). # level) into the artifact dir. Best-effort — never changes the verdict (R7).
run_artifact_dir = os.path.join(results_mod.runs_dir(), results_mod.run_id()) run_artifact_dir = os.path.join(results_mod.runs_dir(), results_mod.run_id())
# Collision guard (2026-10-05): drone build numbers restarted when its DB was re-created while
# old-era run dirs (same numbers, other recipes' artifacts — e.g. a fossil screenshot.png) are
# still on disk. Wipe the files THIS run owns so nothing stale survives into the card.
results_mod.fresh_artifact_dir(run_artifact_dir)
junit_dir = os.path.join(run_artifact_dir, "junit") junit_dir = os.path.join(run_artifact_dir, "junit")
records: list[dict] = [] records: list[dict] = []
+2 -2
View File
@@ -13,10 +13,10 @@ set -euo pipefail
: "${GITEA_USERNAME:?set GITEA_USERNAME}" : "${GITEA_USERNAME:?set GITEA_USERNAME}"
: "${GITEA_PASSWORD:?set GITEA_PASSWORD}" : "${GITEA_PASSWORD:?set GITEA_PASSWORD}"
GITEA="${GITEA:-https://git.autonomic.zone}" GITEA="${GITEA:-https://git.autonomic.zone}"
DRONE="${DRONE:-https://drone.ci.commoninternet.net}" DRONE="${DRONE:-https://drone.ci.autonomic.zone}"
CLIENT_ID="${CLIENT_ID:-ab4cdb9d-ee96-4867-875f-87384505fc52}" CLIENT_ID="${CLIENT_ID:-ab4cdb9d-ee96-4867-875f-87384505fc52}"
REPO="${REPO:-recipe-maintainers/cc-ci}" REPO="${REPO:-recipe-maintainers/cc-ci}"
RES=(--resolve "drone.ci.commoninternet.net:443:127.0.0.1") RES=(--resolve "drone.ci.autonomic.zone:443:127.0.0.1")
export PATH=/run/current-system/sw/bin:"$PATH" export PATH=/run/current-system/sw/bin:"$PATH"
cj=$(mktemp) cj=$(mktemp)
+1 -1
View File
@@ -15,7 +15,7 @@ sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "runner")
from harness import lifecycle # noqa: E402 from harness import lifecycle # noqa: E402
HELPERS = os.path.join(os.path.dirname(__file__), "helpers.py") HELPERS = os.path.join(os.path.dirname(__file__), "helpers.py")
DOMAIN = "test-abc123.ci.commoninternet.net" # matches RUN_APP_RE DOMAIN = "test-abc123.ci.autonomic.zone" # matches RUN_APP_RE
class HelperPool: class HelperPool:
+23 -9
View File
@@ -23,6 +23,7 @@ import shutil
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import time
import urllib.error import urllib.error
import urllib.request import urllib.request
@@ -45,7 +46,15 @@ def _lfs_available() -> bool:
return os.path.exists(os.path.join(abra_dir, "recipes", "gitea", _LFS_OVERLAY)) return os.path.exists(os.path.join(abra_dir, "recipes", "gitea", _LFS_OVERLAY))
def _api(domain, path, method="GET", body=None, user="", password=""): API_TIMEOUT = 20
def _api(domain, path, method="GET", body=None, user="", password="", timeout=API_TIMEOUT):
"""One API call → (status, JSON). Never raises: transient errors (connection refused /
dropped mid-restart, DNS, socket timeout) return (0, {}) so a caller's bounded poll can
simply retry, mirroring lifecycle.http_fetch. Only HTTPError was caught before; a socket
timeout on a slow post-restart boot (gitea 28.0.0 migration window, PR #10 run #31) escaped
as TimeoutError and aborted the restart poll itself."""
data = json.dumps(body).encode() if body is not None else None data = json.dumps(body).encode() if body is not None else None
auth = base64.b64encode(f"{user}:{password}".encode()).decode() auth = base64.b64encode(f"{user}:{password}".encode()).decode()
headers = {"Authorization": f"Basic {auth}"} headers = {"Authorization": f"Basic {auth}"}
@@ -55,7 +64,7 @@ def _api(domain, path, method="GET", body=None, user="", password=""):
f"https://{domain}/api/v1{path}", data=data, headers=headers, method=method f"https://{domain}/api/v1{path}", data=data, headers=headers, method=method
) )
try: try:
with urllib.request.urlopen(req, timeout=20, context=_CTX) as r: with urllib.request.urlopen(req, timeout=timeout, context=_CTX) as r:
raw = r.read() raw = r.read()
return r.status, (json.loads(raw) if raw else {}) return r.status, (json.loads(raw) if raw else {})
except urllib.error.HTTPError as e: except urllib.error.HTTPError as e:
@@ -64,6 +73,8 @@ def _api(domain, path, method="GET", body=None, user="", password=""):
return e.code, json.loads(raw) return e.code, json.loads(raw)
except (ValueError, json.JSONDecodeError): except (ValueError, json.JSONDecodeError):
return e.code, {} return e.code, {}
except (urllib.error.URLError, TimeoutError, OSError): # TimeoutError ⊃ socket.timeout (UP041)
return 0, {}
def _run_git(args, cwd, env=None): def _run_git(args, cwd, env=None):
@@ -164,6 +175,8 @@ def test_lfs_roundtrip(live_app):
finally: finally:
shutil.rmtree(fresh_dir, ignore_errors=True) shutil.rmtree(fresh_dir, ignore_errors=True)
import recipe_meta # noqa: E402 — per-recipe declared timeouts (tests/STYLE.md §5)
# 7. JWT-secret stability: restart gitea + assert LFS_JWT_SECRET unchanged. # 7. JWT-secret stability: restart gitea + assert LFS_JWT_SECRET unchanged.
# Read the current secret from inside the container's rendered app.ini. # Read the current secret from inside the container's rendered app.ini.
current_jwt = lifecycle.exec_in_app( current_jwt = lifecycle.exec_in_app(
@@ -182,14 +195,15 @@ def test_lfs_roundtrip(live_app):
capture_output=True, capture_output=True,
timeout=120, timeout=120,
) )
# Wait for gitea to come back up # Wait for gitea to come back up. Window + probe are DERIVED from the recipe's declared
# readiness (recipe_meta.HTTP_TIMEOUT / READY_PROBE — the /api/v1/version readiness that
# Re-read meta from the live_app fixture (meta is not in scope here — use the stored meta) # recipe_meta already declares for the harness, not a guess) — tests/STYLE.md §5.
import time wait_timeout = int(getattr(recipe_meta, "HTTP_TIMEOUT", 300))
probe_path = "/version" # READY_PROBE's readiness path (rules carry only ok + domain)
deadline = time.time() + 120 deadline = time.time() + wait_timeout
status2 = 0
while time.time() < deadline: while time.time() < deadline:
status2, _ = _api(live_app, "/version", user=user, password=password) status2, _ = _api(live_app, probe_path, user=user, password=password)
if status2 == 200: if status2 == 200:
break break
time.sleep(5) time.sleep(5)
+2 -1
View File
@@ -72,7 +72,8 @@ def UPGRADE_SECRET_PREP(ctx):
import subprocess import subprocess
# abra derives STACK_NAME from the domain by replacing dots with underscores # abra derives STACK_NAME from the domain by replacing dots with underscores
# (e.g. gite-e1cb78.ci.commoninternet.net → gite-e1cb78_ci_commoninternet_net). # (e.g. gite-e1cb78.ci.autonomic.zone → gite-e1cb78_ci_autonomic_zone; legacy
# *.ci.commoninternet.net run stacks are matched too during the bake window).
stack_name = ctx.domain.replace(".", "_") stack_name = ctx.domain.replace(".", "_")
docker_secret = f"{stack_name}_lfs_jwt_secret_v1" docker_secret = f"{stack_name}_lfs_jwt_secret_v1"
value = base64.urlsafe_b64encode(_os.urandom(32)).rstrip(b"=").decode() value = base64.urlsafe_b64encode(_os.urandom(32)).rstrip(b"=").decode()
+7
View File
@@ -59,6 +59,13 @@ def EXTRA_ENV(ctx):
# postgres, redis, collabora ~1GB, onlyoffice ~2GB). Bump so abra waits long enough for # postgres, redis, collabora ~1GB, onlyoffice ~2GB). Bump so abra waits long enough for
# convergence; kept under DEPLOY_TIMEOUT (1800) so Python never kills abra mid-wait. # convergence; kept under DEPLOY_TIMEOUT (1800) so Python never kills abra mid-wait.
"TIMEOUT": "1500", "TIMEOUT": "1500",
# abra's pinned-version `app new` regenerates the .env from generation hints and DROPS
# SECRET_COLLABORA_P_VERSION (not a generatable secret — the .env.sample ships its
# version line manually). Without it every secret-touching abra command for the app
# FATAs "missing version for secret? (collabora_p)" → the base deploy goes RED at
# install. env_set runs after app_new, so re-asserting the shipped version pin here
# restores the sample's contract.
"SECRET_COLLABORA_P_VERSION": "v1",
} }
+16
View File
@@ -171,6 +171,22 @@ def test_extra_env_dict_and_callable_forms(tmp_path):
assert meta_mod.extra_env(meta2, ctx2) == {} # unset EXTRA_ENV resolves to {} assert meta_mod.extra_env(meta2, ctx2) == {} # unset EXTRA_ENV resolves to {}
TESTS_DIR = os.path.join(os.path.dirname(__file__), "..")
def test_lasuite_drive_extra_env_carries_collabora_p_version():
"""(REGRESSION, 2026-10-25) abra's pinned-version `app new` regenerates the .env from
generation hints and drops SECRET_COLLABORA_P_VERSION (the .env.sample ships it manually),
so every secret-touching abra command for the app FATAs "missing version for secret?
(collabora_p)" and the cold base deploy goes RED at install. The per-recipe EXTRA_ENV must
re-assert the shipped version pin (env_set runs after app_new). Pin it here so abra/upstream
drift that loses it again surfaces as a red unit test, not a weekly red build."""
meta = meta_mod.load("lasuite-drive", tests_dir=TESTS_DIR)
ctx = meta_mod.hook_ctx("lasuite-drive-pr0-abc123.ci.commoninternet.net", meta)
extra = meta_mod.extra_env(meta, ctx)
assert extra.get("SECRET_COLLABORA_P_VERSION") == "v1"
# ---- P3: uniform ctx hook convention ------------------------------------------------------------- # ---- P3: uniform ctx hook convention -------------------------------------------------------------
+47
View File
@@ -489,3 +489,50 @@ def test_write_results_roundtrip(tmp_path):
assert path.endswith("/42/results.json") assert path.endswith("/42/results.json")
with open(path) as f: with open(path) as f:
assert json.load(f)["level"] == 3 assert json.load(f)["level"] == 3
def test_fresh_artifact_dir_removes_owned_stale_artifacts(tmp_path):
"""Run-dir collision guard (2026-10-05): drone build numbers restarted while old-era run dirs
linger in /var/lib/cc-ci-runs — a new build reusing an old number must not mix the old run's
artifacts (fossil screenshot.png from ANOTHER recipe) into its card. Only OWNED artifacts go."""
d = tmp_path / "35"
d.mkdir()
(d / "results.json").write_text('{"old": true}')
(d / "screenshot.png").write_bytes(b"stale-fossil")
(d / "summary.png").write_bytes(b"stale")
(d / "badge.svg").write_text("<svg/>")
(d / "lint.txt").write_text("old")
(d / "summary.html").write_text("<html/>")
(d / "junit").mkdir()
(d / "junit" / "old.xml").write_text("<x/>")
(d / "untracked-thing.bin").write_bytes(b"keep me") # NOT owned — must survive
R.fresh_artifact_dir(str(d))
for name in (
"results.json",
"screenshot.png",
"summary.png",
"badge.svg",
"lint.txt",
"summary.html",
):
assert not (d / name).exists(), f"stale owned artifact {name} survived"
assert not (d / "junit").exists(), "stale junit dir survived"
assert (d / "untracked-thing.bin").read_bytes() == b"keep me", "unowned file was deleted"
def test_fresh_artifact_dir_creates_missing_dir(tmp_path):
d = tmp_path / "36"
out = R.fresh_artifact_dir(str(d))
assert out == str(d) and d.is_dir()
def test_fresh_artifact_dir_tolerant_of_odd_entries(tmp_path):
"""R7: never raise into the run — a dir where an owned name is itself a dir must still work."""
d = tmp_path / "37"
d.mkdir()
(d / "screenshot.png").mkdir() # perverse: a DIRECTORY named screenshot.png
(d / "junit").mkdir()
R.fresh_artifact_dir(str(d))
assert not (d / "screenshot.png").exists() and not (d / "junit").exists()
+75
View File
@@ -184,3 +184,78 @@ def test_screenshot_reachable_through_real_load_path(tmp_path):
hook = S._load_screenshot_hook(meta) hook = S._load_screenshot_hook(meta)
assert callable(hook), "SCREENSHOT hook did not survive the orchestrator load path (R2)" assert callable(hook), "SCREENSHOT hook did not survive the orchestrator load path (R2)"
assert S._load_screenshot_hook(meta_mod.load("no-such", tests_dir=str(tmp_path))) is None assert S._load_screenshot_hook(meta_mod.load("no-such", tests_dir=str(tmp_path))) is None
# ---- capture() fossil guard (2026-10-05) -----------------------------------------------------
# Drone build numbers restarted when its DB was re-created while /var/lib/cc-ci-runs still holds
# old-era dirs numbered into the thousands; run 35 (!testme gitea #10) collided with a May-31 run
# 35 and embedded custom-html-tiny's fossil screenshot.png because the hook branch SKIPPED the
# snap when out_path already existed. capture() must always snap fresh.
class _FakeLaunchable:
def __init__(self, page):
self._page = page
self.chromium = self
def launch(self, args=None):
return self
def new_context(self, **kw):
return self
def new_page(self):
return self._page
def close(self):
pass
class _FakeSyncPlaywright:
"""sync_playwright() replacement: context-manager factory handing out one fake browser."""
def __init__(self, page):
self._page = page
def __call__(self):
return self
def __enter__(self):
return _FakeLaunchable(self._page)
def __exit__(self, *exc):
return False
def _install_fake_playwright(monkeypatch, page):
import types
fake_api = types.ModuleType("playwright.sync_api")
fake_api.sync_playwright = _FakeSyncPlaywright(page)
fake = types.ModuleType("playwright")
fake.sync_api = fake_api
monkeypatch.setitem(sys.modules, "playwright", fake)
monkeypatch.setitem(sys.modules, "playwright.sync_api", fake_api)
def test_capture_overwrites_fossil_screenshot(tmp_path, monkeypatch):
"""A pre-existing out_path in a recycled run artifact dir must never be embedded: the hook
branch snaps fresh regardless (the old exists-guard returned the fossil as 'captured')."""
out_path = str(tmp_path / "screenshot.png")
with open(out_path, "wb") as f:
f.write(b"STALE-FOSSIL-FROM-ANOTHER-RECIPE" * 400)
fired = {}
page = _FakePage([35707]) # one real-sized frame, like a painted page
def hook(p, ctx):
fired["yes"] = True # navigation only — hooks never save the PNG themselves
_install_fake_playwright(monkeypatch, page)
got = S.capture("gitea.example.org", out_path, recipe_meta={"SCREENSHOT": hook})
assert got == out_path, "capture must report success after snapping fresh"
assert fired.get("yes"), "SCREENSHOT hook must run"
assert page.shots == 1, "capture must snap exactly once — the fossil is not a captured frame"
with open(out_path, "rb") as f:
assert f.read(4) == b"\x89PNG", "out_path still holds the fossil — snap did not overwrite"