Commit Graph
100 Commits
Author SHA1 Message Date
autonomic-bot 78af2c995a note(redfix): post-verdict JOURNAL consult — warm-state dimension never considered (F-redfix-4 is a blind spot, not a deferral)
continuous-integration/drone/push Build is failing
2026-07-08 23:58:32 +00:00
autonomic-bot 941d5e81c1 review(redfix): VETO — keycloak enrollment not collision-free in warm state (F-redfix-4)
continuous-integration/drone/push Build is failing
Post-reboot re-confirmation #8. New break-it probe on the two HARNESS fixes (previously
only sha-reachability checked, never content, never second-order effects).

mumble 07fc6d4: CLEAN — attempts 12->36 @5.0s = claimed 180s; all assertions unchanged,
so a dead server still FAILs. Budget widened without weakening the test.

keycloak 61211db: correct at the domain/stack layer, but its own invariant is FALSE.
canonical_domain() separates the two deployments by domain, yet warm STATE is keyed by
recipe: warmsnap.snap_dir("keycloak") is ONE slot shared by the live-warm reconciler
(warm-keycloak, stateful=True) and the newly-enrolled data-warm canonical
(warm-canon-keycloak, seeded via promote_canonical -> seed_canonical, no WARM_DOMAINS guard).

Proved by execution on cc-ci in a scratch CCCI_WARM_ROOT against the real idle canon stack:
snapshot(canon) -> snapshot(other stack) destroys the canonical known-good, and
restore(canon) raises SnapshotError. Fails closed (no cross-stack data write), but:
 1. every stateful reconciler upgrade deterministically destroys the canonical snapshot;
 2. the reconciler's rollback restore() is OUTSIDE its try/except and its snapshot->restore
    window spans deploy+wait_healthy (health_timeout 900). A sweep promote landing there makes
    the rollback raise AFTER abra.undeploy(live) -> live keycloak left undeployed = outage of
    the shared OIDC provider lasuite-*/drone depend on. Exactly the hazard canon §2.B's
    de-enrollment exception existed to prevent.
 3. prune_stale()'s documented "keycloak untouched" invariant becomes false once seeded.

Undetectable by the M2 run: /var/lib/ci-warm/keycloak/ holds only last_good (no canonical.json,
no snapshot/) vs cryptpad which has both — the promote deployed but seed_canonical never ran,
registry-advance being deferred to the operator's merge. First seed happens post-merge, unexercised.

VETO scoped to keycloak. M1 classifications and the discourse/mattermost-lts/gitea/bluesky-pds/
mumble fixes are unaffected and remain PASS. Clears when the two deployments use disjoint
warm-state paths and each restore() returns its own stack's volumes.

Node left clean: real warm root untouched, volumes intact, live keycloak 200 throughout.
2026-07-08 23:58:01 +00:00
autonomic-botandClaude Opus 4.8 4682aa1216 review(redfix): post-reboot re-confirmation #7 — DONE stands, no VETO. Zero delta since #6 (Builder JOURNAL append only). New break-it probe: verified fix CONTENT (not just sha reachability) at every M2 anchor via cold bare clones — mattermost-lts 4ca7f418 adds backupbot.restore.post-hook, gitea a0f2db8 adds app.ini.init + docker-setup cp, bluesky-pds 4987ba9 adds FQ APP_HOST with no leftover bare app alias — each with its parent commit as a negative control (0 matches pre-fix). All 4 recipe fixes now content-anchored. All findings closed
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HHWWoceR5fHLTEwW8b3QnQ
2026-07-08 23:42:52 +00:00
autonomic-botandClaude Opus 4.8 c6593bbf6e review(redfix): post-reboot re-confirmation #6 — DONE stands, no VETO. Zero delta since #5 (only a Builder JOURNAL append). Break-it probe on evidence-anchor rot: cold bare-cloned all 3 remaining recipe mirrors — mattermost-lts 4ca7f418, gitea a0f2db8, bluesky-pds 4987ba9 all REACHABLE with subjects matching their asserted fixes, and harness branch redfix-m2-harness still pins exactly 07fc6d4 (mumble) with 61211db (keycloak) reachable. Rot confined to discourse (F-redfix-3, CLOSED). All findings closed
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WY7MNKwa6gascKiXER9ZbN
2026-07-08 23:36:00 +00:00
autonomic-bot 805c44e42d review(redfix): post-reboot re-confirmation #5 — DONE stands, no VETO. Audited Builder's F-redfix-3 addendum (612412c): append-only (+54/-0), my files + DONE line + historical shas untouched, no gate reopened. Ran its published repro verbatim in a fresh clone: ede6399 → official image 3.5.3 + 0 sidekiq, both EXPECTED met. Ruled out vacuous zero (file exists, blob, services:app: present) and added negative control (d7c8c47 added sidekiq, 0c4539b removed it) — fix COMMIT still reachable, only sha labels rotted. F-redfix-3 remedy verified, not just asserted
continuous-integration/drone/push Build is failing
2026-07-08 23:29:09 +00:00
autonomic-bot 67e86ded88 review(redfix): F-redfix-2 CLOSED — gitignore remedy cold-verified (fresh-clone + add -A replay, origin/main, no HTTP exposure); full 51-char key never committed (my prefix-grep was self-contaminated); Builder's symlink correction confirmed — one repo, fix fully applied; rotation left to operator; DONE stands, no VETO; consumed ADVERSARY-INBOX
continuous-integration/drone/push Build is failing
2026-07-08 23:13:26 +00:00
autonomic-bot a16c367091 review(redfix): DONE re-confirmed post-reboot #3 (M1+M2 PASS, no VETO) + NEW non-blocking finding F-redfix-2 — live API key in un-gitignored config.json at both Builder clone roots (never committed; one 'git add -A' from a push)
continuous-integration/drone/push Build is failing
2026-07-08 23:06:44 +00:00
autonomic-bot ddf3788483 note(redfix): post-reboot re-confirmation — phase remains DONE (M1+M2 PASS, no VETO); node healthy; no pending claim; loop stopped
continuous-integration/drone/push Build is failing
2026-07-08 20:29:07 +00:00
autonomic-bot 337931065a review(redfix-M2): PASS 6/6 — discourse re-verified level=5 (F-redfix-1 CLOSED); all 6 canon-sweep fixes cold-verified; node clean; no VETO; Builder cleared to DONE
continuous-integration/drone/push Build is failing
2026-06-18 07:06:27 +00:00
autonomic-bot 70afd937c3 note(redfix-M2): BUILDER-INBOX heads-up — discourse smtpauth sidekiq remedy; other 5 solid, don't redo
continuous-integration/drone/push Build is failing
2026-06-18 06:46:59 +00:00
autonomic-bot 3f5eddfdbd review(redfix-M2): FAIL — 5/6 PASS (keycloak/mumble/gitea/bluesky/mattermost), discourse FAIL (F-redfix-1: incomplete migration, dangling image-less sidekiq in compose.smtpauth.yml -> R011 lint regression + breaks smtp-auth; run #849 also level=4)
continuous-integration/drone/push Build is failing
2026-06-18 06:45:46 +00:00
autonomic-bot 21e8ca336e note(redfix-M2): bluesky-pds component VERIFIED (4/6) — chaos-deploy fix, caddy resolves own app 10.0.5.5 (bare app=foreign 10.10), health 200 {0.4.219}, 0 conn-refused; node clean
continuous-integration/drone/push Build is failing
2026-06-18 06:21:42 +00:00
autonomic-bot 319ec9cd36 note(redfix-M2): gitea component VERIFIED (3/6) — chaos-deploy fix, no read-only crash, app.ini seeded 1862B, API 1.24.2; canonical unchanged; merge-gating honest
continuous-integration/drone/push Build is failing
2026-06-18 06:17:57 +00:00
autonomic-bot 6ff71f76b3 note(redfix-M2): mumble component VERIFIED (2/6) — handshake PASS 10.3s (flake confirmed, fix non-weakening); consume inbox (b96b8a4 staleness is bluesky-only, keycloak/mumble unaffected)
continuous-integration/drone/push Build is failing
2026-06-18 06:13:34 +00:00
autonomic-bot 5babd027f0 note(redfix-M2): keycloak component VERIFIED (1/6) — promote at warm-canon-keycloak, live SSO undisturbed (up 4d, 200); gate verdict pending 5 more
continuous-integration/drone/push Build is failing
2026-06-18 06:09:23 +00:00
autonomic-bot 966edb3042 note(redfix): idle break-it probe — live keycloak 200 (undisturbed), gitea canonical unchanged (no false promote during rework); M2 not yet claimed
continuous-integration/drone/push Build is failing
2026-06-18 04:12:39 +00:00
autonomic-bot fca936ef50 note(redfix): M2 interim corroboration — mattermost-lts run #901 restore tier (test_restore_returns_state) PASSES, clean teardown + no leak; non-contending artifact check, not a verdict; M2 not yet claimed
continuous-integration/drone/push Build is failing
2026-06-18 02:15:17 +00:00
autonomic-bot c742f9adc4 journal(redfix): cc-ci-side verification mechanism (temp-checkout run) + M2 progress snapshot
continuous-integration/drone/push Build is failing
2026-06-18 01:51:54 +00:00
autonomic-bot 125e1ba675 journal(redfix): M2 bluesky — abra drops compose net aliases (proven); pivot to service rename app->pds + coupled cc-ci exec-ref update
continuous-integration/drone/push Build is failing
2026-06-18 01:50:26 +00:00
autonomic-bot c3854a9bcc status+journal(redfix): M2 — mattermost-lts FIXED (run #901 all green, restore fixed); discourse #4 green; bluesky PR #4 created (promote-path verify next)
continuous-integration/drone/push Build is failing
2026-06-18 01:30:57 +00:00
autonomic-bot abfbe8b0aa journal+status(redfix): M2 recon — discourse #4 (official-image) already !testme-green; mattermost #1 (pg-restore) triggered for verify
continuous-integration/drone/push Build is failing
continuous-integration/drone Build is passing
2026-06-18 01:24:48 +00:00
autonomic-bot 6771c713f0 inbox(redfix): consume Adversary M1-PASS heads-up — node clean (gitea idle 3.5.3 unchanged, keycloak healthy); proceeding to M2
continuous-integration/drone/push Build is failing
2026-06-18 01:20:27 +00:00
autonomic-bot 191ddc9fb8 status(redfix): M1 PASS (Adversary cold-verified all 6 classifications CORRECT); begin M2 fixes
continuous-integration/drone/push Build is failing
2026-06-18 01:20:15 +00:00
autonomic-bot b6038e9796 inbox(redfix): heads-up to Builder — M1 PASS, node restored clean (gitea idle 3.5.3 canonical unchanged), cleared for M2
continuous-integration/drone/push Build is failing
2026-06-18 01:19:52 +00:00
autonomic-bot edee91341c review(redfix-M1): PASS — all 6 classifications cold-verified by my own isolation re-runs. discourse=stale overlay (no timeout, my run converged in min), mattermost=deterministic restore RED, mumble=flake (handshake green isolated), bluesky=recipe app-alias proxy collision (getent app->10.10.0.4, not machinery), gitea=read-only app.ini JWT crash (canonical unchanged), keycloak=warm-domain collision. No VETO. Node clean before+after.
continuous-integration/drone/push Build is failing
2026-06-18 01:19:27 +00:00
autonomic-bot 14aa55f02b note(redfix): M1 interim — gitea CONFIRMED by my run + container crash log (LoadCommonSettings JWT save to read-only /etc/gitea/app.ini config mount); genuine recipe defect
continuous-integration/drone/push Build is failing
2026-06-18 01:09:49 +00:00
autonomic-bot c9c870f0a6 note(redfix): M1 interim — mattermost CONFIRMED deterministic restore RED (ci_marker does not exist, 91s isolation; no restore.post-hook); genuine recipe defect not load-race
continuous-integration/drone/push Build is failing
2026-06-18 01:02:16 +00:00
autonomic-bot 968780234b note(redfix): M1 interim — discourse CONFIRMED (no timeout/wedge; install+backup+restore+custom pass, upgrade reds on PR-faithfulness overlay asserting unreleased official:3.5.3/no-sidekiq); stale overlay test
continuous-integration/drone/push Build is failing
2026-06-18 00:56:57 +00:00
autonomic-bot 5512dcaba5 note(redfix): M1 interim — mumble CONFIRMED flake (handshake test PASSED in my isolation run, all 5 tiers green, promote ok); bluesky orphan cleaned up
continuous-integration/drone/push Build is failing
2026-06-18 00:44:44 +00:00
autonomic-bot 0c11b0b39d note(redfix): M1 interim — bluesky-pds CONFIRMED by my reproduction (getent app->10.10.0.4 proxy collision, real app 10.0.5.6 never resolved; deterministic 000); recipe routing defect not machinery/flake
continuous-integration/drone/push Build is failing
2026-06-18 00:26:19 +00:00
autonomic-bot 65fe47feea journal(redfix): M2 prep — bluesky fix refinement (unique internal alias, not service rename)
continuous-integration/drone/push Build is failing
2026-06-18 00:25:57 +00:00
autonomic-bot 4777ba8edc backlog(redfix): M2 fix designs from M1 evidence (mattermost/bluesky/gitea recipe PRs; keycloak/mumble harness; discourse overlay-scope) — execution gated on M1 PASS
continuous-integration/drone/push Build is failing
2026-06-18 00:20:14 +00:00
autonomic-bot 0a06c411a6 claim(redfix-M1): all 6 canon-sweep failures investigated in isolation + classified (results table + cold-verify guide). discourse=stale overlay test, mattermost-lts=recipe restore defect, mumble=load FLAKE (2x green), bluesky=app-alias proxy collision, gitea=app.ini RO crash, keycloak=warm-domain collision. 2 canon root-causes corrected.
continuous-integration/drone/push Build is failing
2026-06-18 00:18:09 +00:00
autonomic-bot 00fca8a33e journal+status(redfix): M1 gitea app.ini read-only JWT crash CONFIRMED on warm advance (recipe defect); 6/6 classified
continuous-integration/drone/push Build is failing
2026-06-18 00:14:32 +00:00
autonomic-bot 88c9ebcce4 status(redfix): M1 tracker — keycloak classified (harness collision); 5/6 done, gitea app.ini advance reproducing
continuous-integration/drone/push Build is failing
2026-06-18 00:08:40 +00:00
autonomic-bot 93e1e7d87a note(redfix): M1 pre-staging — mattermost (no restore.post-hook) + discourse (PR-faithfulness overlay) static claims corroborated via code; owe own discourse isolation run + bluesky diag before any PASS
continuous-integration/drone/push Build is failing
2026-06-18 00:08:31 +00:00
autonomic-bot 8a54c4d0ea journal(redfix): M1 keycloak (harness warm-domain collision, design-complete) + gitea first-run already-deployed confound
continuous-integration/drone/push Build is failing
2026-06-18 00:08:25 +00:00
autonomic-bot f8ba0c3a1f journal(redfix): M1 bluesky-pds — 000 reproduces deterministically; root cause = caddy↔app cross-stack 'app' alias collision on shared proxy (recipe defect)
continuous-integration/drone/push Build is failing
2026-06-18 00:02:26 +00:00
autonomic-bot 41e161a433 status(redfix): M1 tracker — discourse/mattermost/mumble classified; bluesky promote in flight
continuous-integration/drone/push Build is failing
2026-06-17 23:53:13 +00:00
autonomic-bot 9a58268e12 journal(redfix): M1 mumble isolation GREEN — load/timing flake confirmed
continuous-integration/drone/push Build is failing
2026-06-17 23:44:24 +00:00
autonomic-bot 8df74d7bc0 journal(redfix): M1 mattermost-lts isolation — DETERMINISTIC restore fail; genuine recipe defect (no restore.post-hook vs immich)
continuous-integration/drone/push Build is failing
2026-06-17 23:41:29 +00:00
autonomic-bot 23b439db83 journal(redfix): M1 discourse isolation — canon root-cause wrong; deploys fine, only upgrade overlay (unreleased official-image migration) fails
continuous-integration/drone/push Build is failing
2026-06-17 23:33:18 +00:00
autonomic-bot 3e61473365 chore(redfix): bootstrap phase state files (STATUS/BACKLOG/JOURNAL); M1 investigation tracker seeded
continuous-integration/drone/push Build is failing
2026-06-17 23:20:55 +00:00
autonomic-bot a30e71825e review(redfix): open phase — REVIEW skeleton, cold access to cc-ci confirmed healthy, awaiting Builder bootstrap + M1 claim
continuous-integration/drone/push Build is failing
2026-06-17 23:19:36 +00:00
autonomic-bot de4d69072c status(nixenv): mark phase DONE in STATUS (M1+M2 both PASS, no VETO)
continuous-integration/drone/push Build is failing
2026-06-17 23:18:36 +00:00
autonomic-botandClaude Opus 4.8 0b84452290 review(M2-nixenv): PASS — live parity cold-verified on cc-ci (claim f7b6f26, deploy d11f8f5). Deploy byte-identical to M1 build; host healthy post-sweep (systemctl --failed empty, timer+services active, endpoints 200, no orphan test stacks, live cc-ci-run=zxlx9jn). gitea test_lfs_roundtrip GREEN under BOTH real timer fire (git-lfs from runtimeInputs; unit PATH has no git-lfs) AND Drone #871 (cc-ci-run runner/run_recipe_ci.py). No regression: ZERO missing-tool signatures across whole sweep; SKIPs/promotes correct; gitea promote-fail (warm-gitea already deployed) + discourse/mattermost reds (image-assertion / postgres relation, docker resolved) all proven pre-existing — identical in OLD-env pre-deploy fires, runner/ unchanged since canon f94de22. No defects, no VETO. M1+M2 fresh PASS → DONE cleared.
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 18:21:16 +00:00
autonomic-bot f7b6f26859 claim(M2-nixenv): live parity proven on BOTH paths — gitea test_lfs_roundtrip green under the real timer fire (@17:57:54Z, git-lfs from cc-ci-run runtimeInputs; unit PATH has no git-lfs) AND the Drone path (build #871, RECIPE=gitea REF=357926f2 PR=1). Deploy d11f8f5 healthy post-sweep (systemctl --failed empty, timer+oneshots active, endpoints 200). No regression: sweep SKIPs/promotes correct; gitea promote-fail + discourse/mattermost reds all pre-existing (identical pre-deploy, runner/ unchanged since canon f94de22). Awaiting Adversary.
continuous-integration/drone/push Build is failing
2026-06-17 18:18:53 +00:00
autonomic-botandClaude Opus 4.8 e0c296e0e6 inbox(nixenv): consumed Builder M2 heads-up — Drone-path witness #871 in flight; concur promote-failure pre-existing. Will independently verify both witnesses before verdict.
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 18:12:00 +00:00
autonomic-bot c8d4528cbc inbox(nixenv): Drone-path LFS witness build #871 in flight (RECIPE=gitea REF=357926f2 PR=1); timer-fire witness already PASS
continuous-integration/drone/push Build is failing
2026-06-17 18:11:25 +00:00
autonomic-bot bfdfd10098 inbox(nixenv): consume Adversary M2 heads-up — concur GREEN-BUT-PROMOTE-FAILED is pre-existing (nixenv diff dd6712c..d11f8f5 is nix/+docs only, runner/nightly_sweep.py unchanged since canon f94de22; warm-gitea up since 08:39Z → 'already deployed')
continuous-integration/drone/push Build is failing
continuous-integration/drone Build is passing
2026-06-17 18:07:05 +00:00
autonomic-botandClaude Opus 4.8 b278082272 note(nixenv): heads-up to Builder — gitea LFS witness GREEN under timer fire, but sweep hit GREEN-BUT-PROMOTE-FAILED (warm-gitea already deployed); asking claim to establish it's pre-existing not nixenv-caused (runner promote path unchanged)
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 18:05:58 +00:00
autonomic-bot 2cc7328c5c status(M2-nixenv): timer-fire LFS witness PASS (test_lfs_roundtrip green from cc-ci-run runtimeInputs; systemd unit PATH has no git-lfs). GREEN-BUT-PROMOTE-FAILED is pre-existing abra warm-deploy idempotency, not a regression. Drone-path witness pending sweep completion.
continuous-integration/drone/push Build is failing
2026-06-17 18:05:29 +00:00
autonomic-botandClaude Opus 4.8 d9eab45557 status(M2-nixenv): deployed clean (system byte-identical to M1 review); real timer fire started — gitea LFS witness in flight
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:36:09 +00:00
autonomic-botandClaude Opus 4.8 c0ac552441 status(M2-nixenv): M1 PASS recorded; M2 deploy in flight on cc-ci(hetzner)
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:28:37 +00:00
autonomic-botandClaude Opus 4.8 d11f8f56c4 review(M1-nixenv): PASS — single-source harness runtime env cold-verified (claim 8b8fc1f). Both hosts build (no collision); withPackages/pytest-playwright/ccciRuntimeTools each single-def; sweep+Drone both exec byte-identical cc-ci-run zxlx9jn… (15-tool PATH incl git-lfs-3.6.1+openssl-3.3.3, ends :$PATH so nothing dropped); host configs textually identical, cc-ci sw/bin GAINS git-lfs+openssl, DEFECT-3 host-PATH patch removed; future-dep propagation single-source by construction. No defects, no VETO. M2 (deploy+live LFS witness) awaits.
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:26:56 +00:00
autonomic-botandClaude Opus 4.8 8b8fc1ff8e claim(M1-nixenv): single-source harness runtime env — ccciPyEnv+ccciRuntimeTools+cc-ci-run in packages.nix, referenced by harness/sweep/both hosts; sweep execs cc-ci-run (no dup pyEnv, no DEFECT-3 PATH patch); cc-ci host gains git-lfs+openssl; both #cc-ci and #cc-ci-hetzner build; awaiting Adversary
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:23:28 +00:00
autonomic-botandClaude Opus 4.8 706583bee3 review(nixenv): cold-prep — enumerate 3 current env declarations + union the shared set must cover; noted cc-ci/hetzner host git-lfs divergence as break-it target. Awaiting M1 claim.
continuous-integration/drone/push Build is failing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:12:06 +00:00
autonomic-bot 7507cf4736 review(M2): PASS — live full per-recipe history verified (image 11ac2a1e6c07 1/1; bluesky-pds 8/ghost 24/immich 28/discourse 25 = host, plausible+custom-html capped 30; exact ts order incl mixed-id trap; cap keeps newest=758; overview+badge 200; live traversal/injection 404, no leak; retention no-trim confirmed). M1+M2 fresh PASS, no VETO.
continuous-integration/drone/push Build is failing
2026-06-17 16:39:35 +00:00
autonomic-bot 84ac65f6d2 review(M1): PASS — local-artifact history cold-verified vs host (bluesky-pds=8 exact ts order, mixed-id trap handled, 308 rows, cap keeps newest, malformed dirs skip no-500, security guards intact, stdlib-only, 13/13 unit). No defects.
continuous-integration/drone/push Build is failing
continuous-integration/drone Build is passing
2026-06-17 16:28:09 +00:00
autonomic-bot 2d5211f401 review(dash): pre-claim independent ground truth baseline — 432 run dirs/308 parseable/124 unparseable, bluesky-pds=8 runs w/ mixed numeric+named ids (timestamp-sort trap), per-recipe counts, break-test plan
continuous-integration/drone/push Build is passing
2026-06-17 16:20:53 +00:00
autonomic-bot 4f6d73302a review(canon): CLOSE DEFECT-1/2/3 — all re-verified resolved at M2 PASS (honest labels, faithful-install promote 16 clean, env-parity git-lfs proven in production timer fire)
continuous-integration/drone/push Build is passing
2026-06-17 16:16:35 +00:00
autonomic-botandClaude Opus 4.8 86d61fe662 status(canon): ## DONE — M1+M2 fresh Adversary PASS (8149a2c, no VETO), §5 DoD fully cold-verified
continuous-integration/drone/push Build is passing
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 16:16:02 +00:00
autonomic-bot 8149a2cd4a review(M2): PASS — canonical sweep proven end-to-end, no VETO. 16 canonicals commit==tag (cold re-derived), real non-hollow timer fire (Result=success, single serial, custom-html 1.11→1.13 advance), determinism 2nd sweep 15-skip/5-documented-exception-run (no overlap, launched 14:41 after 14:37 fire end), tagged-gate both ways, samever step-back never fires in-sweep, UPGRADE_BASE_VERSION retired (plausible dynamic base 3.0.1 re-derived), my own --quick warm reattach reuses retained volume + 200, all 6 exceptions in DECISIONS, AI-free. DEFECT-3 CLOSED (parity byte-match + gitea lfs PASS in prod fire). M1+M2 fresh PASS → Builder may write ## DONE
continuous-integration/drone/push Build is passing
2026-06-17 16:15:28 +00:00
autonomic-bot a4f1df435b claim(M2): canonical sweep proven end-to-end — real timer fire promoted 16 canonicals (custom-html 1.11→1.13 live advance), determinism 2nd sweep clean (15 at-latest SKIP, only documented exceptions RUN), tagged-promote/samever-orthogonality/disk-budget/UPGRADE_BASE_VERSION-retirement all proven; 6 exceptions in DECISIONS; AI-free runtime
continuous-integration/drone/push Build is passing
2026-06-17 16:07:18 +00:00
autonomic-bot 29ca9b92a1 status(canon): stage M2 claim body (all sub-items WHAT/HOW/EXPECTED/WHERE) — finalizing on determinism 2nd sweep completion
continuous-integration/drone/push Build is passing
2026-06-17 15:59:05 +00:00
autonomic-bot 009bc60dc0 decisions(canon): record M2.7 warm-volume disk budget — 38G free, all-enrolled sustainable, no recipe dropped
continuous-integration/drone/push Build is passing
2026-06-17 15:57:14 +00:00
autonomic-bot 245c937ed7 chore(canon): consume ADVERSARY-INBOX — clean determinism 2nd sweep heads-up (M2.3 evidence in flight, pid 2248547); staying off-node, will verify SKIP/RUN partition + single-serial at M2 claim
continuous-integration/drone/push Build is passing
2026-06-17 14:42:52 +00:00
autonomic-bot 5c67543f6d inbox(canon): heads-up — clean determinism 2nd sweep in flight (M2.3 evidence), single node, ~96m
continuous-integration/drone/push Build is passing
2026-06-17 14:42:07 +00:00
autonomic-bot e8822165dd journal(canon): production re-fire COMPLETE (Result=success, gitea cold-green via lfs PASS under parity PATH) — DEFECT-3 closed; launched clean determinism 2nd sweep (custom-html now at 1.13.0 → all 16 promoted at-latest)
continuous-integration/drone/push Build is passing
2026-06-17 14:41:45 +00:00
autonomic-bot cf0659fc1f review(canon): production-env real timer fire COMPLETED clean (Result=success, single serial) — custom-html promoted 1.11→1.13, 14 SKIP, 6 documented exceptions; DEFECT-3 prod re-validation favorable, closes at M2 claim
continuous-integration/drone/push Build is passing
2026-06-17 14:39:43 +00:00
autonomic-bot 1fd89dbaa1 review(canon): DEFECT-3 parity REAL (sweep PATH byte-matches Drone, git-lfs present) + live timer re-fire re-validating — gitea lfs PASSED cold-green, custom-html 1.11→1.13 promoted, promoted set SKIPs; favorable but M2 unclaimed, won't close until fire completes
continuous-integration/drone/push Build is passing
2026-06-17 14:28:34 +00:00
autonomic-bot 1cc14aa98e journal(canon): resume reconstruction — parity fix deployed, real timer re-fire in flight (custom-html 1.11→1.13 promoted)
continuous-integration/drone/push Build is passing
2026-06-17 13:20:26 +00:00
autonomic-bot cd897a1885 review(canon): assess DEFECT-3 env-parity fix (2c61f2f, host PATH=Drone parity) — right fix; DEFECT-3 stays OPEN until nixos-rebuild + real-timer re-fire re-validates promoted set in production env (verify parity real, gitea flips cold-green)
continuous-integration/drone/push Build is passing
2026-06-17 13:10:14 +00:00
autonomic-botandClaude Opus 4.8 2c61f2fadf fix(canon): sweep runs with host PATH = Drone-runner env parity (DEFECT-3 git-lfs etc.)
continuous-integration/drone/push Build is passing
The real timer fire redded gitea at the custom tier (git: 'lfs' is not a git command) — the
nightly-sweep writeShellApplication had a clean nix-only PATH, while Drone's recipe-CI runner runs
with PATH=/run/current-system/sw/bin:/run/wrappers/bin (where git-lfs + all host tooling live). My
manual sweeps used a login PATH that masked this. Prepend the host system PATH so the timer sweep
validates recipes in the SAME environment as Drone — one fix for git-lfs/bash/openssl/etc. parity.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 13:00:18 +00:00
autonomic-bot c387ee1dd8 chore(canon): consume BUILDER-INBOX (DEFECT-3 git-lfs/env-parity — fixing sweep PATH, will re-fire as M2.2 evidence)
continuous-integration/drone/push Build is passing
2026-06-17 12:59:27 +00:00
autonomic-bot bd0a565680 review+inbox(canon): DEFECT-3 — real timer fire reds gitea on MISSING git-lfs in nightly-sweep.service runtimeInputs (same class as bash gap); manual sweep env (had git-lfs, gitea cold-green) != production timer env → M2.2 promote evidence must be re-validated under the real timer; heads-up sent
continuous-integration/drone/push Build is passing
2026-06-17 12:57:58 +00:00
autonomic-bot 7f2e256866 review(canon): §2.G strip code-level CONFIRMED complete (no live UPGRADE_BASE_VERSION; only removal comments; KEYS 15->14; plausible dynamic base 3.0.1) — M2.8 favorable, re-run units+plausible at claim; M2.5 bash-fix needs redeploy+fresh fire
continuous-integration/drone/push Build is passing
2026-06-17 12:35:14 +00:00
autonomic-botandClaude Opus 4.8 cebd293c5a fix(canon): add bash to nightly-sweep runtimeInputs (real timer fire caught missing bash)
continuous-integration/drone/push Build is passing
The deployed sweep service (writeShellApplication) sets a clean PATH from runtimeInputs only;
mirror_sync shells out via subprocess.run(['bash', recipe-mirror-sync.sh, r]) → FileNotFoundError
'bash' on the real systemd fire (manual ssh runs had bash on PATH and masked it). Add bash.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:34:03 +00:00
autonomic-botandClaude Opus 4.8 83c183d985 feat(canon): §2.G strip UPGRADE_BASE_VERSION entirely (plausible verified dynamic-base green)
continuous-integration/drone/push Build is passing
Gate satisfied — live: with the pin removed, plausible's upgrade tier resolves base 3.0.1+v2.0.0 via
the same-version step-back (canonical 3.1.0 == head 3.1.0 → newest-older = 3.0.1, NOT the broken
3.0.0) and passes install+upgrade green (level 5/5). The pin is redundant, so removed everywhere:
- meta.py KEYS entry (RecipeMeta field auto-drops; 15→14 keys).
- run_recipe_ci.resolve_upgrade_base override branch + docstrings.
- tests/unit/test_meta.py (count 15→14, dropped None-assert), test_upgrade_base.py (override test).
- docs/recipe-customization.md (regenerated table + mentions), docs/testing.md.
- tests/plausible/recipe_meta.py (pin removed), tests/bluesky-pds (re-enable note → dynamic base).
294 unit tests pass; lint clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:31:53 +00:00
autonomic-botandClaude Opus 4.8 f611dda893 feat(canon): §2.G remove plausible UPGRADE_BASE_VERSION pin (dynamic base resolves 3.0.1 via step-back)
continuous-integration/drone/push Build is passing
plausible's canonical is established at 3.1.0+v2.0.0 (latest), so the dynamic resolver no longer
needs the explicit pin: a same-version head steps back to newest-older = 3.0.1+v2.0.0 (NOT the
broken 3.0.0). Verifying live before stripping the key globally (§2.G gate).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:26:25 +00:00
autonomic-bot 8e15def15d review(canon): acceptance bar for gitea-exception (VERIFY custom-html advance really promoted + gitea app.ini-RO is recipe not machinery mount) + M2.3 reframing (accept IFF 2nd sweep: 15 skip / only documented exceptions run; flag as literal-DoD deviation for operator)
continuous-integration/drone/push Build is passing
2026-06-17 12:22:52 +00:00
autonomic-bot bdc2ec4773 decisions(canon): gitea 3.6.0 warm-advance exception (app.ini read-only, recipe issue; 3.5.3 valid) + M2.3 determinism framing
continuous-integration/drone/push Build is passing
2026-06-17 12:19:04 +00:00
autonomic-bot 9ffbba57e3 review(canon): authoritative sweep DONE rc=0 @12:00:03Z (single serial, 11:25:57->12:00:03); determinism preview visible (promoted recipes SKIP); awaiting gitea fix + M2.3/5/6/7/8 proofs before claim
continuous-integration/drone/push Build is passing
2026-06-17 12:10:44 +00:00
autonomic-bot 930335972a chore(canon): consume BUILDER-INBOX (gitea 3.6.0 advance — fixing; drone promoted clean)
continuous-integration/drone/push Build is passing
2026-06-17 12:00:53 +00:00
autonomic-bot a6c506844a review+inbox(canon): final-sweep crux — drone PROMOTED CLEAN (residue fix works, DEFECT-2 closing) but gitea 3.6.0 advance FAILED AGAIN (GREEN-BUT-PROMOTE-FAILED, canon kept 3.5.3) → CLAIM-BLOCKER for M2.6 (advance undemonstrated) + M2.3 (green recipe re-runs, not a red); heads-up sent
continuous-integration/drone/push Build is passing
2026-06-17 11:59:14 +00:00
autonomic-bot 35d629452b decisions(canon): record 4 recipe RED exceptions (discourse upstream-compose / mattermost+mumble test-red / bluesky warm-routing) — genuine, tests unmodified, left intact
continuous-integration/drone/push Build is passing
2026-06-17 11:37:33 +00:00
autonomic-bot 31fbed13b6 review(canon): CONFIRMED final authoritative sweep @12acf94 contains both ca89d44+d072d7e (recency criterion MET); list red-diagnosis verifications (discourse/mattermost-lts/mumble/bluesky) — verify genuine+not-weakened+DECISIONS-recorded at claim
continuous-integration/drone/push Build is passing
2026-06-17 11:35:51 +00:00
autonomic-bot 2ce31b4035 status(canon): FINAL authoritative M2.2 sweep launched (post-fix /etc/cc-ci@12acf94, enrolled=20, serial); red diagnoses recorded
continuous-integration/drone/push Build is passing
2026-06-17 11:26:19 +00:00
autonomic-bot 12acf94b91 review(canon): pre-fix sweep DONE (15 canonicals); NEW red mumble rc=1 (must fix-or-document); plausible promoted 3.1.0+v2.0.0 not 3.0.1 → §2.8 retirement must re-derive dynamic base vs actual canonical
continuous-integration/drone/push Build is passing
2026-06-17 11:23:53 +00:00
autonomic-bot 32c9703ffe review(canon): VERIFIED fresh-seed-teardown × live-keycloak footgun MITIGATED — keycloak de-enrolled (enrolled=20, not in set), live warm-keycloak 200 + 1/1 unharmed by pre-fix sweep; carry: check no other recipe domain collides with a live service
continuous-integration/drone/push Build is passing
2026-06-17 11:12:25 +00:00
autonomic-bot 618ac1ef6f status(canon): M2 snapshot — 10 clean promotes incl. lasuite-* (warm dep works); plan for authoritative post-fix sweep
continuous-integration/drone/push Build is passing
2026-06-17 11:03:00 +00:00
autonomic-bot 3bcc11f7b5 review(canon): note residue fix (ca89d44, likely drone root cause) + keycloak de-enroll (d072d7e, §2.B exception, enrolled=20); set M2-evidence recency criterion — accepted sweep must postdate both fixes, single serial, drone promotes-or-exception
continuous-integration/drone/push Build is passing
2026-06-17 11:00:24 +00:00
autonomic-botandClaude Opus 4.8 d072d7e2c2 fix(canon): de-enroll keycloak (live-warm OIDC provider) — §2.B exception
continuous-integration/drone/push Build is passing
keycloak is the always-on shared OIDC dep provider at warm-keycloak.ci..., the SAME stable domain a
data-warm canonical would use → the sweep's promote would collide with the live provider that
lasuite-*/drone depend on. keycloak is kept current by roll_warm_infra (WC1.1) instead.
WARM_CANONICAL=False; exception recorded in DECISIONS. Enrolled set now 20.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:54:14 +00:00
autonomic-botandClaude Opus 4.8 ca89d44c05 fix(canon): promote clears stale warm-stack on a fresh seed (failed-promote secret residue)
continuous-integration/drone/push Build is passing
A once-failed promote left swarm secrets (e.g. drone's gitea client_secret_v1) behind; the retry's
install_steps 'abra app secret insert' then FATAd 'already exists', so a recipe could never recover
its canonical. promote_canonical now teardown_app()s the warm domain when there is NO existing
canonical (fresh seed) — clearing leftover secrets/.env/partial volumes — while a re-promote
(canonical exists) still reattaches its retained known-good volume untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:51:01 +00:00
autonomic-bot d32940d3e1 review(canon): clean-serial sweep obs — drone STILL promote-fails clean (lock fix cured hang, not promote; M2 risk); gitea new-tag 3.5.3->3.6.0 advance = live M2.6 evidence
continuous-integration/drone/push Build is passing
2026-06-17 10:48:12 +00:00
autonomic-bot d4a053dfcc chore(canon): consume ADVERSARY-INBOX (concurrent sweeps killed, drone tainted-canonical discarded, ONE clean serial sweep relaunched pid1741209); carry to claim — verify 7 kept canonicals' ts outside concurrency window
continuous-integration/drone/push Build is passing
2026-06-17 10:25:01 +00:00
autonomic-bot 1f4aa25a2b inbox+status(canon): killed concurrent sweeps, cleaned residue, cleared concurrency-tainted drone canonical; ONE clean serial sweep relaunched
continuous-integration/drone/push Build is passing
2026-06-17 10:24:06 +00:00
autonomic-bot fb2fe307dc chore(canon): consume BUILDER-INBOX (concurrent-sweep alert — killing wedged old sweep, will re-run clean serial)
continuous-integration/drone/push Build is passing
2026-06-17 10:21:42 +00:00
autonomic-bot 4d5b03b485 inbox+review(canon): TWO concurrent sweeps — wedged old sweep (PID1712141, drone deadlock child ~46m) still alive alongside new re-run (PID1736506); violates §4 serial + breaks release_app_locks precondition; M2 evidence from overlapping run not acceptable
continuous-integration/drone/push Build is passing
2026-06-17 10:20:49 +00:00
autonomic-bot 88293702b2 status(canon): mirror-sync master-detection + cold-dep lock-release fixes deployed; validating drone
continuous-integration/drone/push Build is passing
2026-06-17 10:05:13 +00:00