recipe-maintainers/cc-ci-orchestrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

journal: unstuck weekly upgrade wedged on discourse Swarm scheduling hiccup

Browse Source
This commit is contained in:
autonomic-bot
2026-06-12 00:31:29 +00:00
parent a1cceef3d4
commit 1eb720e95a
1 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
cc-ci-plan/JOURNAL.md
View File

@ -518,3 +518,13 @@ session cc-ci-orchestrator-stale can be killed; recipe-mirrors org still private
stopped its loops + the watchdog; phase-idx preserved at 9. Upgrade left running. stopped its loops + the watchdog; phase-idx preserved at 9. Upgrade left running.
- RESUME cfold (restart watchdog → phase-idx 9) once /upgrade-all is confirmed DONE. See - RESUME cfold (restart watchdog → phase-idx 9) once /upgrade-all is confirmed DONE. See
memory [[cfold-paused-pending-upgrade]]. Will action on supervision wakes. memory [[cfold-paused-pending-upgrade]]. Will action on supervision wakes.
## 2026-06-12 ~00:30 — unstuck the weekly upgrade (wedged on discourse)
- /upgrade-all froze ~2h on discourse: its iteration-2 !testme chaos deploy disc-50cc8a
had app+sidekiq stuck 0/1 in Swarm 'New' state 24min (db/redis up, box 5.8Gi free) —
transient scheduler wedge, NOT a recipe defect (discourse L5 @build #450 ~5h prior).
drone build waited on it; testme-on-pr.sh blocked polling; agent frozen.
- Fix: docker stack rm disc-50cc8a (freed box+build); Esc-interrupted the upgrader; nudged
it with the diagnosis → "one clean discourse retry then move on regardless; comment+skip
if it re-wedges". Agent recovered, now checking build state before retry. Rest of queue
(ghost/immich/keycloak/lasuite-*/mailu/matrix-synapse) still ahead. cfold still paused.