recipe-maintainers/cc-ci-orchestrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(orchestrator): wake the live monitor session

Browse Source
This commit is contained in:
autonomic-bot
2026-06-01 18:51:05 +00:00
parent 9fe9d49cac
commit 8f7265e948
5 changed files with 138 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@ -21,3 +21,6 @@
*age*.txt
*.age
master-age.txt
# Runtime completion markers
/cc-ci-plan/phase6-phase7.done

32
cc-ci-plan/JOURNAL.md
View File

@ -76,6 +76,27 @@ After 7 bug fixes (wrong inference host, opencode run exits, --dir exits, env: n
---
## Session 2026-06-01 14:13 UTC — OpenCode GPT-5.4
**Left off:** Completed the assistant-owned phase 6 mirror reconcile pass and phase 7 targeted recipe-upgrade pass, wrote the operator summary, and dropped a `phase6-phase7.done` marker.
**Phase / loop state:**
- Builder/Adversary loops still on phase **5 [11/11]** separately from this assistant work.
- Assistant phase 6 summary/result file: `cc-ci-plan/phase6-phase7-summary-2026-06-01.md`
- Assistant phase 6/7 completion marker: `cc-ci-plan/phase6-phase7.done`
**Open items:**
- Bridge enrollment does **not** match the full phase-2 18-recipe set. Repo/live poll set = `custom-html`, `custom-html-tiny`, `cryptpad`, `hedgedoc`, `keycloak`, `lasuite-docs`, `lasuite-meet`, `matrix-synapse`, `n8n` (+ `cc-ci`). Missing vs phase-2 set: `bluesky-pds`, `discourse`, `ghost`, `immich`, `lasuite-drive`, `mailu`, `mattermost-lts`, `mumble`, `plausible`, `uptime-kuma`. Extra: `hedgedoc`.
- `ghost` phase-7 PR is open but not CI-triggerable until bridge enrollment includes `recipe-maintainers/ghost`.
- Review whether recipes still intended to be enrolled without mirrors: `lasuite-drive`, `mailu`, `mumble`, `uptime-kuma`.
**Notes:**
- Phase 6 reconciled all 18 enrolled recipes from scratch clones. Stale mirror PRs auto-closed on `lasuite-docs` (#1/#2/#3) and `keycloak` (#1). Four enrolled recipes currently have no mirror repo.
- Phase 7 outcomes: `n8n` stable PR `#3` went GREEN on build `61`; `matrix-synapse` existing PR `#1` re-ran and failed on build `53`; `ghost` PR `#2` opened successfully but verification is blocked by bridge enrollment mismatch.
- The bridge service rolled during verification; earlier `!testme` comments posted before/re-during the restart were swallowed as pre-existing by the poller startup pass. A clean re-run on stable `n8n` after the rollout confirmed the live path.
---
## Session 2026-05-31 ~04:00 UTC — Claude Sonnet 4.6
**Left off:** Completed the orchestrator → Hetzner migration (cpx22, server 134487234, public
@ -169,3 +190,14 @@ activation. Hetzner rescue + VNC console showed emergency mode; mounted journal
(`fatlabel /dev/sda15 ESP`) and rebooting normally. Follow-up investigation item: determine why the
wrong boot layout was activated and prevent future use of `#cc-ci` on the Hetzner server when the
correct host target is `#cc-ci-hetzner`.
### Event 18:53 — scheduled supervision pass
Checked Builder, Adversary, and Assistant live state. `ssh cc-ci hostname` still returns `nixos` after
the corrected Hetzner rebuild. Builder is active on a fresh matrix-synapse rerun under the restored
bridge path; Adversary was nudged to re-orient to that live state; Assistant remains idle after
finishing phase 6/7 and recording the bridge-enrollment mismatch against the full 18-recipe phase-2 set.
### Event 16:34 — progress monitor nudged stalled phase-5 workers
`launch.py status` showed builder, adversary, and watchdog running; `ssh cc-ci hostname` succeeded (`nixos`).
Assistant session was present and already idle after its completed phase 6/7 pass (`phase6-phase7.done` exists).
Builder was still blocked on a model usage-limit retry and adversary was parked past `WAITING-UNTIL 2026-06-01T14:24:51Z`, so both received tmux nudges to re-read the live phase-5 status and continue from current evidence.

35
cc-ci-plan/ai-progress-monitor-prompt.txt Normal file
View File

@ -0,0 +1,35 @@
You are the cc-ci orchestrator and this is your scheduled wake-up prompt.
Every time you receive this prompt:
1. Check the current state of the three worker sessions:
- `cc-ci-builder`
- `cc-ci-adv`
- `cc-ci-assistant`
2. Use the real workspace and live state:
- `python3 cc-ci-plan/launch.py status`
- `tmux capture-pane -pt cc-ci-builder`
- `tmux capture-pane -pt cc-ci-adv`
- `tmux capture-pane -pt cc-ci-assistant`
- `ssh cc-ci hostname`
3. Keep them moving:
- If Builder is stalled or waiting past its stated wait, nudge it to continue phase 5.
- If Adversary is stale, behind, or still parked on old evidence, nudge it to re-orient to the current phase-5 state.
- If Assistant is stalled, nudge it to continue phase 6 and 7.
- If Assistant is not running, restart it via `python3 cc-ci-plan/launch-assistant.py start` with the opencode backend and re-send its assignment.
4. Completion behavior:
- Treat phase 5 as complete when `/srv/cc-ci/cc-ci/machine-docs/STATUS-5.md` contains `## DONE`.
- Treat phase 6/7 as complete when `/srv/cc-ci-orch/cc-ci-plan/phase6-phase7.done` exists.
- When BOTH are complete:
- stop the watchdog (`tmux kill-session -t cc-ci-watchdog` if it exists)
- tell Builder, Adversary, and Assistant to remain idle
- append a completion event to `/srv/cc-ci-orch/cc-ci-plan/JOURNAL.md`
- then exit successfully
5. Do not make unrelated code changes. Only supervise, nudge, restart if needed, and record concise journal notes when appropriate.
6. Be proactive and decisive. If everything is healthy and active, make no unnecessary changes.
7. If work is already in progress from the previous wake, just continue from the live state instead of restarting the analysis from scratch.

24
cc-ci-plan/ai-progress-monitor.sh Executable file
View File

@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -euo pipefail
SESSION="${AI_MONITOR_ORCH_SESSION:-cc-ci-orchestrator-oc}"
PROMPT_FILE="${AI_MONITOR_PROMPT_FILE:-/srv/cc-ci-orch/cc-ci-plan/ai-progress-monitor-prompt.txt}"
LOG_FILE="${AI_MONITOR_LOG_FILE:-/srv/cc-ci/.cc-ci-logs/ai-progress-monitor.log}"
SLEEP_SECS="${AI_MONITOR_INTERVAL:-900}"
mkdir -p "$(dirname "$LOG_FILE")"
while true; do
ts="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
{
printf '[%s] wake tick\n' "$ts"
if tmux has-session -t "$SESSION" 2>/dev/null; then
tmux send-keys -t "$SESSION" -l -- "$(cat "$PROMPT_FILE")"
tmux send-keys -t "$SESSION" C-m
printf '[%s] prompted %s\n' "$ts" "$SESSION"
else
printf '[%s] session missing: %s\n' "$ts" "$SESSION"
fi
} >>"$LOG_FILE" 2>&1 || true
sleep "$SLEEP_SECS"
done

44
cc-ci-plan/phase6-phase7-summary-2026-06-01.md Normal file
View File

@ -0,0 +1,44 @@
# Phase 6 / Phase 7 Operator Summary — 2026-06-01
## Phase 6 — mirror reconcile
Expected upstream source for every enrolled recipe below: `https://git.coopcloud.tech/coop-cloud/<recipe>`.
- `bluesky-pds` — mirror `main` synced; open PR #1 left open.
- `cryptpad` — mirror `main` synced; open PR #3 left open.
- `custom-html` — mirror `main` synced; open PRs #1, #2, #3 left open.
- `custom-html-tiny` — mirror `main` synced; open PRs #2, #5 left open.
- `discourse` — mirror `main` synced; open PR #1 left open.
- `ghost` — mirror `main` synced; open PR #1 was left open during phase 6 and later replaced in phase 7.
- `immich` — mirror `main` synced; open PR #1 left open.
- `keycloak` — mirror `main` created/synced from upstream `master`; stale PR #1 closed because its changes were already upstream.
- `lasuite-docs` — mirror `main` synced; stale PRs #1, #2, #3 closed because their changes were already upstream.
- `lasuite-drive` — no `recipe-maintainers/lasuite-drive` mirror repo exists; manual attention needed if this recipe should stay enrolled.
- `lasuite-meet` — mirror `main` synced; open PR #2 left open.
- `mailu` — no `recipe-maintainers/mailu` mirror repo exists; manual attention needed if this recipe should stay enrolled.
- `matrix-synapse` — mirror `main` synced; open PR #1 left open.
- `mattermost-lts` — mirror `main` synced; open PR #1 left open.
- `mumble` — no `recipe-maintainers/mumble` mirror repo exists; manual attention needed if this recipe should stay enrolled.
- `n8n` — mirror `main` synced; open PR #2 was left open during phase 6 and later replaced in phase 7.
- `plausible` — mirror `main` synced; open PR #1 left open.
- `uptime-kuma` — no `recipe-maintainers/uptime-kuma` mirror repo exists; manual attention needed if this recipe should stay enrolled.
## Phase 7 — targeted upgrades
- `n8n` — upstream target chosen: stable `2.22.6` (not pre-release `2.23.x`), plus `pgautoupgrade:18-alpine`; PR opened: `https://git.autonomic.zone/recipe-maintainers/n8n/pulls/3`; superseded pre-release PR #2 was closed; after the bridge rollout settled, a fresh `!testme` completed GREEN on `https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/61`.
- `ghost` — upstream target chosen: latest published Docker Hub stable image `6.42.0-alpine` (GitHub releases were already at `6.43.x`, but no matching published Alpine image was available); PR opened: `https://git.autonomic.zone/recipe-maintainers/ghost/pulls/2`; prior open PR #1 was closed as superseded by the current mirror workflow; verification is currently blocked because `recipe-maintainers/ghost` is not enrolled in the bridge poll set, so `!testme` comments never enter the real CI path.
- `matrix-synapse` — existing PR `https://git.autonomic.zone/recipe-maintainers/matrix-synapse/pulls/1` already matches latest stable target `7.2.0+v1.153.0`; that PR already had an older cc-ci pass comment on build `87`, and a fresh re-run today produced a new failure comment on `https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/53`.
## Bridge enrollment audit
- Phase-2 18-recipe test set: `bluesky-pds`, `cryptpad`, `custom-html`, `custom-html-tiny`, `discourse`, `ghost`, `immich`, `keycloak`, `lasuite-docs`, `lasuite-drive`, `lasuite-meet`, `mailu`, `matrix-synapse`, `mattermost-lts`, `mumble`, `n8n`, `plausible`, `uptime-kuma`.
- Repo `nix/modules/bridge.nix` poll set at completion time: `custom-html`, `custom-html-tiny`, `cryptpad`, `hedgedoc`, `keycloak`, `lasuite-docs`, `lasuite-meet`, `matrix-synapse`, `n8n` plus `recipe-maintainers/cc-ci`.
- Live bridge poll set on `cc-ci` at completion time matched the repo poll set above.
- Missing from bridge enrollment relative to the phase-2 18-recipe set: `bluesky-pds`, `discourse`, `ghost`, `immich`, `lasuite-drive`, `mailu`, `mattermost-lts`, `mumble`, `plausible`, `uptime-kuma`.
- Extra in bridge enrollment relative to the phase-2 18-recipe set: `hedgedoc`.
## Manual attention
- Decide whether `lasuite-drive`, `mailu`, `mumble`, and `uptime-kuma` should remain enrolled without mirror repos.
- Review whether the mirror workflow's current "close any other open PR when opening an upgrade PR" behavior is still desired for recipes like `ghost`, where the superseded PR was unrelated to the upgrade.
- Align bridge enrollment with the full 18-recipe test set if `ghost` and the other phase-2 recipes should be triggerable via `!testme`.