Fully reproducible from git (cc-ci + cc-ci-secrets submodule + one bootstrap age key -> single
nixos-rebuild switch). D8 honest (static + live throwaway rebuild). Caught+fixed the abra-init race
and the non-deterministic Drone bot token en route.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Clean-room finding caught by the e2e: DRONE_USER_CREATE had no token: => a fresh-DB rebuild's Drone
auto-generates a random bot token, so the committed (sops) bridge_drone_token gets 401 and the bridge
can't trigger builds. The original cc-ci only matched because its token was captured out-of-band. Now
the bot's machine token == bridge_drone_token deterministically on every rebuild. (Evolves the toplevel
again; re-establish byte-identical on cc-ci after the e2e + Adversary re-verifies C1.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Found during the e2e: when the bot already granted the shared Drone OAuth app, Gitea 302s straight to
the code callback (no consent form), so the consent-form parse yielded empty _csrf/state and set -e
aborted. Now: if authorize returns a Location, use it directly; else POST the consent form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On a FRESH host the reconcile oneshots ran abra concurrently against an uninitialised ~/.abra and
raced on catalogue/recipe init, leaving deploy-proxy/deploy-drone failed after a blank-VM rebuild
(observed on the W4 throwaway). Ordering-only `after` chain serializes them so a single
nixos-rebuild switch converges. Logically correct too (all need the proxy/abra state first).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cc-ci /var/lib/sops-nix/key.txt provisioned = host-derived age key (pub == &host recipient), so
adding keyFile is safe (sops-install-secrets aborts if a configured keyFile is missing).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- secrets/ is now the private cc-ci-secrets repo (submodule). defaultSopsFile path unchanged.
- secrets.nix: add wildcard_cert/wildcard_key sops secrets -> path=/var/lib/ci-certs/live/*.
- proxy.nix: cert is sops-from-git, not an operator file drop (reframed; FATAL guard kept as decrypt-path check).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnosed via instrumented diag: lasuite-docs upgrade reported 'FATA deploy failed' while all 9
services converged 1/1 — abra's convergence poll gives up too early on the slow stop-first roll
(pulling new images). Disable abra's check; the harness wait_healthy + data-survival assertion is
the real, more-patient gate (a genuine failure still fails the test: app never gets healthy).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
architecture.md: components, the !testme flow, network/TLS, resource safety, enrollment.
runbook.md: where to look, common failure modes (timeout/rate-limit/auth/skip/health/data), orphan
cleanup, re-trigger, cancel. Completes the D9 doc set (README+install+enroll+secrets+arch+runbook).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
git fetch --tags <url> without a refspec errors 'couldn't find remote ref HEAD'; use
'refs/tags/*:refs/tags/*'. Verified: brings custom-html's 18 upstream version tags into the mirror
PR clone so the upgrade stage finds a previous published version (was skipping).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fetch_recipe (SRC+REF/PR path) now read-only fetches published version tags from the public upstream
into the mirror clone, so the upgrade stage finds a previous published version (mirror PR branches
carry no tags → upgrade would skip). Guardrail-safe: only fetches tags, never pushes to the recipe
repo; plain git so the bot token isn't sent to upstream. Adds the 6 D10 recipes to the bridge
POLL_REPOS so !testme on their PRs triggers runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After triggering a build, the bridge spawns a watcher thread that polls the Drone build to
completion and edits its run-link PR comment to ✅ passed / ❌ <status> (Gitea PATCH
issues/comments/{id}, verified). post_comment now returns the comment id. Also gives the bridge
image a content-hash tag so the swarm service actually rolls on bridge.py changes (was stuck on
:latest). Completes the D7 'PR comment reflects outcome' requirement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
docs/secrets.md documents the 3 secret classes (A1 external, A2 internal-generated, B recipe-app),
the sops-nix decryption chain, and rotation procedures for each (cert version bump, sops re-encrypt +
swarm-secret version bump, recipe-app ephemeral). run_recipe_ci streams each stage's output through a
redaction filter that masks any /run/secrets/* value (>=8 chars) before it reaches Drone logs —
belt-and-suspenders over 'harness never prints secrets + abra doesn't echo'. Live streaming + exit
code preserved (locally tested). Recipe-ci clones cc-ci fresh per build, so this applies next run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stdlib HTTP service (like the bridge): polls the Drone API for recipe-CI builds (event=custom),
groups latest-run-per-recipe, renders a YunoHost-CI-like overview table with pass/fail/running
badges + links to the canonical Drone run, plus /badge/<recipe>.svg. Nix-built OCI image, swarm
service on proxy, traefik Host(ci.commoninternet.net) (the bridge's /hook rule stays higher
priority by length). Reuses the Drone token (read-only). Reconcile oneshot like bridge/drone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
9-service stack (frontend/backend/celery/y-provider/docspec/postgres/redis/minio/nginx) converges
9/9 and serves the SPA; install 2 passed on host. Root-caused a deploy timeout: cold-pulling ~9
large images exceeds abra's default 300s convergence TIMEOUT -> bumped to 900 via EXTRA_ENV (the
generic per-recipe mechanism, no harness surgery). upgrade/backup use a postgres marker (docs/docs)
exercising the pg_backup.sh DB-dump hook; verifying next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
install 2 passed on host (~2.7m): synapse client API 200 + real versions JSON, no extra config
(SYNAPSE_SERVER_NAME=DOMAIN). upgrade/backup author postgres-marker assertions exercising the
recipe's pg_backup.sh dump/restore hook (the meaningful matrix data path); verifying next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
backup-bot-two's .env.sample ends with a newline-less comment, so set_env's bare
append concatenated RESTIC_REPOSITORY onto it (commenting it out). The backupbot
container then lacked RESTIC_REPOSITORY and 'abra app backup create' KeyError'd —
breaking the backup stage for recipes without a custom backup hook (cryptpad).
set_env now ensures a trailing newline before appending (applied to drone.nix too,
same latent bug). Re-verify keycloak backup, which earlier passed off an older deploy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a shared-harness EXTRA_ENV mechanism (recipe_meta.py dict or domain-callable),
applied in deploy_app at every deploy path — no per-recipe harness surgery (D5).
cryptpad uses it for its required distinct SANDBOX_DOMAIN. Tests assert data
survival via a marker file in the backed-up cryptpad_data volume (exec_in_app,
since cryptpad data isn't HTTP-served).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes surfaced by the first real recipe-ci run through Drone:
- abra app backup/restore now pass -C -o (current checkout, no remote fetch) like
every other recipe-touching call — without -o they fetch recipe tags from the
(private) remote and fail 'authentication required: Unauthorized'.
- fetch_recipe's catalogue path rm's the recipe dir first so a leftover private-mirror
remote from a prior SRC+REF run can't poison version resolution / backup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The exec runner sets HOME to a per-build workspace, leaving ~/.abra empty
(FATA directory is empty: .../home/drone/.abra/servers). Force HOME=/root in the
step so abra and the harness's ~/.abra/recipes resolve to the real config, as the
manual runs did. Safe at capacity=1 (no concurrent build shares /root/.abra).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits .drone.yml into a push-triggered self-test pipeline and a custom-triggered
recipe-ci pipeline. The bridge fires event=custom builds with RECIPE/REF/PR/SRC
params; recipe-ci runs the shared harness (install/upgrade/backup + recipe-local)
with STAGES set and CCCI_JANITOR_MAX_AGE=0 (safe at capacity=1), concurrency limit 1.
Connects the verified !testme trigger to actual recipe CI (D2/D10 path).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bound live test apps on the single 28GiB node. DRONE_RUNNER_CAPACITY=1 (MAX_TESTS)
caps concurrent builds; Drone auto-queues the rest natively. deploy-drone reconcile
sets the cc-ci repo build timeout to 60m (best-effort PATCH, non-fatal) so a hung
build is killed and frees its slot. Janitor remains the backstop for SIGKILL'd builds.
Verified on host: DRONE_RUNNER_CAPACITY=1; repo timeout=60 via Drone API.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Polling is now the primary, read-only trigger (always-on thread); the /hook
webhook is an optional admin-registered push optimization deduped by comment id.
Authorize commenters via GET /orgs/{owner}/members/{user} (204, read-level) +
optional allowlist, replacing the admin-requiring /collaborators permission
endpoint. Bot never self-registers webhooks. Enroll = POLL_REPOS + tests/<recipe>/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
D4 snapshots recipe-shipped tests/ and runs them against the live app. abra -C -o
everywhere + token clone for private mirror PRs. keycloak install green with no
harness surgery (D5). docs/enroll-recipe.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
keycloak+mariadb deployed via only tests/keycloak/recipe_meta.py + test_install.py
(realm health + Playwright admin login). Proves recipe-agnostic enrollment (D5).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Recipe-agnostic harness (no surgery to enroll a recipe): recipe_meta.py for
health path/codes/timeouts; run_recipe_local discovers + runs recipe-shipped
tests/ against the live app. install non-regressed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
