The first cut of this fix added a team INSERT next to the existing sites INSERT.
That fixed the symptom and kept the cause: a fixture that knows the apps table
layout breaks whenever the app changes it, which is exactly what happened here.
_register_site now calls Plausible.Sites.create/2 through the app release console,
so the app provisions whatever its data model currently requires - including the
team it introduced in v3. Verified against BOTH versions on cc-ci: the identical
expression works on v2.0.0, which has no teams table at all, and on v3.2.1. No
version gate is needed because the fixture no longer depends on the schema.
The HTTP provisioning API (POST /api/v1/sites) would have been first choice, but
it is gated behind a paid plan and answers :upgrade_required on CE. That is
recorded in the code so the next person does not re-derive it.
tests/STYLE.md writes the rule down, along with the others this failure exercised:
gate on version rather than supporting both schemas (old-version tests can just be
deleted - the older version is only exercised through the upgrade tier); correct
the fixture or the wait but never the assertion; assert stored state rather than a
202 ack; size waits from the recipes declared readiness; and read the apps own
telemetry before deciding a test is stale.
Full cold suite against the recipe PR head: level 5 of 5, GREEN.
plausible v3 (community-edition) only ingests events for a site that belongs to
a TEAM. The custom tier registered a site row and nothing else, which was enough
for v2 — under v3 the POST still acks 202 and the row still exists in postgres,
but every event is discarded. ClickHouse records the reason itself in
ingest_counters as dropped_not_found, and events_v2 stays empty, so it presents
as a silent ingestion stall.
Verified on cc-ci against v3.2.1: identical site row with no team ->
dropped_not_found and 0 rows; with a team linked -> buffered and the rows land.
_register_site now provisions a team and links the site, guarded on the schema
actually having teams so it stays a no-op on v2 (the upgrade tier deploys the
older base first).
Separately, the custom health check waited 60s for /api/health. That tier runs
after backup/restore, which disrupts postgres under the app and restarts it, and
v3 boots through sleep 10 + createdb + migrate + cache warmers before health
flips to 200. Widened to 300s, still far inside the recipe HTTP_TIMEOUT of 1200.
The assertion is unchanged: a hard 200 from the real readiness endpoint.
Neither change weakens a test - the event tests still require the row to arrive
in ClickHouse and match what was sent.
test_head_runs_official_image_not_bitnamilegacy hardcoded the migration-era pin
discourse/discourse:3.5.3 and went stale on the first legitimate app bump
(2026.7.1, weekly 2026-08-03 — caught by verify run 2: the upgrade converged,
head image was discourse/discourse:2026.7.1, only the frozen pin failed). The
guarded property is the image FAMILY (official vs bitnamilegacy), not a frozen
version — now asserts the discourse/discourse: prefix. Not weakened: the
bitnami-leak check + official-prefix check together still assert exactly the
migration faithfulness; the concrete head pin is exercised by the deploy.
The 0.8.x->1.0.0 discourse recipe family switched app bitnami->official AND db
pgvector/pg17->discourse/postgres:pg18. That db-family change is a structural
break (bitnami cluster lacks the discourse role; pg_upgrade preserves-not-creates
roles) with NO supported in-place path. The dynamic base resolver's step-back
kept selecting 0.8.1+3.5.0 (newest tag below the unbumped 1.0.0+3.5.3 label) and
the upgrade tier red'd twice on this unsupported path (drone #1165, #1171 —
classified stale-test both times; recipe verified green on the real
official->official path).
Adds UPGRADE_BASE_FLOOR (phase basefloor) to resolve_upgrade_base: a recipe_meta
declaration naming the first post-break published version. Resolution stays
fully dynamic (this is NOT the removed sec2.G static pin): the floor only
EXCLUDES below-floor candidates (canonical, step-back, no-canonical fallback);
when no >=floor predecessor exists the tier records a DECLARED skip, never a
silent pass. main-tip fallback unaffected (post-break by construction).
tests/discourse/recipe_meta.py declares UPGRADE_BASE_FLOOR = 1.0.0+3.5.3 with
the full rationale. Unit-verified: head=1.0.0+3.5.3 -> declared skip (was:
0.8.1+3.5.0 wrong pick); post-release head=2026.x -> base 1.0.0+3.5.3 (the real
migration path). No assertion weakened - below-floor in-place upgrades were
never supported coverage.
Enrolls wordpress as a maintained recipe (operator request 2026-08-03):
- tests/wordpress/: recipe_meta (install-wizard-aware health 200/302, 900s deploy
timeout for mariadb+core-copy first boot, WARM_CANONICAL), custom suite:
health check, install-wizard completion + REST API round-trip (?rest_route= vs
/wp-json/ splits DB vs .htaccess failure layers), and the sec4.3 post round-trip
(XML-RPC write -> REST read -> permalink HTML, unique marker). PARITY.md documents
the baseline (no recipe-maintainer parity corpus for wordpress).
- nix/modules/bridge.nix: POLL_REPOS += recipe-maintainers/wordpress (!testme bridge
enrollment; deploy to the cc-ci host follows separately after the in-flight
/upgrade-all run - test-before-switch policy).
Mirror recipe-maintainers/wordpress created + main synced to coopcloud upstream
(adcd0e9f) with published tags. used-recipes.md gains 'wordpress weekly' in the
orchestrator repo.
test_create_room_get_livekit_token_and_read_back authenticated with a raw OIDC user
access token as 'Authorization: Bearer'; meet v1.22.0 hardened API auth to reject
user access tokens (release notes: 'reject user access tokens on the API'), so the
test went RED with 401 on the v1.24.0 upgrade (drone build #1137; same at v1.23.0
in build #1122).
Updated to the successor auth path: recipe-local _oidc_session.py (same helper as
tests/lasuite-docs) drives the real OIDC authorization-code flow (app -> keycloak
login form -> callback -> Django session cookie, CSRF on unsafe methods).
- NEW assertion: a raw OIDC Bearer token is REJECTED (401/403) - the v1.22.0
hardening asserted as the new correct behavior.
- The full meeting flow (create 201 + LiveKit JWT grant, read-back, DELETE) is
unchanged, now over the session-authenticated API. No assertion weakened.
Stale-test fix for recipe PR
recipe-maintainers/lasuite-meet#8
(carry-over from /upgrade-all 2026-07-24).
test_oidc_login_via_keycloak and test_create_doc_and_read_back authenticated with
'Authorization: Bearer <password-grant JWT>'; impress v5.4.0 removed Bearer/JWT
auth on the API (SessionAuthentication only), so both went RED with 401 on the
v5.4.1 upgrade.
Updated to the successor auth path: a new recipe-local _oidc_session.py drives the
real OIDC authorization-code flow (app -> keycloak login form -> callback ->
Django session cookie, with CSRF headers on unsafe methods).
- test_oidc_login: still asserts the unauth challenge redirect; NOW also asserts a
raw Bearer JWT is REJECTED (401/403 - the v5.4.0 hardening, asserted as the new
correct behavior); then asserts the session-authenticated whoami returns the
provisioned user. No assertion weakened - the auth proof is stronger than before.
- test_create_doc: same create+read-back round-trip assertions, now over the
session-authenticated API.
Stale-test fix for recipe PR
recipe-maintainers/lasuite-docs#7
(carry-over from /upgrade-all 2026-07-24).
history_for() now enumerates run dirs' results.json, groups by recipe, sorts
newest-first by finished timestamp (mixed numeric+named ids — timestamp is the
only correct key), caps at HISTORY_CAP=30, skips malformed/empty/no-recipe dirs.
Overview + badges + /runs + security guards + stdlib-only unchanged.
Local verify: 13/13 unit tests; full-fixture vs 308 real results.json →
bluesky-pds=8 in exact ts order, plausible capped 30 newest, edge dirs skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
plausible's canonical is established at 3.1.0+v2.0.0 (latest), so the dynamic resolver no longer
needs the explicit pin: a same-version head steps back to newest-older = 3.0.1+v2.0.0 (NOT the
broken 3.0.0). Verifying live before stripping the key globally (§2.G gate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
keycloak is the always-on shared OIDC dep provider at warm-keycloak.ci..., the SAME stable domain a
data-warm canonical would use → the sweep's promote would collide with the live provider that
lasuite-*/drone depend on. keycloak is kept current by roll_warm_infra (WC1.1) instead.
WARM_CANONICAL=False; exception recorded in DECISIONS. Enrolled set now 20.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WARM_CANONICAL=True added to every recipe in cc-ci-plan/used-recipes.md (20 weekly +
uptime-kuma external). enrolled_recipes() now returns all 21. Test fixtures
(custom-html-*-bad, concurrency, regression) intentionally left unenrolled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- should_promote_canonical gains a `tagged` requirement (canon §2.A): a green cold
latest run promotes only when the tested head version is a published release tag;
an untagged main commit never becomes a canonical.
- warm_reconcile.is_released_version(recipe, version): release-tag membership (exact or
by version_key). Caller computes `tagged` so the gate stays pure.
- unit tests: untagged -> no promote; is_released_version cases.
- drive-by (pre-existing reds, unrelated to canon, now green): test_warm_reconcile
traefik assertion was stale vs the phase-pxgate spec (probes /api/version, no
health_domain); meta.py UPGRADE_BASE_VERSION KEYS help synced to the prevb doc text.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
resolve_upgrade_base now reads the head's published version (abra.head_compose_version,
the coop-cloud.<stack>.version label) and, when the last-green warm-canonical version
equals it, steps back to the newest published version strictly older than head instead
of deploying a same-version no-op. warm_reconcile gains version_key + newest_older_version
(single coop-cloud ordering source; sort_versions refactored onto version_key, no behavior
change). Skip only when no older published predecessor exists. Step-back returns kind=version
so it inherits F1d-2 pinned-tag checkout. Extends tests/unit/test_upgrade_base.py (13 pass).
The custom tier runs on the PR head — now genuinely the official discourse/discourse image (prevb
stopped the overlay reverting it to bitnamilegacy). mint_admin hardcoded /opt/bitnami/discourse (404 on
official) → create-topic roundtrip failed. Detect /var/www/discourse, re-export DISCOURSE_DB_PASSWORD
from /run/secrets (entrypoint exports it only for boot), run bin/rails; keep bitnami fallback.
docker stack deploy doesn't prune services the head compose dropped (discourse PR#4 drops sidekiq),
leaving them orphaned on the base image. perform_upgrade now reconciles the live stack to the head
compose service set (lifecycle.prune_orphan_services). Makes the deployed stack faithfully reflect
the head — no test weakened. No-op when service sets match / compose unresolvable.
abra does NOT write STACK_NAME to the app's .env file — it derives it at runtime
by replacing dots with underscores (e.g. gite-e1cb78.ci.commoninternet.net →
gite-e1cb78_ci_commoninternet_net). Build #691 failed with 'STACK_NAME not found'
because the env file read was looking for a key that doesn't exist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Blocker 4 fix: abra `secret generate --all` uses .env.sample for length hints; the
lfs-plain-gitea PR has SECRET_LFS_JWT_SECRET_VERSION=v1 COMMENTED OUT, so abra produces
a wrong-length secret. gitea requires exactly 43 chars (32 bytes base64 URL-safe); wrong
length → gitea fatals trying to save the JWT secret to the read-only Docker Config
app.ini → health check fails → swarm rolls back.
Fix: new UPGRADE_SECRET_PREP hook (meta.py) called before `abra secret generate --all`
in the upgrade path. abra's `--all` is idempotent (skips existing secrets), so the
correctly pre-inserted secret survives. gitea's recipe_meta.py implements the hook using
`docker secret create` directly to guarantee correct format regardless of .env.sample.
Also consumes machine-docs/BUILDER-INBOX.md (Adversary Blocker 4 digest).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Blocker 1 (LFS roundtrip fails on PR #1):
- Add UPGRADE_EXTRA_ENV to gitea recipe_meta.py — after PR-head checkout
(compose.lfs.yml now in ABRA_DIR), add compose.lfs.yml to COMPOSE_FILE
and set SECRET_LFS_JWT_SECRET_VERSION=v1 so the upgrade chaos redeploy
actually runs with LFS enabled. Without this, the base install checks out
the 3.5.x tag (compose.lfs.yml removed), EXTRA_ENV sees no LFS, and the
upgrade chaos redeploy inherits the no-LFS .env — so the LFS test runs
(compose.lfs.yml is restored by recipe_checkout_ref) but LFS is off.
- Add abra.secret_generate(domain) in generic.perform_upgrade when
upgrade_env is non-empty — generates lfs_jwt_secret before chaos redeploy.
Blocker 2 (REF=main upgrade fails HC1):
- Always use recipe_head_commit (git rev-parse HEAD) for head_ref instead
of using ref directly. When ref="main" (a branch name), the HC1 commit
check "head_ref.startswith(chaos_commit)" always fails since "main" ≠ SHA.
recipe_head_commit returns the actual SHA after the fetch/checkout.
Side-fix (stale creds — build #675):
- ops.py pre_install: delete the per-domain creds file before calling
_ensure_admin. A fresh install wipes gitea's DB; any creds file from a
prior run on the same domain is stale and causes 401s in all API calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gitea 1.22+ (including 1.24.2 on cc-ci) requires explicit scopes
when creating API tokens. Add read:user + read:organization to satisfy
the token creation endpoint and the read-back assertions that follow.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Empty-repo HTTPS push with git clone exits 0 but silently fails (remote
branch creation on an empty clone is unreliable). Fix:
- Create repo with auto_init=True + default_branch=main (initial commit present)
- Clone into a non-existing subdir (git clone must target non-existing path)
- Push via explicit cred_url (bypasses remote config; no tracking needed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_git_push.py + test_lfs_roundtrip.py: use cred_url (https://user:pass@host/...)
instead of GIT_CONFIG_COUNT insteadOf rewriting, which silently failed to
propagate credentials to the push step (repo remained empty after push exit 0).
Also add GIT_SSL_NO_VERIFY=true and GIT_TERMINAL_PROMPT=0.
- test_lfs_roundtrip.py: fix restart health-poll path /api/v1/version → /version
(_api() already prepends /api/v1; double prefix produced 404 and a 120s timeout).
- nix/hosts/cc-ci/configuration.nix: add git-lfs to systemPackages (required for
the LFS capstone test on the lfs-plain-gitea PR branch).
Adversary pre-M1 findings: Issue 1 (git-lfs absent) + Issue 2 (double path) both fixed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_csrf is a hidden field; wait_for_selector defaults to state=visible
and times out. Switch to the visible username input which proves the
login form rendered.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When _enrich_deps_with_sso raises after deploy_deps succeeds (e.g., gitea API
call fails), deps_state stays {} and the finally block's `if deps_state:` guard
skips teardown, orphaning the dep at its deterministic domain.
Fix: add an `else` branch after the `if deps_state:` block that reads
$CCCI_DEPS_FILE (the legacy-list written by deploy_deps) and calls
teardown_deps on the cold entries so no dep is left running.
Unit tests: test_load_run_state_provides_fallback_for_enrichment_failure and
test_fallback_skips_warm_entries verify the data-flow that the fallback relies on.
19/19 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_scm_configured.py was following ALL redirects via urlopen; gitea redirects
unauthenticated users from /login/oauth/authorize → /user/login, so the path
assertion always failed even for a correctly-wired drone.
Fix: _CaptureOneRedirect urllib handler stops after drone's first 303 and reads
the Location header directly, before gitea's own redirect chain runs.
- Consume BUILDER-INBOX.md (ADV-drone-01 finding delivered and addressed)
- Close ADV-drone-01 in BACKLOG-drone.md
- Update test_gitea_dep.py terminology: "location_url" not "final_url"
- All 10 unit tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
level.py: RUNGS += lint; statuses {pass,fail,skip,unver}; compute_level = max passed
rung with all below pass-or-skip (fail/unver block); cap_reason/capped DELETED.
harness/lint.py: lint executor — pristine scratch clone of the per-run tree at the
exact tested ref (mirror-origin + untracked-overlay pollution solved by context, no
rule filtered), PTY via script -qec, 60s hard budget, lint.txt artifact, table-parse
classifier (rc only signals FATA), unver on any non-run (never silent pass).
results.py: derive_rungs classifies every N/A source (structural/declared → skip,
else unver), lint rung + synthetic lint stage + lint block in results.json, schema 2,
cap fields removed. run_recipe_ci.py: lint call before tiers (double-wrapped,
verdict-neutral), badge = level only. card/dashboard: 0-5 ramp, cap line → 'level N
of {4|5}', unverified rows, badge number+colour only, lint.txt servable, old schema-1
artifacts render untouched. Unit suite rewritten: 245 passed on cc-ci venv.
cc-ci
autonomic-bot
