# Plan — lasuite-drive OIDC-setup flakiness: harness restructure + recipe robustness PR **Status:** QUEUED — picks up the deferred lasuite-drive `[~]` item (Q3.2) when the loops return to it. The disk blocker is now lifted (host grew to 70 GB, 2026-05-29), so the upgrade tier can run too. **Owner:** Builder + Adversary loops. **Two deliverables:** (A) a cc-ci harness change, (B) a **lasuite-drive recipe PR** (we maintain the recipe via recipe-maintainer). **This file:** `/srv/cc-ci/cc-ci-plan/plan-lasuite-drive-oidc-robustness.md` --- ## 0. The problem (ground it in real logs FIRST) lasuite-drive's `setup_custom_tests` wires OIDC by doing a **full 12-service `abra app deploy --chaos` redeploy** of an already-running heavy stack. That reconverge is **flaky**: a collabora reconverge race + a **transient backend gunicorn-perms / WOPI-404 window**. Only **backend/app** consume the OIDC env, so re-converging collabora/onlyoffice/minio/db is unnecessary exposure. The Builder correctly did NOT claim Q3.2 (a flaky setup isn't a reliable green) and filed this. **Step 0 (do before fixing):** capture the actual failure from a flaky run — collabora WOPI-discovery timing, the backend log at the 404, and the exact gunicorn perms error — so we fix the real root cause, not a guess. Record in JOURNAL. --- ## Part A — cc-ci harness: wire OIDC at INSTALL, eliminate the redeploy Leverage the now **live-warm keycloak** (WC1): instead of deploy-recipe-then-redeploy-with-OIDC, the dep is already running, so configure OIDC **before the single deploy**: 1. Create the per-run namespaced realm+client in the **warm keycloak** (lightweight API calls — no deploy). 2. Set lasuite-drive's OIDC env in its `.env` **before** the first `abra app deploy`. 3. Deploy lasuite-drive **once**, OIDC already wired. **No mid-run `--chaos` reconverge** → the flaky window is gone. - **Use REAL abra commands only** (operator decision, 2026-05-29): the install is a normal `abra app deploy`. Do **NOT** use `docker service update`/`docker service scale` to surgically patch services — CI must exercise the same abra path a real operator uses. If any redeploy ever remains necessary, it is `abra app deploy` (+ proper readiness gating, §C), never a docker-level bypass. - **Generic-first invariant still holds.** keycloak is live-warm (persistent infra, ~always up), so there's no per-run dep *deploy* that could fail and break a generic tier. Generic tiers still assert serving recipe-alone; if realm creation or the provider is unreachable, only the OIDC *custom* test fails (isolated). This is SAFE **iff** the recipe boots with OIDC-env-set even when the provider isn't reachable at boot — which Part B guarantees (lazy OIDC). A and B reinforce each other. - **Revises `plan-sso-dep-testing.md`** for recipes whose app tolerates OIDC-at-boot: prefer install-time wiring against the warm dep over the post-deploy `setup_custom_tests` redeploy. Keep the redeploy path only for recipes that genuinely can't take OIDC env at first boot. - **Verify:** lasuite-drive full suite green **without** a mid-run reconverge; run it repeatedly (e.g. 3×) to show the flakiness is gone, not just absent once. ## Part B — lasuite-drive recipe PR (root-cause robustness, we're maintainers) > **Broken out into its own sub-plan: `plan-lasuite-drive-recipe-pr.md`** (operator, 2026-05-29) — > it now also folds in the F2-12 upgrade-convergence fix (collabora healthcheck) so cc-ci can return > to abra-native convergence. The summary below is retained; the sub-plan is authoritative. Fix the fragility in the recipe itself so it's robust under ANY reconverge — this helps real operators, not just CI, and is the real payoff of maintaining the recipe. On a branch of the lasuite-drive recipe (recipe-maintainer): 1. **Order backend behind collabora's WOPI readiness.** Add a Docker **healthcheck on collabora** (WOPI discovery endpoint → 200) and make backend **wait for it / retry WOPI discovery with backoff** instead of failing during collabora startup. Turns the race into deterministic ordering. 2. **Fix the gunicorn-perms race.** The transient perms error is a volume-permission race at startup (gunicorn can't read/write a mount until perms are set). Fix in the entrypoint: set perms before exec'ing gunicorn (or a proper init step). 3. **Make OIDC discovery lazy / retrying.** If backend validates the OIDC provider eagerly at boot and dies when it's briefly unreachable, that's what forces the deploy-with-dep dance. Resolve OIDC discovery lazily (at first login, with retry) so the app boots without the provider up. (This is also the property Part A relies on for the generic-first invariant.) ### The PR is "working" ONLY when cc-ci verifies it green (operator rule, 2026-05-29) A lasuite-drive recipe change is **not** considered done until **our CI server has run the full suite on the PR and it passes** — install + upgrade + backup + restore + custom (incl. the OIDC login test), repeatedly-green (no flakiness). Concretely: push the recipe branch/PR, trigger cc-ci (`!testme` on the lasuite-drive PR), and require a clean green (Adversary cold-verified). **Only then does the operator merge** the recipe PR. This dogfoods cc-ci: the CI that surfaced the bug also gates its fix. ## C. If any redeploy must remain (fallback only) If a recipe genuinely needs OIDC env applied after first boot, do it with **`abra app deploy`** (real abra — never `docker service update`), and **gate readiness properly**: poll collabora WOPI = 200 + backend health with retries/timeout before declaring ready. This is strictly a fallback; Part A + Part B should remove the need. ## Definition of done - [ ] Step 0 root-cause logs captured + recorded. - [ ] Part A: lasuite-drive tested via install-time OIDC wiring against the warm keycloak, **no mid-run reconverge**, full suite green **3× in a row** (flakiness gone), Adversary cold-verified. - [ ] Part B: lasuite-drive recipe branch with the WOPI-healthcheck-gating + gunicorn-perms + lazy-OIDC fixes; **cc-ci runs the full suite (incl. upgrade tier) on the PR and it is repeatedly green**; Adversary cold-verifies; operator merges. - [ ] Q3.2 (lasuite-drive) claimed + DEFERRED.md entry closed (flaky-OIDC item) — only after the above. - [ ] `plan-sso-dep-testing.md` updated to prefer install-time OIDC wiring (warm dep) where the recipe supports it.