feat(harness): P3 per-run ABRA_DIR — structural recipe-tree isolation, recipe flock deleted
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
- run_recipe_ci.setup_run_abra_dir(): builds <runs_dir>/<run-id>/abra with servers/ and
catalogue/ symlinked to the canonical ~/.abra (app .env files keep landing in the shared
canonical path, so janitor discovery and env-based teardown are unchanged; per-domain
filenames + the P2 app-domain lock prevent write conflicts) and a FRESH empty recipes/ —
each run clones + checkouts its own recipe trees. Exported as $ABRA_DIR (honored by the
abra CLI, verified on-host) before ANY abra call. Manual runs get manual-<pid> isolation.
- fetch_recipe(): plain clone into $ABRA_DIR/recipes/<recipe> — no shared-tree rm-rf, no lock.
CCCI_SKIP_FETCH=1 now copies the canonically-staged clone into the per-run tree (same staging
workflow, run reads staged state).
- abra.abra_dir()/recipe_dir(): single resolution rule ($ABRA_DIR else ~/.abra), used by
recipe_checkout, has_lightweight_version_tags, recipe_head_commit, recipe_versions,
generic._recipe_dir, lifecycle.prepull_images, snapshot_recipe_tests, and
warm_reconcile._recipe_dir (which keeps the canonical default for its own systemd runs but
follows the per-run tree when imported by promote_canonical inside a run).
- deleted: lifecycle.acquire_recipe_lock, RECIPE_LOCK_DIR, the main() call site and the
must-lock-before-fetch ordering rule.
- tests/{ghost,discourse}/install_steps.sh: RECIPE_DIR resolves ${ABRA_DIR:-$HOME/.abra} so the
compose.ccci.yml overlay lands in the tree the run actually deploys from (mechanical path fix
required by per-run trees; no assertion/gate touched — see DECISIONS.md).
- .drone.yml comments updated (HOME=/root rationale now via the servers symlink).
This commit is contained in:
@ -37,31 +37,9 @@ class TeardownError(RuntimeError):
|
||||
# however it dies. The janitor probes the lock (LOCK_NB) to tell a live concurrent run (held →
|
||||
# leave it) from a crashed run's orphan (acquirable → reap it); it never inspects pids and never
|
||||
# steals a held lock. Recipe-tree corruption between same-recipe runs is gone structurally (each
|
||||
# run deploys from its own per-run ABRA_DIR), and same-domain runs (double-!testme of one PR)
|
||||
# serialise on this app lock. See docs/concurrency.md.
|
||||
RECIPE_LOCK_DIR = "/run/lock"
|
||||
|
||||
|
||||
def acquire_recipe_lock(recipe: str):
|
||||
"""Per-recipe exclusive lock serialising same-recipe runs on the shared ~/.abra/recipes
|
||||
checkout. P3 of the restructure deletes this (per-run ABRA_DIR makes the shared tree, and
|
||||
with it this lock, structurally unnecessary); until then the caller keeps the returned file
|
||||
alive for the whole run and release is implicit at process exit."""
|
||||
path = os.path.join(RECIPE_LOCK_DIR, f"cc-ci-recipe-{recipe}.lock")
|
||||
# PEP 446: the fd is non-inheritable, so subprocess children never carry the lock.
|
||||
f = open(path, "w") # noqa: SIM115 — deliberately held for the lifetime of the run
|
||||
try:
|
||||
fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except BlockingIOError:
|
||||
print(
|
||||
f"== recipe lock: another {recipe} run is in flight — waiting for {path} "
|
||||
"(shared ~/.abra/recipes checkout) ==",
|
||||
flush=True,
|
||||
)
|
||||
fcntl.flock(f, fcntl.LOCK_EX)
|
||||
print(f"== recipe lock: acquired {path} ==", flush=True)
|
||||
return f
|
||||
|
||||
# run deploys from its own per-run ABRA_DIR — there is no shared recipe tree and no recipe lock),
|
||||
# and same-domain runs (double-!testme of one PR) serialise on this app lock.
|
||||
# See docs/concurrency.md.
|
||||
|
||||
# Acquired app-lock file objects are retained here for the REMAINING PROCESS LIFETIME: if the
|
||||
# caller drops the returned file object, GC would close the fd and silently release the lock —
|
||||
@ -209,9 +187,9 @@ def prepull_images(recipe: str, domain: str) -> None:
|
||||
app-INIT time (slow-init apps like collabora/immich still need their recipe healthcheck/READY_PROBE).
|
||||
Best-effort on resolution failure (skip + let the deploy pull as usual); HARD-fails on a real
|
||||
pull error (don't mask it)."""
|
||||
import os
|
||||
|
||||
recipe_dir = os.path.expanduser(f"~/.abra/recipes/{recipe}")
|
||||
recipe_dir = abra.recipe_dir(recipe) # per-run tree inside a CI run
|
||||
# The app .env lives in the CANONICAL servers path (the per-run ABRA_DIR's servers/ is a
|
||||
# symlink to it, so abra and this path agree on the same file).
|
||||
env_path = os.path.expanduser(f"~/.abra/servers/default/{domain}.env")
|
||||
if not os.path.isdir(recipe_dir) or not os.path.isfile(env_path):
|
||||
print(f" prepull: recipe dir or .env missing for {recipe} — skipping", flush=True)
|
||||
|
||||
Reference in New Issue
Block a user