autonomic-bot
f7971d949d
Operator (2026-05-29): on one host Docker's local image store already IS the cache; the churn was over-pruning, not a missing cache. So 2pc = conservative prune policy + confirm local-store retention + daemon auth (PC1-3). Registry pull-through cache deferred to IDEAS with a concrete revisit condition (multi-node, or measured cold-deploy bottleneck on recreate-surviving storage). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4.5 KiB
cc-ci Phase 2pc — Sane image-prune policy (retain Docker's local image cache)
Status: ACTIVE — a small interjection into Phase 2 (operator, 2026-05-29). Phase 2
(plan-phase2-recipe-tests.md) is PAUSED at its current progress (STATUS-2/BACKLOG-2 preserved);
the loops do this short phase, then Phase 2 resumes automatically.
Transition: auto — on ## DONE in machine-docs/STATUS-2pc.md the watchdog returns to Phase 2.
Owner: Builder + Adversary loops. This file: /srv/cc-ci/cc-ci-plan/plan-phase2pc-image-cache.md
Phase order: … 1e → 2w → 2(paused) → 2pc → 2(resume) → 2b → 3 → 4.
Why (and why a separate registry cache is NOT in scope)
Image handling is the demonstrated hot spot: the Docker Hub rate-limit was hit twice, and
docker image prune -af (run to free disk) wiped cached images mid-run → forced a full cold
re-pull of 12 images → rate limit (JOURNAL-2).
But the root cause was over-pruning, not lack of a cache. On this single host, Docker's own local image store already IS the cache — a pulled image stays, and re-deploys (cold tests, warm canonical, reboots) reuse the local layers with no re-download; the daemon is PAT-authenticated, so the residual per-deploy manifest checks sit comfortably under the 200/6h per-account budget. So keeping the local store (stop aggressive pruning) recovers ~all the benefit a cache would give.
A separate registry:2 pull-through cache is deliberately OUT of scope here — its distinctive
wins don't apply to a single authenticated, non-pruning host: multi-node fan-out (we have one
node), surviving prune/VM-rebuild on separate storage (ours would be co-located, lost on a
recreate anyway), and cache-miss authentication (the daemon is already authenticated). It would add a
registry service + daemon-mirror config + cache GC for marginal gain. Deferred to IDEAS / Phase 2b
with a concrete revisit condition (see Guardrails).
Definition of Done (Adversary cold-verifies → machine-docs/REVIEW-2pc.md)
- PC1 — Conservative prune policy. Stop reflexive
docker image prune -af. Never prune during a deploy/test run. Keep base/in-use images. Prune only truly-orphaned, old layers (dangling + age-gated) and only under genuine disk pressure (now bounded — host 70 GB, ~43 G free). Wherever the harness/janitor/CI prunes today, make it surgical, not-af. The per-run teardown must keep removing the run's app volumes/secrets/services (sacred) but must NOT remove images. - PC2 — Local cache retained + authenticated (confirm). Confirm the Docker daemon stays
PAT-authenticated for
docker.ioand that the local image store is retained across runs, teardowns, and reboots — so a repeat deploy of a previously-pulled image reuses local layers (no re-download) and makes at most an authenticated manifest check. - PC3 — Verified + documented. Adversary proof: deploy a recipe, tear it down, redeploy
→ the redeploy does not re-download image layers (served from the local store; show via
dockerevents/pull output / a measured pull-time drop), and a normal run no longer evicts cached base images while disk stays bounded without-af.docs/notes the prune policy; deviations inDECISIONS.md.
When PC1–PC3 hold and are Adversary-verified, write ## DONE to machine-docs/STATUS-2pc.md →
watchdog auto-returns to Phase 2.
Guardrails / constraints
- Bounded scope — prune policy + confirm local-store retention/auth ONLY. Do NOT build a registry pull-through cache here, and do NOT expand into concurrency/readiness-tuning/dedup (those are measurement-driven Phase 2b).
- Real pull path — no special-casing pulls; abra/swarm pull through the normal authenticated daemon.
- Don't weaken any test — the retained cache must not mask a genuinely-broken image (pinned versions still resolve correctly; a real new digest still pulls).
- Registry pull-through cache — DEFERRED (IDEAS / Phase 2b), revisit ONLY if: (a) cc-ci ever goes multi-node, OR (b) Phase-2b measurement shows cold-cache / fresh-deploy pull time is a real bottleneck (e.g. D8 throwaway-rebuild or fresh-canonical seeding) AND the cache is hosted on recreate-surviving storage (an Incus volume / a path on host b1, not the VM's ephemeral disk). Otherwise it's complexity without payoff on a single host.