From aae31775ae0d16fb81050ccfc51e4441c7096b23 Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Thu, 28 May 2026 21:18:50 +0100 Subject: [PATCH] status(2): Gitea outage resolved + git reconciled; Docker Hub rate-limit block stands (registry-creds finding) Co-Authored-By: Claude Opus 4.8 (1M context) --- machine-docs/STATUS-2.md | 41 +++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/machine-docs/STATUS-2.md b/machine-docs/STATUS-2.md index 0c446bf..d087c49 100644 --- a/machine-docs/STATUS-2.md +++ b/machine-docs/STATUS-2.md @@ -245,29 +245,32 @@ ssh cc-ci 'cd /root/cc-ci && cc-ci-run -m pytest tests/unit -v && RECIPE=custom- ``` ## Blocked -**@2026-05-28 ~19:45Z — two concurrent EXTERNAL (Class A1) infra blocks; operator notified.** +**@2026-05-28 ~21:10Z — ONE standing EXTERNAL (Class A1) block: Docker Hub pull rate limit.** +(The earlier Gitea outage is RESOLVED — see below — and git state is reconciled/pushed.) -1. **Docker Hub anonymous pull rate limit (registry creds finding, plan §1.5).** All docker.io - pulls from cc-ci's IP now fail with `toomanyrequests: You have reached your unauthenticated pull - rate limit`. Verify: `ssh cc-ci 'docker pull minio/minio:RELEASE.2025-09-07T16-13-09Z'` → - rate-limit error. Traced to: today's many recipe deploys + a `docker image prune -af` (run to - clear a disk-full that broke the lasuite-drive deploy) forcing a full cold re-pull. This blocks - **every** new recipe deploy (all pull from docker.io). Per §1.5 this is a finding → **request - registry pull credentials** (authenticated/Team Docker Hub, or a pull-through cache). Recurs for - all remaining Q3.5/Q4 enrollments. Self-resolves partially as the rolling 6h window ages out. +**Docker Hub anonymous pull rate limit (registry-creds finding, plan §1.5).** docker.io pulls from +cc-ci's IP fail with `toomanyrequests: You have reached your unauthenticated pull rate limit`. Verify: +`ssh cc-ci 'docker pull redis:8.6.3'` → rate-limit error. After the Gitea outage I re-tested: exactly +**1** pull (minio) trickled through as the rolling 6h window aged, then the next 3 (redis/nginx/ +mailcatcher) hit the limit again — so the quota is still effectively exhausted, dribbling ~1 pull at a +time. Traced to: today's many recipe deploys + a `docker image prune -af` (run to clear a disk-full +that broke the drive deploy) forcing a full cold re-pull. Blocks **every** new recipe deploy. Per §1.5 +this is a finding → **request registry pull credentials** (authenticated/Team Docker Hub, or a +pull-through cache). Recurs for all remaining Q3.5/Q4 enrollments. Operator notified @~19:45Z. -2. **Gitea (git.autonomic.zone) outage.** Bare `/`, unauth `/api/v1/version`, and authed repo API - all return plain `404 page not found` (Go ServeMux default → backend down). Same from my sandbox - AND cc-ci (IP 116.203.211.204) — a real instance outage, not creds/path. Verify: - `curl -s -o /dev/null -w '%{http_code}' https://git.autonomic.zone/api/v1/version` → 404. - Blocks all push/pull → **coordination is down**: two watchdog pings (REVIEW-2 update + - BUILDER-INBOX.md) are unconsumable until Gitea recovers. Local commits queued; will push + process - the Adversary's messages the instant it's back. +Impact on Q3.2 lasuite-drive: base deploy got 8/12 services up (incl. heavy onlyoffice+collabora; big +image LAYERS now cached on cc-ci so a re-run is light) but the last 3 small images can't pull. Will +re-run the moment pulls flow (creds or window reset). cc-ci is CLEAN (teardown verified: 0 stack, 0 +residue volumes/secrets; 6.8 GB disk + 6.5 GB RAM free). -Local build work proceeds where it needs no new pulls / no push. Loop idle-retries both ~15-20m. +**Gitea outage (RESOLVED @~21:08Z).** git.autonomic.zone returned blanket `404 page not found` for +~1.5h (backend down; same from my sandbox AND cc-ci). Orchestrator confirmed it back online; I +re-ran `git pull --rebase` (up to date) and pushed the 2 queued local commits — `origin/main` is now +`4a118ea`. The 3 watchdog pings during the outage were phantoms (Adversary's failed push retries); +the remote has NO pending BUILDER-INBOX and NO new REVIEW-2 verdict, so nothing was lost on my side. -**Prior bootstrap state (pre-outage):** access re-verified @2026-05-28: `ssh cc-ci` ok (root, NixOS -24.11), Gitea API HTTP 200, wildcard DNS resolves to gateway 143.244.213.108. +**Prior bootstrap state:** access re-verified @2026-05-28: `ssh cc-ci` ok (root, NixOS 24.11), Gitea +API HTTP 200, wildcard DNS resolves to gateway 143.244.213.108. ## Carryover from Phase 1e (not blockers for Phase 2) - **F1e-2** [adversary] — concurrent same-recipe `abra recipe fetch` race in