recipe-maintainers/cc-ci
0
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

upgrade base: prefer the newest published release over a stale canonical #16

Open
autonomic-bot wants to merge 1 commits from test/upgrade-base-prefer-newest-release-20260810 into main
pull from: test/upgrade-base-prefer-newest-release-20260810
merge into: :main
Conversation 0 Commits 1 Files Changed 1
+30
autonomic-bot commented 2026-08-10 18:16:49 +00:00
Owner
Copy Link
Copy Source

Upgrade base: prefer the newest published release over a stale canonical

The canonical is last-green, not current. Its promotion (WC5) deploys a warm-<recipe> app and
health-checks it — when that fails the canonical stays pinned at an old release indefinitely, and
nothing in the upgrade tier notices.

This is not hypothetical. The 2026-08-09 sweep logged GREEN-BUT-PROMOTE-FAILED for five
recipes — bluesky-pds, drone, gitea, lasuite-drive, lasuite-meet — so their canonicals are frozen
(gitea's at 3.5.3+1.24.2-rootless, dated 2026-06-17, because warm-gitea crash-loops on a
read-only app.ini; open recipe fix: gitea PR #4).

What it cost us

The gitea upgrade tier based on 3.5.3 and went green, while the upgrade real installs perform was
broken:

transition APP_INI_VERSION outcome
3.5.3 → head (what CI tested) v21 → v22 config renamed → Swarm creates a new object → green
3.6.1 → 3.6.2 (what users run) v22 → v22 content changed under the same name → abort: only updates to Labels are allowed

That shipped in 3.6.2+1.27.1-rootless and blocked every existing install from taking the 1.27.1
security release (CVE-2026-60004, CVE-2026-59774).

The change

When the newest published release older than head is newer than the canonical, use the release
and log why — naming the stale canonical, so the failing promote gets noticed instead of silently
degrading coverage. Resolution stays dynamic; UPGRADE_BASE_FLOOR still filters structurally
invalid candidates; the canonical remains the base whenever it is current.

Unit-verified on the real gitea tag set: base moves 3.5.3+1.24.2-rootless →
3.6.1+1.26.2-rootless — precisely the transition that reproduces the break.

Follow-up (not in this PR)

The five failing WC5 promotes are a separate bug worth fixing at the source; this change stops them
from silently weakening upgrade coverage in the meantime.

## Upgrade base: prefer the newest published release over a **stale** canonical The canonical is *last-green*, not *current*. Its promotion (WC5) deploys a `warm-<recipe>` app and health-checks it — when that fails the canonical stays pinned at an old release indefinitely, and nothing in the upgrade tier notices. **This is not hypothetical.** The 2026-08-09 sweep logged `GREEN-BUT-PROMOTE-FAILED` for **five** recipes — bluesky-pds, drone, gitea, lasuite-drive, lasuite-meet — so their canonicals are frozen (gitea's at `3.5.3+1.24.2-rootless`, dated **2026-06-17**, because `warm-gitea` crash-loops on a read-only `app.ini`; open recipe fix: gitea PR #4). ### What it cost us The gitea upgrade tier based on `3.5.3` and went green, while the upgrade real installs perform was broken: | transition | `APP_INI_VERSION` | outcome | |---|---|---| | `3.5.3 → head` (what CI tested) | v21 → v22 | config **renamed** → Swarm creates a new object → green | | `3.6.1 → 3.6.2` (what users run) | v22 → v22 | content changed under the same name → **abort**: `only updates to Labels are allowed` | That shipped in `3.6.2+1.27.1-rootless` and blocked every existing install from taking the 1.27.1 security release (CVE-2026-60004, CVE-2026-59774). ### The change When the newest published release older than head is **newer than the canonical**, use the release and log why — naming the stale canonical, so the failing promote gets noticed instead of silently degrading coverage. Resolution stays dynamic; `UPGRADE_BASE_FLOOR` still filters structurally invalid candidates; the canonical remains the base whenever it is current. **Unit-verified** on the real gitea tag set: base moves `3.5.3+1.24.2-rootless` → `3.6.1+1.26.2-rootless` — precisely the transition that reproduces the break. ### Follow-up (not in this PR) The five failing WC5 promotes are a separate bug worth fixing at the source; this change stops them from silently weakening upgrade coverage in the meantime.
autonomic-bot added 1 commit 2026-08-10 18:16:49 +00:00
upgrade base: prefer the newest published release over a STALE canonical
continuous-integration/drone/push Build is failing
Details
fc68e5ef6e 
The canonical is last-GREEN, not CURRENT. Its promotion (WC5) deploys a
warm-<recipe> app and health-checks it; when that fails the canonical stays
pinned at an old release indefinitely — and the failure is invisible to the
upgrade tier, which keeps basing on it.

Live evidence (2026-08-10): gitea's canonical was 3.5.3+1.24.2-rootless from
2026-06-17 because warm-gitea crash-loops on a read-only app.ini (open fix:
recipe PR #4). Four other recipes are in the same state — the 2026-08-09 sweep
logged GREEN-BUT-PROMOTE-FAILED for bluesky-pds, drone, gitea, lasuite-drive and
lasuite-meet.

Consequence: the upgrade tier tested 3.5.3 -> head, a transition no deployment
performs, and MISSED the one they do. 3.5.3 -> head crosses an APP_INI_VERSION
change (v21 -> v22) so Swarm creates a fresh config object; the real
3.6.1 -> 3.6.2 upgrade keeps v22 and aborts on Swarm's immutable-config rule
('only updates to Labels are allowed'). That shipped in 3.6.2 and broke every
existing install taking the 1.27.1 security release.

Now: when the newest published release older than head is NEWER than the
canonical, use the release and say so in the log (naming the stale canonical so
the promote failure gets noticed). Resolution stays dynamic; UPGRADE_BASE_FLOOR
still filters structurally-invalid candidates; the canonical remains the base
whenever it is current.

Unit-verified on the real gitea tag set: base moves 3.5.3+1.24.2-rootless ->
3.6.1+1.26.2-rootless, exactly the transition that reproduces the break.
autonomic-bot requested review from trav 2026-08-10 18:16:49 +00:00
autonomic-bot requested review from notplants 2026-08-10 18:16:50 +00:00
Some required checks failed
continuous-integration/drone/push Build is failing
Details
You are not authorized to merge this pull request.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin test/upgrade-base-prefer-newest-release-20260810:test/upgrade-base-prefer-newest-release-20260810
git checkout test/upgrade-base-prefer-newest-release-20260810
Sign in to join this conversation.
Reviewers
No Reviewers
trav
notplants
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
autonomic autonomic-bot kawaiipunk (KawaiiPunk) notplants trav (Trav Fryer)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: recipe-maintainers/cc-ci#16