upgrade base: prefer the newest published release over a STALE canonical
continuous-integration/drone/push Build is failing
continuous-integration/drone/push Build is failing
The canonical is last-GREEN, not CURRENT. Its promotion (WC5) deploys a warm-<recipe> app and health-checks it; when that fails the canonical stays pinned at an old release indefinitely — and the failure is invisible to the upgrade tier, which keeps basing on it. Live evidence (2026-08-10): gitea's canonical was 3.5.3+1.24.2-rootless from 2026-06-17 because warm-gitea crash-loops on a read-only app.ini (open fix: recipe PR #4). Four other recipes are in the same state — the 2026-08-09 sweep logged GREEN-BUT-PROMOTE-FAILED for bluesky-pds, drone, gitea, lasuite-drive and lasuite-meet. Consequence: the upgrade tier tested 3.5.3 -> head, a transition no deployment performs, and MISSED the one they do. 3.5.3 -> head crosses an APP_INI_VERSION change (v21 -> v22) so Swarm creates a fresh config object; the real 3.6.1 -> 3.6.2 upgrade keeps v22 and aborts on Swarm's immutable-config rule ('only updates to Labels are allowed'). That shipped in 3.6.2 and broke every existing install taking the 1.27.1 security release. Now: when the newest published release older than head is NEWER than the canonical, use the release and say so in the log (naming the stale canonical so the promote failure gets noticed). Resolution stays dynamic; UPGRADE_BASE_FLOOR still filters structurally-invalid candidates; the canonical remains the base whenever it is current. Unit-verified on the real gitea tag set: base moves 3.5.3+1.24.2-rootless -> 3.6.1+1.26.2-rootless, exactly the transition that reproduces the break.
This commit is contained in:
@@ -175,6 +175,36 @@ def resolve_upgrade_base(
|
||||
flush=True,
|
||||
)
|
||||
rec = None
|
||||
# STALE-CANONICAL GUARD (phase relbase): the canonical is only "last-green", NOT "current". Its
|
||||
# promotion can fail for reasons unrelated to any PR (the WC5 promote deploys a warm-<recipe> app
|
||||
# and health-checks it), and a failed promote leaves the canonical pinned at an OLD release
|
||||
# indefinitely — gitea sat at 3.5.3+1.24.2 from 2026-06-17 to 2026-08-10 because warm-gitea
|
||||
# crash-looped on a read-only app.ini. Basing the upgrade tier on that stale release tests a
|
||||
# transition no deployment performs, and can silently MISS breaks in the transition users do
|
||||
# perform: gitea 3.5.3→head crosses an APP_INI_VERSION change (v21→v22) so Swarm creates a fresh
|
||||
# config, while the real 3.6.1→3.6.2 upgrade keeps v22 and aborts on Swarm's immutable-config
|
||||
# rule. Prefer the newest published release older than head whenever it is newer than the
|
||||
# canonical: that is what real installs upgrade from.
|
||||
if rec and rec.get("version") and not skip_canonicals and head_version:
|
||||
_rel_tags = warm_reconcile.recipe_tags(recipe)
|
||||
if floor:
|
||||
_rel_tags = [t for t in _rel_tags if not _below_floor(t)]
|
||||
newest_rel = warm_reconcile.newest_older_version(_rel_tags, head_version)
|
||||
if newest_rel and warm_reconcile.version_key(newest_rel) > warm_reconcile.version_key(
|
||||
rec["version"]
|
||||
):
|
||||
print(
|
||||
f"== upgrade base: newest published release {newest_rel} is NEWER than the "
|
||||
f"last-green canonical {rec['version']} — using the release (what deployments "
|
||||
f"actually upgrade from); canonical is stale, check its WC5 promote",
|
||||
flush=True,
|
||||
)
|
||||
return BasePlan(
|
||||
"version",
|
||||
newest_rel,
|
||||
None,
|
||||
f"newest published release older than head (canonical {rec['version']} is stale)",
|
||||
)
|
||||
if rec and rec.get("version") and not skip_canonicals:
|
||||
canon = rec["version"]
|
||||
same = head_version is not None and warm_reconcile.version_key(
|
||||
|
||||
Reference in New Issue
Block a user