fix: seed app.ini into a writable config volume for Gitea 1.24+ #4

Open
autonomic-bot wants to merge 1 commits from fix-app-ini-writable into main
Owner

Make app.ini writable so Gitea can persist config

Gitea 1.24+ writes settings back to app.ini on boot (e.g. WORK_PATH). When app.ini is mounted directly as a read-only swarm config, that write fails:

[E] Unable to update WORK_PATH=/var/lib/gitea to config "/etc/gitea/app.ini":
    failed to save "/etc/gitea/app.ini": open /etc/gitea/app.ini: read-only file system

It is non-fatal (the app stays healthy) but Gitea warns it "might cause bugs when accessing the git repositories" and cannot persist any runtime config change.

Fix

  • Mount the rendered config at /etc/gitea/app.ini.init (read-only swarm config).
  • Seed a writable copy at /etc/gitea/app.ini in the config volume via docker-setup.sh.
  • Use ! -s (empty), not ! -f: upgrades from the old direct-mount layout leave a 0-byte app.ini placeholder that still needs seeding.
  • Bump DOCKER_SETUP_SH_VERSION v1 → v3.

Verified on cctest (1.26.2)

After the fix, /etc/gitea/app.ini is -rw------- and Gitea persists to it (file grew 1886 → 1901 bytes as it wrote WORK_PATH); the read-only error is gone; /api/healthz passes.

Split out from the image-update PR (#3) so the update can ship without this behaviour change.

## Make app.ini writable so Gitea can persist config Gitea 1.24+ writes settings back to `app.ini` on boot (e.g. `WORK_PATH`). When `app.ini` is mounted directly as a read-only swarm config, that write fails: ``` [E] Unable to update WORK_PATH=/var/lib/gitea to config "/etc/gitea/app.ini": failed to save "/etc/gitea/app.ini": open /etc/gitea/app.ini: read-only file system ``` It is **non-fatal** (the app stays healthy) but Gitea warns it "might cause bugs when accessing the git repositories" and cannot persist any runtime config change. ### Fix - Mount the rendered config at `/etc/gitea/app.ini.init` (read-only swarm config). - Seed a writable copy at `/etc/gitea/app.ini` in the `config` volume via `docker-setup.sh`. - Use `! -s` (empty), not `! -f`: upgrades from the old direct-mount layout leave a 0-byte `app.ini` placeholder that still needs seeding. - Bump `DOCKER_SETUP_SH_VERSION` v1 → v3. ### Verified on cctest (1.26.2) After the fix, `/etc/gitea/app.ini` is `-rw-------` and Gitea persists to it (file grew 1886 → 1901 bytes as it wrote `WORK_PATH`); the read-only error is gone; `/api/healthz` passes. Split out from the image-update PR (#3) so the update can ship without this behaviour change.
Author
Owner

Reproduced on cc-ci — this PR is required, and the trigger is LFS (not an upgrade artefact)

The warm-gitea canonical has been down (0/1), crash-looping on exactly what this PR fixes:

LoadCommonSettings() [F] Unable to load settings from config: error saving JWT Secret for
custom config: failed to save "/etc/gitea/app.ini": open /etc/gitea/app.ini: read-only file system

I isolated it today by deploying the current release (3.6.3+1.27.1-rootless) in several
configurations on cc-ci:

config result
compose.yml + compose.mariadb.yml (recipe default) 1/1, healthy
compose.yml + compose.sqlite3.yml + compose.lfs.yml (what warm-gitea used) crash-loop
compose.yml + compose.mariadb.yml + compose.lfs.yml crash-loop

So the trigger is compose.lfs.yml, not sqlite and not stale state — I also wiped both of
warm-gitea's volumes and redeployed clean, and it still crashed. With LFS_START_SERVER=true, gitea
writes a secret back to app.ini at boot, which cannot work while the rendered config is mounted
directly as a read-only swarm config. This PR's app.ini.init + seeded writable copy is the right
fix.

Note the recipe cannot simply be run without LFS either: app.ini.tmpl references
{{ secret "lfs_jwt_secret" }} unconditionally, so dropping the overlay fails earlier still, at
config expansion — secret target lfs_jwt_secret not found.

Why CI stayed green throughout: the tests deploy the default (mariadb, no LFS), which is the one
combination that works. The defect is invisible to the suite.

This PR is stale — it is based on 1.24.2-rootless / APP_INI_VERSION=v22, while main is now
1.27.1-rootless / v23. It needs reconciling with main before it can go green.

I have left warm-gitea undeployed rather than crash-looping (it was churning containers every
~6s). Its .env is untouched, so redeploying restores its intended config once this merges. Nothing
is blocked meanwhile: canonical resolution falls back to a release tag, and gitea's own CI passes.

### Reproduced on cc-ci — this PR is required, and the trigger is LFS (not an upgrade artefact) The `warm-gitea` canonical has been down (0/1), crash-looping on exactly what this PR fixes: ``` LoadCommonSettings() [F] Unable to load settings from config: error saving JWT Secret for custom config: failed to save "/etc/gitea/app.ini": open /etc/gitea/app.ini: read-only file system ``` I isolated it today by deploying the **current release (3.6.3+1.27.1-rootless)** in several configurations on cc-ci: | config | result | |---|---| | `compose.yml` + `compose.mariadb.yml` (recipe default) | **1/1, healthy** | | `compose.yml` + `compose.sqlite3.yml` + `compose.lfs.yml` (what warm-gitea used) | crash-loop | | `compose.yml` + `compose.mariadb.yml` + `compose.lfs.yml` | **crash-loop** | So the trigger is **`compose.lfs.yml`**, not sqlite and not stale state — I also wiped both of warm-gitea's volumes and redeployed clean, and it still crashed. With `LFS_START_SERVER=true`, gitea writes a secret back to `app.ini` at boot, which cannot work while the rendered config is mounted directly as a read-only swarm config. This PR's `app.ini.init` + seeded writable copy is the right fix. Note the recipe cannot simply be run without LFS either: `app.ini.tmpl` references `{{ secret "lfs_jwt_secret" }}` unconditionally, so dropping the overlay fails earlier still, at config expansion — `secret target lfs_jwt_secret not found`. **Why CI stayed green throughout:** the tests deploy the default (mariadb, no LFS), which is the one combination that works. The defect is invisible to the suite. **This PR is stale** — it is based on `1.24.2-rootless` / `APP_INI_VERSION=v22`, while main is now `1.27.1-rootless` / `v23`. It needs reconciling with main before it can go green. I have left `warm-gitea` **undeployed** rather than crash-looping (it was churning containers every ~6s). Its `.env` is untouched, so redeploying restores its intended config once this merges. Nothing is blocked meanwhile: canonical resolution falls back to a release tag, and gitea's own CI passes.
Author
Owner

!testme

!testme
autonomic-bot added 1 commit 2026-08-11 16:49:43 +00:00
Gitea 1.24+ persists settings back to app.ini on boot (e.g. WORK_PATH).
Mounting app.ini directly as a read-only swarm config makes that write
fail (logged error, possible repo-access bugs). Mount the rendered config
as app.ini.init and seed a writable copy in the config volume instead.
autonomic-bot force-pushed fix-app-ini-writable from 83605ff3fa to d0243cf724 2026-08-11 16:49:43 +00:00 Compare
Author
Owner

🌻 cc-cigitea @ d0243cf7 failurehttps://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1244

(summary card unavailable — see the run for details.) full logs · dashboard

<!-- cc-ci:testme --> 🌻 **cc-ci** — `gitea` @ `d0243cf7` ❌ **failure** → https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1244 _(summary card unavailable — see the run for details.)_ [full logs](https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1244) · [dashboard](https://ci.commoninternet.net/)
Author
Owner

!testme

!testme
Author
Owner

🌻 cc-cigitea @ d0243cf7 failurehttps://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1245

(summary card unavailable — see the run for details.) full logs · dashboard

<!-- cc-ci:testme --> 🌻 **cc-ci** — `gitea` @ `d0243cf7` ❌ **failure** → https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1245 _(summary card unavailable — see the run for details.)_ [full logs](https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1245) · [dashboard](https://ci.commoninternet.net/)
Author
Owner

!testme

!testme
Author
Owner

🌻 cc-cigitea @ d0243cf7 failurehttps://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1246

(summary card unavailable — see the run for details.) full logs · dashboard

<!-- cc-ci:testme --> 🌻 **cc-ci** — `gitea` @ `d0243cf7` ❌ **failure** → https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1246 _(summary card unavailable — see the run for details.)_ [full logs](https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1246) · [dashboard](https://ci.commoninternet.net/)
Author
Owner

!testme

!testme
Author
Owner

🌻 cc-cigitea @ d0243cf7 failurehttps://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1247

(summary card unavailable — see the run for details.) full logs · dashboard

<!-- cc-ci:testme --> 🌻 **cc-ci** — `gitea` @ `d0243cf7` ❌ **failure** → https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1247 _(summary card unavailable — see the run for details.)_ [full logs](https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1247) · [dashboard](https://ci.commoninternet.net/)
Author
Owner

!testme

!testme
Author
Owner

🌻 cc-cigitea @ d0243cf7 failurehttps://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1249

(summary card unavailable — see the run for details.) full logs · dashboard

<!-- cc-ci:testme --> 🌻 **cc-ci** — `gitea` @ `d0243cf7` ❌ **failure** → https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1249 _(summary card unavailable — see the run for details.)_ [full logs](https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1249) · [dashboard](https://ci.commoninternet.net/)
Author
Owner

Reconciled with main and verified GREEN

Rebased onto main (was 12 commits behind). The branch is now 0 behind, one commit, original
author preserved. Conflict was in abra.sh only, resolved to keep both sides:

main this branch
APP_INI_VERSION v23 v23 (main's — the app.ini content is unchanged by this fix)
DOCKER_SETUP_SH_VERSION v1 v3 (bumped — docker-setup.sh.tmpl content changes)

v3 rather than v2 deliberately: v2 was released on main historically with different content
(036f070, c4154c8), and swarm configs are immutable, so reusing that name would fail to update.
v3 has only ever carried this fix.

compose.yml merged cleanly — it now has main's gitea/gitea:1.27.1-rootless and this fix's
target: /etc/gitea/app.ini.init. Total diff vs main is 3 files, +10/-2.

Verification

1. The fix resolves the actual bug. Deployed the exact configuration that crash-loops on main
(compose.yml:compose.sqlite3.yml:compose.lfs.yml) from this branch on cc-ci:

replicas: 1/1        read-only app.ini errors: 0

-rw-------  git  git   1874  /etc/gitea/app.ini        <- writable copy, gitea wrote to it
-r--r--r--  root root  1857  /etc/gitea/app.ini.init   <- the read-only swarm config

On main that same config crash-loops indefinitely. Note the seeded copy is larger than the
source — that is gitea successfully writing back the settings it could not write before.

2. Full suite, cold, against this branch head (d0243cf):

install : pass   upgrade : pass   backup : pass   restore : pass   custom : pass
level = 5 of 5   VERDICT: GREEN

About the red !testme badge — it is not this PR

Every !testme since build 1244 dies before the harness starts, on any recipe (plausible fails
identically right now), with:

OSError: [Errno 28] No space left on device: '/var/lib/cc-ci-runs/<build>'

while the host has 110 GB free and 16% inodes used. It is not disk: I sampled df every 2s
across a whole failing build and available space never moved. The same mkdir succeeds as root over
ssh, in the runner's own mount namespace, and 61/61 times in a stress loop; and the same harness,
same recipe, same numeric run id, run by hand outside drone, proceeds normally
. Restarting
drone-runner-exec and recreating the runs directory with a fresh inode both changed nothing.

So it is specific to the drone step's execution context, and it is a pre-existing CI-server fault
unrelated to this change. The green verdict above comes from the same harness !testme invokes,
just driven directly.

### Reconciled with main and verified GREEN Rebased onto `main` (was 12 commits behind). The branch is now **0 behind**, one commit, original author preserved. Conflict was in `abra.sh` only, resolved to keep **both** sides: | | main | this branch | |---|---|---| | `APP_INI_VERSION` | v23 | **v23** (main's — the app.ini content is unchanged by this fix) | | `DOCKER_SETUP_SH_VERSION` | v1 | **v3** (bumped — `docker-setup.sh.tmpl` content changes) | `v3` rather than `v2` deliberately: `v2` was released on main historically with *different* content (`036f070`, `c4154c8`), and swarm configs are immutable, so reusing that name would fail to update. `v3` has only ever carried this fix. `compose.yml` merged cleanly — it now has main's `gitea/gitea:1.27.1-rootless` **and** this fix's `target: /etc/gitea/app.ini.init`. Total diff vs main is 3 files, +10/-2. ### Verification **1. The fix resolves the actual bug.** Deployed the exact configuration that crash-loops on `main` (`compose.yml:compose.sqlite3.yml:compose.lfs.yml`) from this branch on cc-ci: ``` replicas: 1/1 read-only app.ini errors: 0 -rw------- git git 1874 /etc/gitea/app.ini <- writable copy, gitea wrote to it -r--r--r-- root root 1857 /etc/gitea/app.ini.init <- the read-only swarm config ``` On `main` that same config crash-loops indefinitely. Note the seeded copy is *larger* than the source — that is gitea successfully writing back the settings it could not write before. **2. Full suite, cold, against this branch head (`d0243cf`):** ``` install : pass upgrade : pass backup : pass restore : pass custom : pass level = 5 of 5 VERDICT: GREEN ``` ### About the red `!testme` badge — it is not this PR Every `!testme` since build 1244 dies before the harness starts, on **any** recipe (plausible fails identically right now), with: ``` OSError: [Errno 28] No space left on device: '/var/lib/cc-ci-runs/<build>' ``` while the host has **110 GB free and 16% inodes used**. It is not disk: I sampled `df` every 2s across a whole failing build and available space never moved. The same `mkdir` succeeds as root over ssh, in the runner's own mount namespace, and 61/61 times in a stress loop; and the **same harness, same recipe, same numeric run id, run by hand outside drone, proceeds normally**. Restarting `drone-runner-exec` and recreating the runs directory with a fresh inode both changed nothing. So it is specific to the drone step's execution context, and it is a pre-existing CI-server fault unrelated to this change. The green verdict above comes from the same harness `!testme` invokes, just driven directly.
Author
Owner

!testme

!testme
Author
Owner

🌻 cc-cigitea @ d0243cf7 passedhttps://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1250

(summary card unavailable — see the run for details.) full logs · dashboard

<!-- cc-ci:testme --> 🌻 **cc-ci** — `gitea` @ `d0243cf7` ✅ **passed** → https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1250 _(summary card unavailable — see the run for details.)_ [full logs](https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1250) · [dashboard](https://ci.commoninternet.net/)
Author
Owner

Correction to my earlier comment

I wrote that "the recipe cannot simply be run without LFS either: app.ini.tmpl references
{{ secret "lfs_jwt_secret" }} unconditionally." That is wrong — it is already conditional:

LFS_START_SERVER = {{ env "GITEA_LFS_START_SERVER" }}
{{ if (eq (env "GITEA_LFS_START_SERVER") "true") }}
LFS_JWT_SECRET = {{ secret "lfs_jwt_secret" }}
{{ end }}

A deploy without the LFS overlay works fine — I had in fact verified that earlier in this
investigation (default compose.yml + compose.mariadb.yml, healthy 1/1) and failed to reconcile
the two observations.

What actually happened: warm-gitea's .env sets GITEA_LFS_START_SERVER=true independently of
the overlay
. When I dropped compose.lfs.yml from its COMPOSE_FILE while testing, the env var
stayed true, so the conditional evaluated true while the secret was no longer attached to the
service — hence secret target lfs_jwt_secret not found. That was my misconfiguration, not a recipe
defect.

Nothing in this PR changes, and the rest of the diagnosis stands: LFS is still the trigger for
the read-only app.ini crash (mariadb + LFS reproduces it; mariadb alone does not), and this PR is
still the fix. Verified GREEN at build 1250, level 5 of 5.

### Correction to my earlier comment I wrote that "the recipe cannot simply be run without LFS either: `app.ini.tmpl` references `{{ secret "lfs_jwt_secret" }}` unconditionally." **That is wrong — it is already conditional:** ``` LFS_START_SERVER = {{ env "GITEA_LFS_START_SERVER" }} {{ if (eq (env "GITEA_LFS_START_SERVER") "true") }} LFS_JWT_SECRET = {{ secret "lfs_jwt_secret" }} {{ end }} ``` A deploy without the LFS overlay works fine — I had in fact verified that earlier in this investigation (default `compose.yml` + `compose.mariadb.yml`, healthy 1/1) and failed to reconcile the two observations. What actually happened: `warm-gitea`'s `.env` sets `GITEA_LFS_START_SERVER=true` **independently of the overlay**. When I dropped `compose.lfs.yml` from its `COMPOSE_FILE` while testing, the env var stayed `true`, so the conditional evaluated true while the secret was no longer attached to the service — hence `secret target lfs_jwt_secret not found`. That was my misconfiguration, not a recipe defect. **Nothing in this PR changes**, and the rest of the diagnosis stands: LFS is still the trigger for the read-only `app.ini` crash (mariadb + LFS reproduces it; mariadb alone does not), and this PR is still the fix. Verified GREEN at build 1250, level 5 of 5.
All checks were successful
cc-ci/testme cc-ci: success
You are not authorized to merge this pull request.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin fix-app-ini-writable:fix-app-ini-writable
git checkout fix-app-ini-writable
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: recipe-maintainers/gitea#4