From 0c4539b7ad833fa3089589b8310d7255d6ffb46e Mon Sep 17 00:00:00 2001 From: notplants Date: Thu, 18 Jun 2026 21:58:13 +0000 Subject: [PATCH 1/2] feat(discourse): switch app to official discourse/discourse image (experimental) Replaces the paywalled bitnamilegacy app with the official discourse/discourse image behind Traefik. DB is reused as-is; uploads migrate from the legacy bitnami volume idempotently. The wrapper entrypoint injects the db_password and smtp_password secrets (the official image has no *_FILE support). SMTP env vars are renamed to the official names; release notes cover the migration. Recipe 0.8.1+3.5.0 -> 1.0.0+3.5.3 (major: new image, env/volume/port changes). --- .env.sample | 22 +++++---- README.md | 79 +++++++++++++----------------- abra.sh | 3 ++ app-install-ssl.sh | 11 +++++ cc-app-entrypoint.sh | 15 ++++++ compose.smtpauth.yml | 10 +--- compose.yml | 113 +++++++++++++++++++++++-------------------- migrate-uploads.sh | 24 +++++++++ release/1.0.0+3.5.3 | 10 ++++ 9 files changed, 170 insertions(+), 117 deletions(-) create mode 100755 app-install-ssl.sh create mode 100755 cc-app-entrypoint.sh create mode 100755 migrate-uploads.sh create mode 100644 release/1.0.0+3.5.3 diff --git a/.env.sample b/.env.sample index fd54e8f..bbbb5f7 100644 --- a/.env.sample +++ b/.env.sample @@ -5,17 +5,19 @@ DOMAIN=discourse.example.com #EXTRA_DOMAINS=', `www.discourse.example.com`' LETS_ENCRYPT_ENV=production -# Outgoing email -#DISCOURSE_SMTP_HOST= -#DISCOURSE_SMTP_PORT= -#DISCOURSE_SMTP_USER= -#DISCOURSE_SMTP_PROTOCOL= -#DISCOURSE_SMTP_AUTH= -# Set this if you send e-mails from a different domain than noreply@$DOMAIN -#DISCOURSE_NOTIFICATION_EMAIL=$SMTP_USER +# Admin / developer accounts (comma-separated); these become admins on signup +DISCOURSE_DEVELOPER_EMAILS=admin@example.com -# SMTP authentication -#COMPOSE_FILE="compose.yml:compose.smtpauth.yml" +# Outgoing email (official discourse/discourse env names) +#DISCOURSE_SMTP_ADDRESS= +#DISCOURSE_SMTP_PORT=587 +#DISCOURSE_SMTP_USER_NAME= +#DISCOURSE_SMTP_AUTHENTICATION=login +#DISCOURSE_SMTP_ENABLE_START_TLS=true +# Set this if you send e-mail from a different address than noreply@$DOMAIN +#DISCOURSE_NOTIFICATION_EMAIL= + +# SMTP password as a secret #SECRET_SMTP_PASSWORD_VERSION=v1 SECRET_DB_PASSWORD_VERSION=v1 diff --git a/README.md b/README.md index 61dd783..14a478f 100644 --- a/README.md +++ b/README.md @@ -6,67 +6,54 @@ A platform for community discussion * **Category**: Apps -* **Status**: -* **Image**: [`bitnami/discourse`](https://hub.docker.com/r/bitnami/discourse) +* **Status**: 3, experimental +* **Image**: [`discourse/discourse`](https://hub.docker.com/r/discourse/discourse), 4, upstream * **Healthcheck**: yes -* **Backups**: no +* **Backups**: yes * **Email**: yes -* **Tests**: no +* **Tests**: yes * **SSO**: no +> **Note**: this recipe runs the official, **experimental** `discourse/discourse` +> image. Upstream does not yet recommend it for production — see +> . Use with care. + ## Basic usage 1. Set up Docker Swarm and [`abra`] 2. Deploy [`coop-cloud/traefik`] -3. `abra app new discourse --secrets` (optionally with `--pass` if you'd like - to save secrets in `pass`) -4. `abra app config YOURAPPDOMAIN` - be sure to change `$DOMAIN` to something that resolves to - your Docker swarm box +3. `abra app new discourse --secrets` +4. `abra app config YOURAPPDOMAIN` — set `DOMAIN` and `DISCOURSE_DEVELOPER_EMAILS` 5. `abra app deploy YOURAPPDOMAIN` -6. Open the configured domain in your browser to finish set-up +6. Open the configured domain in your browser to finish set-up. The first account + that registers with an address listed in `DISCOURSE_DEVELOPER_EMAILS` becomes + an admin. -[`abra`]: https://git.autonomic.zone/autonomic-cooperative/abra -[`coop-cloud/traefik`]: https://git.autonomic.zone/coop-cloud/traefik +[`abra`]: https://docs.coopcloud.tech/abra/ +[`coop-cloud/traefik`]: https://git.coopcloud.tech/coop-cloud/traefik -## To add a new admin user +The app serves plain HTTP on port 80; Traefik terminates TLS in front of it. The +image's built-in nginx/Let's Encrypt is disabled by the recipe (`install-ssl` +override) so it works behind the reverse proxy. -1. Login to the instance `abra app run APPNAME app sh` -2. `cd /opt/bitnami/discourse` -3. `RAILS_ENV=production bundle exec rake admin:create` and follow prompts. +## Add an admin user -## Install plugins +``` +abra app run YOURAPPDOMAIN app discourse admin create +``` -1. Login to instance `abra app run APPNAME app sh` -2. `cd /bitnami/discourse/plugins/` -3. `git clone plugin.git` for example `https://github.com/discourse/discourse-openid-connect.git` -4. `abra app restart YOURAPPDOMAIN app` +## Migrating from the previous (bitnami) recipe -### Events / calendar plugin +The official image stores uploads under `/shared` rather than bitnami's +`/bitnami/discourse`. On first boot the recipe copies uploads + backups from the +old bitnami volume (mounted read-only at `/legacy`) into `/shared`, once, +idempotently. The Postgres database is reused as-is. After a successful migration +a later recipe version will drop the transitional `/legacy` mount. -We've had some luck running [discourse-events](https://github.com/paviliondev/discourse-events). +If you are upgrading from the bitnami recipe, also remove the now-unused sidekiq +service that swarm leaves behind (sidekiq runs inside the app container now): -## Setup Notes - -Until issue #1 is fixed, the default user is `user` and the default password is `bitnami123` - -## Postgres major version upgrades - -Welcome to hell. - -1. `abra app run YOURAPPDOMAIN db pg_dumpall -U discourse | gzip > YOURAPPDOMAIN_db_DATE.sql.gz` -2. `abra app volume ls YOURAPPDOMAIN`, find the name of the Postgres data volume -3. `scp` the backup to your VPS -4. `abra app undeploy YOURAPPDOMAIN` -5. `abra app volume rm YOURAPPDOMAIN`, choose the Postgres data volume -6. `abra app deploy YOURAPPDOMAIN`, then `abra app undeploy YOURAPPDOMAIN` -7. `ssh` to the VPS, run (replacing `13-alpine` with the new Postgres version) - `docker run -v YOURDATAVOLUME:/var/lib/postgresql/data -e POSTGRES_HOST_AUTH_METHOD=trust -it postgres:13-alpine` -8. In another SSH session on the server, run `docker ps` to find the ID of the - new Postgres container, then `docker exec -it CONTAINERID bash` -9. In the shell you just launched, run `dropdb -U discourse discourse`, then - `createdb -U discourse discourse`, then Ctrl+D or run `exit` -10. In the second SSH session, run `zcat YOURAPPDOMAIN_db_DATE.sql.gz | docker exec -it CONTAINERID psql -U discourse` -11. Exit the second SSH session -12. Back in the first SSH session, Ctrl+C to shut down the database -13. `abra app deploy YOURAPPDOMAIN` +``` +docker service rm YOURSTACK_sidekiq +``` diff --git a/abra.sh b/abra.sh index b08beeb..2fae15c 100644 --- a/abra.sh +++ b/abra.sh @@ -1,2 +1,5 @@ export DB_ENTRYPOINT_VERSION=v3 export PG_BACKUP_VERSION=v2 +export APP_ENTRYPOINT_VERSION=v2 +export APP_INSTALL_SSL_VERSION=v1 +export APP_MIGRATE_UPLOADS_VERSION=v1 diff --git a/app-install-ssl.sh b/app-install-ssl.sh new file mode 100755 index 0000000..418271b --- /dev/null +++ b/app-install-ssl.sh @@ -0,0 +1,11 @@ +#!/bin/bash +# Overrides the official image's /etc/runit/1.d/install-ssl. +# +# The stock install-ssl always runs configure-ssl (and configure-letsencrypt), +# which empties the default `listen 80` nginx outlet and switches to `listen 443 +# ssl` against a cert that does not exist here — nginx then crash-loops, or the +# image tries to obtain its own Let's Encrypt cert. Under Co-op Cloud, Traefik +# terminates TLS and proxies plain HTTP to port 80, so we skip the image's SSL +# setup entirely and let nginx keep its default HTTP-on-80 config. +echo "install-ssl overridden by recipe: serving plain HTTP on :80 behind Traefik" +exit 0 diff --git a/cc-app-entrypoint.sh b/cc-app-entrypoint.sh new file mode 100755 index 0000000..17ab37f --- /dev/null +++ b/cc-app-entrypoint.sh @@ -0,0 +1,15 @@ +#!/bin/bash +# Co-op Cloud wrapper around the official image's /sbin/boot. +# discourse/discourse reads passwords from the process env (pups/Ruby; it has no +# *_FILE support), so inject them from the docker secrets before booting. +set -e + +if [ -f /run/secrets/db_password ]; then + export DISCOURSE_DB_PASSWORD="$(cat /run/secrets/db_password)" +fi + +if [ -f /run/secrets/smtp_password ]; then + export DISCOURSE_SMTP_PASSWORD="$(cat /run/secrets/smtp_password)" +fi + +exec /sbin/boot diff --git a/compose.smtpauth.yml b/compose.smtpauth.yml index 8b23f8d..f18470c 100644 --- a/compose.smtpauth.yml +++ b/compose.smtpauth.yml @@ -3,14 +3,8 @@ version: "3.8" services: app: - environment: - - DISCOURSE_SMTP_PASSWORD_FILE=/var/run/secrets/smtp_password - secrets: - - smtp_password - - sidekiq: - environment: - - DISCOURSE_SMTP_PASSWORD_FILE=/var/run/secrets/smtp_password + # the wrapper entrypoint reads /run/secrets/smtp_password and exports + # DISCOURSE_SMTP_PASSWORD (the official image has no *_FILE support) secrets: - smtp_password diff --git a/compose.yml b/compose.yml index 3365c28..5a3e717 100644 --- a/compose.yml +++ b/compose.yml @@ -3,53 +3,64 @@ version: "3.8" services: app: - image: bitnamilegacy/discourse:3.5.0 + image: discourse/discourse:3.5.3 networks: - proxy - internal - # entrypoint: ['tail', '-f', '/dev/null'] + # official image CMD is /sbin/boot; wrapper injects the DB password secret first + entrypoint: /usr/local/bin/cc-app-entrypoint.sh environment: - - ALLOW_EMPTY_PASSWORD=yes - - DISCOURSE_DATABASE_HOST=${STACK_NAME}_db - - DISCOURSE_DATABASE_NAME=discourse - - DISCOURSE_DATABASE_PASSWORD_FILE=/run/secrets/db_password - - DISCOURSE_DATABASE_USER=discourse - - DISCOURSE_HOST=${DOMAIN} - - DISCOURSE_NOTIFICATION_EMAIL - - DISCOURSE_SMTP_AUTH - - DISCOURSE_SMTP_HOST + - DISCOURSE_HOSTNAME=${DOMAIN} + - DISCOURSE_DEVELOPER_EMAILS=${DISCOURSE_DEVELOPER_EMAILS} + - DISCOURSE_DB_HOST=${STACK_NAME}_db + - DISCOURSE_DB_PORT=5432 + - DISCOURSE_DB_NAME=discourse + - DISCOURSE_DB_USERNAME=discourse + - DISCOURSE_REDIS_HOST=${STACK_NAME}_redis + - DISCOURSE_REDIS_PORT=6379 + - DISCOURSE_SMTP_ADDRESS - DISCOURSE_SMTP_PORT - - DISCOURSE_SMTP_PROTOCOL - - DISCOURSE_SMTP_USER - - PASSENGER_COMPILE_NATIVE_SUPPORT_BINARY=0 + - DISCOURSE_SMTP_USER_NAME + - DISCOURSE_SMTP_PASSWORD + - DISCOURSE_SMTP_AUTHENTICATION + - DISCOURSE_SMTP_ENABLE_START_TLS + - DISCOURSE_NOTIFICATION_EMAIL volumes: - - 'discourse_data:/bitnami/discourse' + - 'discourse_shared:/shared' + # transition only: legacy bitnami volume, read-only, for one-time upload migration + - 'discourse_data:/legacy:ro' secrets: - db_password + configs: + - source: app_entrypoint + target: /usr/local/bin/cc-app-entrypoint.sh + mode: 0555 + - source: app_install_ssl + target: /etc/runit/1.d/install-ssl + mode: 0555 + - source: app_migrate_uploads + target: /etc/runit/1.d/02-migrate-bitnami-uploads + mode: 0555 depends_on: - db - redis deploy: update_config: failure_action: rollback - order: start-first + order: stop-first labels: - "traefik.enable=true" - - "traefik.http.services.${STACK_NAME}.loadbalancer.server.port=3000" + - "traefik.http.services.${STACK_NAME}.loadbalancer.server.port=80" - "traefik.http.routers.${STACK_NAME}.rule=Host(`${DOMAIN}`${EXTRA_DOMAINS})" - "traefik.http.routers.${STACK_NAME}.entrypoints=web-secure" - "traefik.http.routers.${STACK_NAME}.tls.certresolver=${LETS_ENCRYPT_ENV}" - ## Redirect from EXTRA_DOMAINS to DOMAIN - #- "traefik.http.routers.${STACK_NAME}.middlewares=${STACK_NAME}-redirect" - #- "traefik.http.middlewares.${STACK_NAME}-redirect.headers.SSLForceHost=true" - #- "traefik.http.middlewares.${STACK_NAME}-redirect.headers.SSLHost=${DOMAIN}" - - "coop-cloud.${STACK_NAME}.version=0.8.1+3.5.0" + - "coop-cloud.${STACK_NAME}.version=1.0.0+3.5.3" healthcheck: - test: "ruby -e \"require 'uri'; require 'net/http'; uri = URI('http://localhost:3000/srv/status'); res = Net::HTTP.get_response(uri); if res.is_a?(Net::HTTPSuccess) then exit (0) else exit (1) end\"" + test: "curl -fsS http://localhost/srv/status || exit 1" interval: 30s timeout: 10s retries: 6 - start_period: 20m + start_period: 25m db: image: pgvector/pgvector:pg17 @@ -72,6 +83,15 @@ services: - POSTGRES_USER=discourse - POSTGRES_DB=discourse - POSTGRES_PASSWORD_FILE=/run/secrets/db_password + healthcheck: + test: "pg_isready -U discourse -d discourse" + interval: 30s + timeout: 10s + retries: 5 + # generous: a postgres major-version upgrade (apt install + pg_upgrade) runs + # in the entrypoint before the server accepts connections — don't let the + # healthcheck kill an in-progress migration + start_period: 10m deploy: labels: backupbot.backup: "true" @@ -85,35 +105,12 @@ services: - internal volumes: - 'redis_data:/data' - - sidekiq: - image: bitnamilegacy/discourse:3.5.0 - networks: - - proxy - - internal - depends_on: - - discourse - volumes: - - 'discourse_data:/bitnami/discourse' - command: /opt/bitnami/scripts/discourse-sidekiq/run.sh - secrets: - - db_password - environment: - - ALLOW_EMPTY_PASSWORD=yes - - DISCOURSE_DATABASE_HOST=db - - DISCOURSE_DATABASE_NAME=discourse - - DISCOURSE_DATABASE_PASSWORD_FILE=/run/secrets/db_password - - DISCOURSE_DATABASE_PORT_NUMBER=5432 - - DISCOURSE_DATABASE_USER=discourse - - DISCOURSE_HOST=${DOMAIN} - - DISCOURSE_REDIS_HOST=redis - - DISCOURSE_REDIS_PORT_NUMBER=6379 - - DISCOURSE_SMTP_HOST - - DISCOURSE_SMTP_PORT - - DISCOURSE_SMTP_PROTOCOL - - DISCOURSE_SMTP_USER - - PASSENGER_COMPILE_NATIVE_SUPPORT_BINARY=0 - - DISCOURSE_SMTP_AUTH + healthcheck: + test: "redis-cli ping | grep -q PONG" + interval: 30s + timeout: 5s + retries: 5 + start_period: 30s secrets: db_password: @@ -123,6 +120,7 @@ secrets: volumes: postgresql_data: redis_data: + discourse_shared: discourse_data: networks: @@ -131,6 +129,15 @@ networks: internal: configs: + app_entrypoint: + name: ${STACK_NAME}_app_entrypoint_${APP_ENTRYPOINT_VERSION} + file: cc-app-entrypoint.sh + app_install_ssl: + name: ${STACK_NAME}_app_install_ssl_${APP_INSTALL_SSL_VERSION} + file: app-install-ssl.sh + app_migrate_uploads: + name: ${STACK_NAME}_app_migrate_uploads_${APP_MIGRATE_UPLOADS_VERSION} + file: migrate-uploads.sh db_entrypoint: name: ${STACK_NAME}_db_entrypoint_${DB_ENTRYPOINT_VERSION} file: entrypoint.postgres.sh.tmpl diff --git a/migrate-uploads.sh b/migrate-uploads.sh new file mode 100755 index 0000000..3189be5 --- /dev/null +++ b/migrate-uploads.sh @@ -0,0 +1,24 @@ +#!/bin/bash +# One-time, idempotent, NON-destructive migration of uploads + backups from a +# legacy bitnami discourse volume into the official image's /shared. +# +# Runs on every boot as a runit 1.d hook but no-ops after the first success +# (sentinel) and when there is no legacy volume mounted (fresh installs). It only +# ever COPIES from the read-only /legacy mount, so an interruption just re-copies +# on the next boot — there is no move/delete to leave the data half-migrated. +set -e + +SENTINEL=/shared/.bitnami-uploads-migrated +[ -e "$SENTINEL" ] && exit 0 + +if [ -d /legacy/public/uploads ]; then + echo "[migrate-uploads] copying bitnami uploads/backups -> /shared" + mkdir -p /shared/uploads /shared/backups + cp -a /legacy/public/uploads/. /shared/uploads/ 2>/dev/null || true + cp -a /legacy/public/backups/. /shared/backups/ 2>/dev/null || true + # discourse runs as uid 1000; the official boot also chowns /shared, but be explicit + chown -R discourse:discourse /shared/uploads /shared/backups 2>/dev/null || true + echo "[migrate-uploads] done" +fi + +touch "$SENTINEL" diff --git a/release/1.0.0+3.5.3 b/release/1.0.0+3.5.3 new file mode 100644 index 0000000..1d620bd --- /dev/null +++ b/release/1.0.0+3.5.3 @@ -0,0 +1,10 @@ +This release switches from the bitnami image to the official discourse/discourse +image. Some env vars need to be renamed for this migration; everything else +should happen automatically. + +Rename these in your app's .env (the values carry over): + + DISCOURSE_SMTP_HOST --> DISCOURSE_SMTP_ADDRESS + DISCOURSE_SMTP_USER --> DISCOURSE_SMTP_USER_NAME + DISCOURSE_SMTP_AUTH --> DISCOURSE_SMTP_AUTHENTICATION + DISCOURSE_SMTP_PROTOCOL --> DISCOURSE_SMTP_ENABLE_START_TLS (takes a boolean true/false, not the old tls/ssl value, so translate it rather than copying it straight across) -- 2.49.0 From 1f77af93bd52178117ca3f506d71680d9a43873e Mon Sep 17 00:00:00 2001 From: notplants <@notplants> Date: Mon, 22 Jun 2026 19:57:54 +0000 Subject: [PATCH 2/2] feat(db): switch to discourse/postgres image (auto-upgrade) Move the db off the bitnami-era pgvector:pg17 + hand-rolled pg_upgrade entrypoint to discourse/postgres:pg18 (pgvector + discourse's auto-upgrade layer). The image runs the in-place major-version pg_upgrade itself on boot; the recipe configures it via env: - a small inline entrypoint injects the db password secret into $DB_PASSWORD (the image expects it in the env, no *_FILE support) - POSTGRES_USER (the install user pg_upgrade must match) defaults to 'postgres' -- correct for fresh installs and bitnami-origin clusters -- overridable from .env - POSTGRES_INITDB_ARGS=--no-data-checksums so the new pg18 cluster matches pre-18 clusters (pg18 initdb enables checksums by default; pg_upgrade needs a match) - mount postgresql_data at /var/lib/postgresql (versioned PGDATA .../18/docker) - pg_backup.sh uses POSTGRES_USER for the dump/drop/recreate; fix paths - document the POSTGRES_USER override in .env.sample, README and the release note - drop entrypoint.postgres.sh.tmpl Tested on cctest: pg17->pg18 upgrade preserves data and serves over HTTPS; fresh install works; backup+restore round-trips. --- .env.sample | 6 ++++ README.md | 16 ++++++++++ abra.sh | 3 +- compose.yml | 44 +++++++++++++++---------- entrypoint.postgres.sh.tmpl | 64 ------------------------------------- pg_backup.sh | 29 +++++++++-------- release/1.0.0+3.5.3 | 9 ++++++ 7 files changed, 76 insertions(+), 95 deletions(-) delete mode 100644 entrypoint.postgres.sh.tmpl diff --git a/.env.sample b/.env.sample index bbbb5f7..117ce06 100644 --- a/.env.sample +++ b/.env.sample @@ -21,3 +21,9 @@ DISCOURSE_DEVELOPER_EMAILS=admin@example.com #SECRET_SMTP_PASSWORD_VERSION=v1 SECRET_DB_PASSWORD_VERSION=v1 + +# Postgres bootstrap superuser (the cluster's "install user"). Defaults to +# `postgres`, which matches fresh installs and bitnami-origin clusters. Only set +# this if you are upgrading a cluster that was bootstrapped with a different +# superuser (e.g. `discourse`) — a postgres major upgrade fails unless it matches. +#POSTGRES_USER=postgres diff --git a/README.md b/README.md index 14a478f..5f61fb0 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,22 @@ override) so it works behind the reverse proxy. abra app run YOURAPPDOMAIN app discourse admin create ``` +## Postgres major version upgrades + +Handled automatically by the [`discourse/postgres`] image (pgvector + an +auto-upgrade layer). On deploy it finds an older cluster, installs the old +binaries and runs `pg_upgrade` into the new versioned data directory. No manual +dump/restore needed. + +`pg_upgrade` must run as the old cluster's bootstrap superuser (its "install +user"). The recipe uses `POSTGRES_USER`, which defaults to `postgres` — the right +value for fresh installs and for clusters that came from the old bitnami recipe. +If your cluster was bootstrapped with a different superuser (e.g. `discourse`), +set `POSTGRES_USER` in the app `.env` before upgrading, otherwise `pg_upgrade` +will refuse with an install-user mismatch. + +[`discourse/postgres`]: https://github.com/discourse/discourse-postgres + ## Migrating from the previous (bitnami) recipe The official image stores uploads under `/shared` rather than bitnami's diff --git a/abra.sh b/abra.sh index 2fae15c..1d31a8d 100644 --- a/abra.sh +++ b/abra.sh @@ -1,5 +1,4 @@ -export DB_ENTRYPOINT_VERSION=v3 -export PG_BACKUP_VERSION=v2 +export PG_BACKUP_VERSION=v4 export APP_ENTRYPOINT_VERSION=v2 export APP_INSTALL_SSL_VERSION=v1 export APP_MIGRATE_UPLOADS_VERSION=v1 diff --git a/compose.yml b/compose.yml index 5a3e717..8ad31ee 100644 --- a/compose.yml +++ b/compose.yml @@ -63,35 +63,51 @@ services: start_period: 25m db: - image: pgvector/pgvector:pg17 + # discourse/postgres = pgvector + discourse's postgres management layer, which + # auto-upgrades an older cluster in place on boot (pg_upgrade into the versioned + # PGDATA /var/lib/postgresql/${MAJOR}/docker); everything is driven by the env below. + image: discourse/postgres:pg18 networks: - internal secrets: - db_password volumes: - - 'postgresql_data:/var/lib/postgresql/data' + # the image expects the whole cluster tree mounted here (not the data subdir); + # an existing pg17 cluster at the volume root is found and upgraded into /18/docker + - 'postgresql_data:/var/lib/postgresql' configs: - - source: db_entrypoint - target: /docker-entrypoint.sh - mode: 0555 - source: pg_backup target: /pg_backup.sh mode: 0555 - entrypoint: /docker-entrypoint.sh + entrypoint: + - /bin/bash + - -c + - | + if [ -f /run/secrets/db_password ]; then + DB_PASSWORD="$$(cat /run/secrets/db_password)" + export DB_PASSWORD POSTGRES_PASSWORD="$$DB_PASSWORD" + fi + exec run-postgres.sh postgres environment: + # internal-only overlay network; keep all-trust so the app and the + # backup/restore hooks connect without juggling the superuser password - POSTGRES_HOST_AUTH_METHOD=trust - - POSTGRES_USER=discourse - POSTGRES_DB=discourse - - POSTGRES_PASSWORD_FILE=/run/secrets/db_password + - DB_USER=discourse + # pg_upgrade runs as this role and initdb's the new cluster with it; it must + # match the OLD cluster's bootstrap superuser (oid 10). The image default + # `postgres` matches fresh installs and bitnami-origin clusters. Override in + # the app .env (POSTGRES_USER=...) only for a cluster bootstrapped differently. + - POSTGRES_USER=${POSTGRES_USER:-postgres} + # pg18's initdb enables data checksums by default, but pg13-17 clusters here + # have them off and pg_upgrade requires a match -> initialise without them. + - POSTGRES_INITDB_ARGS=--no-data-checksums healthcheck: test: "pg_isready -U discourse -d discourse" interval: 30s timeout: 10s retries: 5 - # generous: a postgres major-version upgrade (apt install + pg_upgrade) runs - # in the entrypoint before the server accepts connections — don't let the - # healthcheck kill an in-progress migration - start_period: 10m + start_period: 15m deploy: labels: backupbot.backup: "true" @@ -138,10 +154,6 @@ configs: app_migrate_uploads: name: ${STACK_NAME}_app_migrate_uploads_${APP_MIGRATE_UPLOADS_VERSION} file: migrate-uploads.sh - db_entrypoint: - name: ${STACK_NAME}_db_entrypoint_${DB_ENTRYPOINT_VERSION} - file: entrypoint.postgres.sh.tmpl - template_driver: golang pg_backup: name: ${STACK_NAME}_pg_backup_${PG_BACKUP_VERSION} file: pg_backup.sh diff --git a/entrypoint.postgres.sh.tmpl b/entrypoint.postgres.sh.tmpl deleted file mode 100644 index cbd032b..0000000 --- a/entrypoint.postgres.sh.tmpl +++ /dev/null @@ -1,64 +0,0 @@ -#!/bin/bash - -set -e - -OLDDATA=$PGDATA/old_data -NEWDATA=$PGDATA/new_data - -echo "Running as $(id)" - -# The migration uses $OLDDATA/$NEWDATA as scratch and removes them when it -# finishes; a leftover *empty* one means a run was interrupted before any data -# moved (data still intact at $PGDATA) so we clear it and retry, while a -# *non-empty* one means data may live only there, so we stop for manual recovery. -for scratch in $OLDDATA $NEWDATA; do - if [ -d "$scratch" ] && [ -n "$(ls -A "$scratch")" ]; then - echo "FATAL: $scratch exists and is not empty - a previous migration did not" - echo "complete and the data may only exist there. manual recovery necessary." - exit 1 - fi -done -rm -rf $OLDDATA $NEWDATA - -if [ -f $PGDATA/PG_VERSION ]; then - DATA_VERSION=$(cat $PGDATA/PG_VERSION) - - if [ -n "$DATA_VERSION" -a "$PG_MAJOR" != "$DATA_VERSION" ]; then - echo "postgres data version $DATA_VERSION found, but need $PG_MAJOR. Starting migration" - echo "Installing postgres $DATA_VERSION" - sed -i "s/$/ $DATA_VERSION/" /etc/apt/sources.list.d/pgdg.list - apt-get update && apt-get install -y --no-install-recommends \ - postgresql-$DATA_VERSION \ - && rm -rf /var/lib/apt/lists/* - # pg_upgrade must run as the old cluster's bootstrap superuser (the "install - # user", oid 10), and the new cluster must be initialised with that same - # user. It is not necessarily $POSTGRES_USER (e.g. clusters created with the - # default "postgres" superuser and a separate app role), so read it from the - # old cluster: briefly start it and ask, connecting as the app role we know. - PGBIN=/usr/lib/postgresql/$DATA_VERSION/bin - gosu postgres $PGBIN/pg_ctl -D $PGDATA -w \ - -o "-c listen_addresses= -c unix_socket_directories=/tmp" start - INSTALL_USER=$(gosu postgres psql -h /tmp -U "$POSTGRES_USER" -d postgres -tAc \ - "select rolname from pg_roles where oid = 10") - gosu postgres $PGBIN/pg_ctl -D $PGDATA -w stop - echo "old cluster install user: $INSTALL_USER" - echo "shuffling around" - gosu postgres mkdir $OLDDATA $NEWDATA - chmod 700 $OLDDATA $NEWDATA - mv $PGDATA/* $OLDDATA/ || true - echo "running initdb" - # abuse entrypoint script for initdb by making server error out; initialise - # the new cluster with the same superuser as the old one so pg_upgrade matches - gosu postgres bash -c "export PGDATA=$NEWDATA POSTGRES_USER=$INSTALL_USER ; /usr/local/bin/docker-entrypoint.sh --invalid-arg || true" - echo "running pg_upgrade" - cd /tmp - gosu postgres pg_upgrade --link -b /usr/lib/postgresql/$DATA_VERSION/bin -d $OLDDATA -D $NEWDATA -U $INSTALL_USER - cp $OLDDATA/pg_hba.conf $NEWDATA/ - mv $NEWDATA/* $PGDATA - rm -rf $OLDDATA - rmdir $NEWDATA - echo "migration complete" - fi -fi - -/usr/local/bin/docker-entrypoint.sh postgres diff --git a/pg_backup.sh b/pg_backup.sh index 382a1d2..9f26058 100755 --- a/pg_backup.sh +++ b/pg_backup.sh @@ -1,44 +1,47 @@ #!/bin/bash -# Postgres backup/restore hook for the discourse `db` service. +# Postgres backup/restore hook for the discourse `db` service (discourse/postgres image). set -e -BACKUP_FILE='/var/lib/postgresql/data/backup.sql' -export PGPASSWORD=$(cat "${POSTGRES_PASSWORD_FILE:-/run/secrets/db_password}") -DB_USER="${POSTGRES_USER:-discourse}" +# dump goes at the volume root so backupbot's backup.sql label finds it +BACKUP_FILE='/var/lib/postgresql/backup.sql' +DATADIR="${PGDATA:-/var/lib/postgresql/18/docker}" DB_NAME="${POSTGRES_DB:-discourse}" +# bootstrap superuser for the dump/drop/recreate; same POSTGRES_USER the db service sets +SU="${POSTGRES_USER:-postgres}" + function backup { - pg_dump -U "$DB_USER" "$DB_NAME" | gzip > "$BACKUP_FILE" + pg_dump -U "$SU" "$DB_NAME" | gzip > "$BACKUP_FILE" } function restore { - cd /var/lib/postgresql/data/ + cd "$DATADIR" # Block all non-local connections so the running discourse app + sidekiq cannot reconnect and # interfere with the drop/recreate/reimport. Restored on exit. restore_hba() { cat pg_hba.conf.bak > pg_hba.conf rm -f pg_hba.conf.bak - su postgres -c 'pg_ctl reload' + su postgres -c "pg_ctl -D '$DATADIR' reload" } cp pg_hba.conf pg_hba.conf.bak echo 'local all all trust' > pg_hba.conf - su postgres -c 'pg_ctl reload' + su postgres -c "pg_ctl -D '$DATADIR' reload" trap restore_hba EXIT INT TERM # terminate any lingering local sessions before recreate # see https://stackoverflow.com/questions/5108876/kill-a-postgresql-session-connection - psql -U "$DB_USER" -d postgres -c \ + psql -U "$SU" -d postgres -c \ "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname='${DB_NAME}' AND pid<>pg_backend_pid();" # drop database and then recreate it - psql -U "$DB_USER" -d postgres -c "DROP DATABASE ${DB_NAME} WITH (FORCE);" - createdb -U "$DB_USER" "$DB_NAME" + psql -U "$SU" -d postgres -c "DROP DATABASE ${DB_NAME} WITH (FORCE);" + createdb -U "$SU" "$DB_NAME" - # reimport data - gunzip -c "$BACKUP_FILE" | psql -U "$DB_USER" -d "$DB_NAME" -1 -v ON_ERROR_STOP=1 -f - + # reimport data + gunzip -c "$BACKUP_FILE" | psql -U "$SU" -d "$DB_NAME" -1 -v ON_ERROR_STOP=1 -f - } $@ diff --git a/release/1.0.0+3.5.3 b/release/1.0.0+3.5.3 index 1d620bd..3ac2345 100644 --- a/release/1.0.0+3.5.3 +++ b/release/1.0.0+3.5.3 @@ -8,3 +8,12 @@ Rename these in your app's .env (the values carry over): DISCOURSE_SMTP_USER --> DISCOURSE_SMTP_USER_NAME DISCOURSE_SMTP_AUTH --> DISCOURSE_SMTP_AUTHENTICATION DISCOURSE_SMTP_PROTOCOL --> DISCOURSE_SMTP_ENABLE_START_TLS (takes a boolean true/false, not the old tls/ssl value, so translate it rather than copying it straight across) + +WARNING: if your deployment's database has an "install user" other than `postgres` +(some older deployments do), you must set the POSTGRES_USER env var in your .env +for this migration, otherwise the postgres upgrade aborts with an install-user +mismatch. + +Check your old deployment's install user before upgrading (if this command returns postgres, then you do not need to set this env): + + abra app run YOURAPPDOMAIN db -- psql -U discourse -tAc 'select rolname from pg_roles where oid = 10' -- 2.49.0