chore: upgrade impress to v5.7.0, nginx to 1.31.7, redis to 8.10.2, switch docspec image to ghcr.io/docspec/api:1.21.4 (port 3000)

.env.sample

@@ -21,6 +21,8 @@ SECRET_DJANGO_SP_VERSION=v1
 SECRET_MINIO_RP_VERSION=v1
 # MINIO_ROOT_USER
 SECRET_MINIO_RU_VERSION=v1
+# COLLABORATION_SERVER_SECRET
+SECRET_COLLAB_SS_VERSION=v1
 # POSTGRES_PASSWORD
 SECRET_POSTGRES_P_VERSION=v1
 # Y_PROVIDER_API_KEY
@@ -104,13 +106,3 @@ LOGGING_LEVEL_LOGGERS_APP=INFO
 # Header the backend reads for media auth subrequests.
 # Upstream default matches the recipe's nginx config (X-Original-URL).
 # MEDIA_AUTH_ORIGINAL_URL_HEADER=HTTP_X_ORIGINAL_URL
-
-##############################################################################
-# COLLABORATION SERVER (yhub) - SOFT MIGRATION
-##############################################################################
-# Since v6 the document content lives in the collaboration server, not in the
-# media bucket. With SOFT_MIGRATION, a pre-v6 document is copied over the first
-# time it is opened. Only set to false once
-# `abra app cmd <app-name> backend migrate_documents` has covered all documents
-# (or on a fresh v6 install); unmigrated documents would otherwise open empty.
-# SOFT_MIGRATION=true

MAINTENANCE.md

@@ -4,21 +4,7 @@ All contributions should be made via pull requests.
 
 ## Pull Requests
 
-A pull request can be merged directly into `main` by any maintainer. 
-No approval is needed, but pull requests should be tested as working before merge.
-
-## Dependency Updates
-
-- patch updates proposed from renovate bot are merged directly - 
-  if there is a CVE, also release a new patch version of this recipe
-- minor updates proposed from renovate bot are tested by one maintainer 
-  and merged if they don't break, then a new minor version of the recipe 
-  is released
-- major updates proposed form renovate bot are tested by two maintainers 
-  and merged if they don't break, then a major version of the recipe is 
-  released. 
-  If the renovate bot PR is older than one week, one maintainer can decide
-  to merge it anyway. 
+A pull request can be merged directly into `main` by any maintainer. No approval is needed, but pull requests should be tested as working before merge.
 
 ## Become a maintainer
 
@@ -26,5 +12,4 @@ Everyone can apply to be a recipe maintainer:
 
 1. Watch the repository to get updates.
 2. Add yourself to the list in [README.md](./README.md) and open a pull request with the change.
-3. Once the pull request is merged, you will be added to the 
-  [La Suite maintainers team](https://git.coopcloud.tech/org/coop-cloud/teams/lasuite-maintainers).
+3. Once the pull request is merged, you will be added to the [La Suite maintainers team](https://git.coopcloud.tech/org/coop-cloud/teams/lasuite-maintainers).

README.md

@@ -4,7 +4,6 @@
 
 <!-- metadata -->
 
-* **Maintainer**: [@trav](https://git.coopcloud.tech/trav), [@notplants](https://git.coopcloud.tech/notplants), [@ineiti](https://git.coopcloud.tech/ineiti)
 * **Category**: Apps
 * **Status**: 2
 * **Image**: [`lasuite/docs`](https://hub.docker.com/r/lasuite/docs), 4, upstream
@@ -27,19 +26,6 @@ You should then be able to visit the landing page of your app, but not yet to lo
 * Migrations run automatically on backend startup. To trigger manually: `abra app cmd <app-name> backend migrate`
 * Minio buckets are created automatically on first deploy. To manually trigger: `abra app cmd <app-name> minio minio_initialize`
 
-## RSA keys
-
-Since v6, the backend and the collaboration server (`yhub`) authenticate each other with signed tokens. Each needs its own RSA private key, never the same one. The recipe generates both automatically on first deploy into the `jwt_keys` volume (the `jwt-keys` service, using the yhub image's Node crypto) and reuses them on later deploys. They are not Swarm secrets because abra cannot generate PEM keys and Swarm secrets must exist before the stack is deployed.
-
-## Upgrading from v5 to v6
-
-v6 replaces the collaboration server. Document content moves from the minio bucket to the new `yhub` service, which stores it in its own `yhub` database. If you skip the migration, every existing document opens **empty**. See [UPGRADE.md](https://github.com/suitenumerique/docs/blob/main/UPGRADE.md) upstream for details.
-
-1. The `collab_ss` secret is no longer used. The two RSA keys are generated automatically on first deploy (see "RSA keys" above).
-2. Leave `SOFT_MIGRATION` at its default (`true`) and deploy. The `yhub` schema is created on startup. A document is copied to yhub the first time someone opens it.
-3. Copy all remaining documents: `abra app cmd <app-name> backend migrate_documents`. You can pass arguments, e.g. `abra app cmd <app-name> backend migrate_documents -- --dry-run`. Re-run it with `--retry-failed` if some documents failed.
-4. Only after that, you may set `SOFT_MIGRATION=false`. Keep the minio bucket and its object versions: they hold the document history from before v6.
-
 ## Configure Authentication
 
 lasuite-docs **requires** an OpenID Connect (OIDC) single sign-on provider; deployment has been tested with [Keycloak](https://git.coopcloud.tech/coop-cloud/keycloak), which we recommend, or you could also try [Authentik](https://git.coopcloud.tech/coop-cloud/authentik), both of which are installable using Co-op Cloud.
@@ -94,3 +80,8 @@ You then need to insert the password for your smtp server as a secret:
 Then redeploy the app, and automated e-mail sending should work:
 
 `abra app deploy <app-name> --force`
+
+## Maintainers
+
+* @trav
+* @notplants

abra-entrypoint.sh

@@ -7,6 +7,7 @@ set -e
 [ -f /run/secrets/minio_ru ] && export AWS_S3_ACCESS_KEY_ID="$(cat /run/secrets/minio_ru)"
 [ -f /run/secrets/django_sp ] && export DJANGO_SUPERUSER_PASSWORD="$(cat /run/secrets/django_sp)"
 [ -f /run/secrets/oidc_rpcs ] && export OIDC_RP_CLIENT_SECRET="$(cat /run/secrets/oidc_rpcs)"
+[ -f /run/secrets/collab_ss ] && export COLLABORATION_SERVER_SECRET="$(cat /run/secrets/collab_ss)"
 [ -f /run/secrets/y_api_key ] && export Y_PROVIDER_API_KEY="$(cat /run/secrets/y_api_key)"
 [ -f /run/secrets/email_pass ] && export DJANGO_EMAIL_HOST_PASSWORD="$(cat /run/secrets/email_pass)"
 

abra.sh

@@ -1,9 +1,9 @@
 # Set any config versions here
 # Docs: https://docs.coopcloud.tech/maintainers/handbook/#manage-configs
-export ABRA_ENTRYPOINT_VERSION=v6
-export NGINX_CONF_VERSION=v4
-export PG_BACKUP_VERSION=v5
-export MINIO_INITIALIZE_VERSION=v2
+export ABRA_ENTRYPOINT_VERSION=v5
+export NGINX_CONF_VERSION=v3
+export PG_BACKUP_VERSION=v4
+export MINIO_INITIALIZE_VERSION=v1
 export MIGRATE_VERSION=v1
 
 environment() {
@@ -18,11 +18,3 @@ migrate() {
 minio_initialize() {
   /minio-initialize.sh
 }
-
-# Hand all pre-v6 documents from the media bucket over to the collaboration
-# server (yhub). Resumable and safe to re-run; extra args are passed through,
-# e.g. --dry-run, --retry-failed, --concurrency N
-migrate_documents() {
-  source /abra-entrypoint.sh -e
-  python manage.py migrate_documents "$@"
-}

compose.yml

@@ -54,10 +54,8 @@ x-common-env: &common-env
   OPENAI_SDK_BASE_URL: https://openaiendpoint.com
   OPENAI_SDK_API_KEY: password
   AI_MODEL: llama
-  # Collaboration server (yhub), reached in-stack
-  YHUB_API_BASE_URL: http://yhub:3002
-  # RSA key signing the tokens the backend presents to yhub and y-provider
-  JWT_PRIVATE_KEY_FILE: /keys/jwt.pem
+  # Collaboration
+  COLLABORATION_API_URL: https://$DOMAIN/collaboration/api/
   # Email app URL (used in email templates; matches recipe DOMAIN)
   DJANGO_EMAIL_URL_APP: "https://${DOMAIN}"
   # Conversion (file upload → Yjs via y-provider)
@@ -93,26 +91,11 @@ x-postgres-env: &postgres-env
 x-yprovider-env: &yprovider-env
   COLLABORATION_LOGGING: "true"
   # Y_PROVIDER_API_KEY supplied via secrets
+  COLLABORATION_API_URL: http://y-provider:4444/api/
   COLLABORATION_SERVER_ORIGIN: https://${DOMAIN}
+  # COLLABORATION_SERVER_SECRET supplied via secrets
   COLLABORATION_BACKEND_BASE_URL: https://${DOMAIN}
-  # The last segment is YHUB_ORG (default: docs)
-  COLLABORATION_WS_URL: wss://${DOMAIN}/collaboration/ws/v1/docs
-
-x-yhub-env: &yhub-env
-  PORT: 3002
-  REDIS: redis://yhub-valkey:6379
-  # POSTGRES is built from the postgres_p secret in the entrypoint
-  COLLABORATION_BACKEND_BASE_URL: https://${DOMAIN}
-  COLLABORATION_SERVER_ORIGIN: https://${DOMAIN}
-  Y_PROVIDER_API_KEY_FILE: /run/secrets/y_api_key
-  YHUB_JWT_PRIVATE_KEY_FILE: /keys/yhub-jwt.pem
-  # Seed documents from the pre-v6 media bucket the first time they are opened.
-  # Only turn off once `abra app cmd <app> backend migrate_documents` covered all documents.
-  SOFT_MIGRATION: "${SOFT_MIGRATION:-true}"
-  LEGACY_S3_ENDPOINT_URL: http://minio:9000
-  LEGACY_S3_ACCESS_KEY_ID_FILE: /run/secrets/minio_ru
-  LEGACY_S3_SECRET_ACCESS_KEY_FILE: /run/secrets/minio_rp
-  LEGACY_S3_BUCKET_NAME: docs-media-storage
+  COLLABORATION_WS_URL: wss://${DOMAIN}/collaboration/ws/
 
 x-minio-env: &minio-env
   MINIO_ROOT_USER_FILE: /run/secrets/minio_ru
@@ -120,14 +103,14 @@ x-minio-env: &minio-env
 
 services:
   app:
-    image: lasuite/impress-frontend:v6.0.2
+    image: lasuite/impress-frontend:v5.7.0
     networks:
       - backend
     deploy:
       labels:
         - "traefik.enable=false"
         - "coop-cloud.${STACK_NAME}.timeout=${TIMEOUT:-120}"
-        - "coop-cloud.${STACK_NAME}.version=0.4.4+v5.6.1"
+        - "coop-cloud.${STACK_NAME}.version=0.4.3+v5.6.1"
     user: "${DOCKER_USER:-1000}"
     healthcheck:
       test: ["CMD", "curl", "-f", "http://localhost:8080"]
@@ -137,7 +120,7 @@ services:
       start_period: 10s
 
   backend:
-    image: lasuite/impress-backend:v6.0.2
+    image: lasuite/impress-backend:v5.7.0
     networks:
       - backend
     environment:
@@ -152,9 +135,7 @@ services:
     user: "${DOCKER_USER:-1000}"
     command: ["gunicorn", "-c", "/usr/local/etc/gunicorn/impress.py", "impress.wsgi:application"]
     entrypoint: >
-      sh -c "while [ ! -s /keys/jwt.pem ]; do sleep 1; done; if [ \"$$AUTO_MIGRATIONS\" = \"true\" ]; then /migrate.sh; fi && exec /abra-entrypoint.sh /usr/local/bin/entrypoint \"$$@\"" --
-    volumes:
-      - jwt_keys:/keys:ro
+      sh -c "if [ \"$$AUTO_MIGRATIONS\" = \"true\" ]; then /migrate.sh; fi && exec /abra-entrypoint.sh /usr/local/bin/entrypoint \"$$@\"" --
     configs:
       - source: abra_entrypoint
         target: /abra-entrypoint.sh
@@ -166,6 +147,7 @@ services:
       - django_sk
       - django_sp
       - oidc_rpcs
+      - collab_ss
       - minio_rp
       - minio_ru
       - postgres_p
@@ -173,7 +155,7 @@ services:
       - email_pass
 
   celery:
-    image: lasuite/impress-backend:v6.0.2
+    image: lasuite/impress-backend:v5.7.0
     networks:
       - backend
     healthcheck:
@@ -186,10 +168,7 @@ services:
     command: ["celery", "-A", "impress.celery_app", "worker", "-l", "INFO"]
     environment:
       <<: [*common-env, *postgres-env, *yprovider-env]
-    entrypoint: >
-      sh -c "while [ ! -s /keys/jwt.pem ]; do sleep 1; done; exec /abra-entrypoint.sh /usr/local/bin/entrypoint \"$$@\"" --
-    volumes:
-      - jwt_keys:/keys:ro
+    entrypoint: ["/abra-entrypoint.sh", "/usr/local/bin/entrypoint"]
     configs:
       - source: abra_entrypoint
         target: /abra-entrypoint.sh
@@ -198,90 +177,15 @@ services:
       - django_sk
       - django_sp
       - oidc_rpcs
+      - collab_ss
       - minio_rp
-      - minio_ru
       - postgres_p
       - y_api_key
       - email_pass
 
-  # Collaboration server: serves the websocket and holds the document content
-  yhub:
-    image: lasuite/impress-yhub:v6.0.2
-    networks:
-      - backend
-    environment: *yhub-env
-    user: "${DOCKER_USER:-1000}"
-    healthcheck:
-      test: ["CMD", "node", "-e", "fetch('http://localhost:3002/collaboration/ready/v1').then(r=>process.exit(r.ok?0:1),()=>process.exit(1))"]
-      interval: 15s
-      timeout: 5s
-      retries: 20
-      start_period: 30s
-    # The server never creates its schema: run the idempotent init-db before every start
-    entrypoint: >
-      sh -c "while [ ! -s /keys/yhub-jwt.pem ]; do sleep 1; done; export POSTGRES=\"postgres://docs:$$(node -e 'process.stdout.write(encodeURIComponent(require(\"fs\").readFileSync(\"/run/secrets/postgres_p\",\"utf8\").trim()))')@db:5432/yhub\" && node node_modules/@y/hub/bin/init-db.js && exec /usr/local/bin/entrypoint \"$$@\"" --
-    command: ["node", "--import", "./dist/sentry.js", "dist/server.js"]
-    volumes:
-      - jwt_keys:/keys:ro
-    secrets:
-      - postgres_p
-      - y_api_key
-      - minio_ru
-      - minio_rp
 
-  # Valkey of yhub: holds updates not yet persisted to postgres, so it is a store, not a cache
-  yhub-valkey:
-    image: valkey/valkey:8.1.10-alpine
-    command: ["valkey-server", "--maxmemory-policy", "volatile-lru", "--appendonly", "yes", "--appendfsync", "everysec"]
-    healthcheck:
-      test: ["CMD", "valkey-cli", "ping"]
-      interval: 15s
-      timeout: 5s
-      retries: 3
-    networks:
-      - backend
-    volumes:
-      - yhub_valkey:/data
-    deploy:
-      labels:
-        backupbot.backup: "${ENABLE_BACKUPS:-true}"
-
-  # Generates the two RSA keys the backend and yhub sign with, into a shared volume.
-  # They cannot be abra-generated secrets (PEM format) and Swarm secrets cannot be
-  # created at deploy time, so this idempotent sidecar creates them once and keeps them
-  # on the volume across redeploys. Both consumers wait for their file before starting.
-  jwt-keys:
-    image: lasuite/impress-yhub:v6.0.2
-    networks:
-      - backend
-    user: "0"
-    volumes:
-      - jwt_keys:/keys
-    healthcheck:
-      test: ["CMD-SHELL", "[ -s /keys/jwt.pem ] && [ -s /keys/yhub-jwt.pem ]"]
-      interval: 15s
-      timeout: 5s
-      retries: 3
-      start_period: 10s
-    entrypoint: ["/bin/sh", "-c"]
-    command:
-      - |
-        set -eu
-        generate() {
-          target="/keys/$$1"
-          if [ ! -s "$$target" ]; then
-            node -e 'const fs=require("fs");const {generateKeyPairSync}=require("crypto");const {privateKey}=generateKeyPairSync("rsa",{modulusLength:2048,publicKeyEncoding:{type:"spki",format:"pem"},privateKeyEncoding:{type:"pkcs8",format:"pem"}});fs.writeFileSync(process.argv[1],privateKey,{mode:0o400});' "$$target"
-          fi
-        }
-        generate jwt.pem
-        generate yhub-jwt.pem
-        chown -R "${DOCKER_USER:-1000}:${DOCKER_USER:-1000}" /keys
-        chmod 0400 /keys/jwt.pem /keys/yhub-jwt.pem
-        while true; do sleep 3600; done
-
-  # Conversion service only since v6 (no longer serves the collaboration)
   y-provider:
-    image: lasuite/impress-y-provider:v6.0.2
+    image: lasuite/impress-y-provider:v5.7.0
     networks:
       - backend
     healthcheck:
@@ -329,7 +233,7 @@ services:
       labels:
         backupbot.backup: "${ENABLE_BACKUPS:-true}"
         backupbot.backup.pre-hook: "/pg_backup.sh backup"
-        backupbot.backup.volumes.postgres.path: "backup.sql,backup-yhub.sql"
+        backupbot.backup.volumes.postgres.path: "backup.sql"
         backupbot.restore.post-hook: '/pg_backup.sh restore'
     configs:
         - source: pg_backup
@@ -380,7 +284,7 @@ services:
       - minio_ru
 
   web:
-    image: nginx:1.31.8
+    image: nginx:1.31.7
     healthcheck:
       test: ["CMD", "curl", "-f", "http://localhost:8083"]
       interval: 15s
@@ -416,8 +320,6 @@ networks:
 volumes:
   postgres:
   minio:
-  yhub_valkey:
-  jwt_keys:
 
 configs:
   nginx_conf:
@@ -450,6 +352,9 @@ secrets:
   postgres_p:
     external: true
     name: ${STACK_NAME}_postgres_p_${SECRET_POSTGRES_P_VERSION}
+  collab_ss:
+    external: true
+    name: ${STACK_NAME}_collab_ss_${SECRET_COLLAB_SS_VERSION}
   minio_rp:
     external: true
     name: ${STACK_NAME}_minio_rp_${SECRET_MINIO_RP_VERSION}

nginx.conf.tmpl

@@ -47,14 +47,7 @@ server {
         try_files $uri @proxy_to_docs_backend;
     }
     
-    location /external_api {
-        try_files $uri @proxy_to_docs_backend;
-    }
-
-    # Collaboration server (yhub). Only the routes below are published: it also
-    # serves prune, create-ydoc, reset-connections, migrate, restore-ydoc and
-    # reset-ydoc, which the backend calls in-stack and which must not be
-    # reachable from the outside.
+    # Proxy auth for collaboration server
     location /collaboration/ws/ {
         # Ensure WebSocket upgrade
         proxy_http_version 1.1;
@@ -62,7 +55,7 @@ server {
         proxy_set_header Connection "Upgrade";
 
         # Collaboration server
-        proxy_pass http://{{ env "STACK_NAME" }}_yhub:3002;
+        proxy_pass http://y-provider:4444;
 
         # Set appropriate timeout for WebSocket
         proxy_read_timeout 86400;
@@ -74,11 +67,9 @@ server {
         proxy_set_header Host $host;
     }
 
-    # Document routes (http fallback, history) and the public keys of yhub
-    location ~ ^/collaboration/(ydoc|activity|changeset|rollback|jwks)/ {
-        proxy_pass http://{{ env "STACK_NAME" }}_yhub:3002;
-        proxy_set_header X-Forwarded-Proto https;
-        proxy_set_header Origin $http_origin;
+    location  /collaboration/api/ {
+        # Collaboration server
+        proxy_pass http://y-provider:4444;
         proxy_set_header Host $host;
     }
 

pg_backup.sh

@@ -3,16 +3,10 @@
 set -e
 
 BACKUP_FILE='/var/lib/postgresql/data/pgdata/backup.sql'
-# database of the collaboration server (yhub), since v6
-YHUB_DB='yhub'
-YHUB_BACKUP_FILE='/var/lib/postgresql/data/pgdata/backup-yhub.sql'
 
 function backup {
   export PGPASSWORD=$(cat $POSTGRES_PASSWORD_FILE)
   pg_dump -U ${POSTGRES_USER} ${POSTGRES_DB} > $BACKUP_FILE
-  if psql -U ${POSTGRES_USER} -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname='${YHUB_DB}'" | grep -q 1; then
-    pg_dump -U ${POSTGRES_USER} ${YHUB_DB} > $YHUB_BACKUP_FILE
-  fi
 }
 
 function restore {
@@ -33,12 +27,6 @@ function restore {
     createdb -U ${POSTGRES_USER} ${POSTGRES_DB}
     psql -U ${POSTGRES_USER} -d ${POSTGRES_DB} -1 -f $BACKUP_FILE
 
-    if [ -f $YHUB_BACKUP_FILE ]; then
-        psql -U ${POSTGRES_USER} -d postgres -c "DROP DATABASE IF EXISTS ${YHUB_DB} WITH (FORCE);"
-        createdb -U ${POSTGRES_USER} ${YHUB_DB}
-        psql -U ${POSTGRES_USER} -d ${YHUB_DB} -1 -f $YHUB_BACKUP_FILE
-    fi
-
     trap - EXIT INT TERM
     restore_config
 }

release/next

@@ -1,13 +0,0 @@
-⚠️ Major upgrade to Docs v6: the collaboration server is replaced by the new `yhub`
-service, and existing documents have to be migrated to it, or they open EMPTY.
-
-`SECRET_COLLAB_SS_VERSION` and the `collab_ss` secret are no longer used. The two
-RSA keys the backend and yhub sign with are generated automatically on first
-deploy into the `jwt_keys` volume (the `jwt-keys` service) and reused afterwards;
-no manual secret insertion is needed.
-
-After deploying (SOFT_MIGRATION is on by default), migrate all documents:
-
-  abra app cmd <app-name> backend migrate_documents
-
-Only then may you set SOFT_MIGRATION=false. See the README for details.