Versioning logic should include all underlying services #47

New Issue

decentral1se · 2021-03-04T12:49:46Z

decentral1se commented

2021-03-04 12:49:46 +00:00

The current way abra handles versioning logic is that it looks at the deploy tags on the ${STACK_NAME}_$SERVICE and then does some comparisons and outputting based on those. A good start.

However, the thought has occurred that, since our app definitions ship their own versions of the underyling services required to run the app - the databses and caches and so on - should we also be taking those versions into account also!?

For example, an app repo packager might bump the version of the MySQL image and nothing else. The current versioning logic will say "nothing changed, no need to deploy". But that should deploy a new database image. And if that upgrade is a major version, you definitely want to know about it on the hoster side of things.

My gut feeling is that abra should take into consideration all service versions! What do you reckon? I think this could be a larger piece of work but will make things more stable. To do it:

Change ABRA_TYPE_VERSION to ABRA_TYPE_<service>_VERSION
Change ABRA_TYPE_DIGEST to ABRA_TYPE_<service>_DIGEST
Have app repos specify version/digest manually for each service in the abra.sh
Change abra logic to take those into account and display an update summary

The current way `abra` handles versioning logic is that it looks at the deploy tags on the `${STACK_NAME}_$SERVICE` and then does some comparisons and outputting based on those. A good start. However, the thought has occurred that, since our app definitions ship their own versions of the underyling services required to run the app - the databses and caches and so on - should we also be taking those versions into account also!? For example, an app repo packager might bump the version of the MySQL image and nothing else. The current versioning logic will say "nothing changed, no need to deploy". But that should deploy a new database image. And if that upgrade is a major version, you definitely want to know about it on the hoster side of things. My gut feeling is that `abra` should take into consideration all service versions! What do you reckon? I think this could be a larger piece of work but will make things more stable. To do it: - Change `ABRA_TYPE_VERSION` to `ABRA_TYPE_<service>_VERSION` - Change `ABRA_TYPE_DIGEST` to `ABRA_TYPE_<service>_DIGEST` - Have app repos specify version/digest manually for each service in the `abra.sh` - Change `abra` logic to take those into account and display an update summary

decentral1se added the

question

label 2021-03-04 12:49:46 +00:00

3wordchant commented

2021-03-10 21:08:03 +00:00

Yeah now that you mention it, definitely! Maybe we can do some find / yq / sed magic to pull the image tags out of all the compose.ymls to get us started 🤔

Yeah now that you mention it, definitely! Maybe we can do some `find` / `yq` / `sed` magic to pull the image tags out of all the `compose.yml`s to get us started 🤔

👀 1

decentral1se commented

2021-03-10 21:44:48 +00:00

Yeah seems like we might need some automation around this extra work load. Something not so magic would good. Maybe just a bash script to run as a packager that spits out all the tags and digests from the local compose.yml file something like app-catalogue.sh.

Yeah seems like we might need some automation around this extra work load. Something not so magic would good. Maybe just a bash script to run as a packager that spits out all the tags and digests from the local compose.yml file something like `app-catalogue.sh`.

decentral1se added

enhancement

and removed

question

labels 2021-03-10 21:45:05 +00:00

decentral1se added this to the Beta release milestone 2021-03-10 21:45:10 +00:00

decentral1se changed title from ~~Should app versioning cover all services in the app definition?~~ to Versioning logic should include all underlying services

2021-03-10 21:45:39 +00:00

decentral1se referenced this issue from coop-cloud/abra

2021-03-10 21:46:46 +00:00

Versioning roadmap guidance #90

decentral1se referenced this issue from a commit

2021-03-13 17:32:17 +00:00

Use all services in new versioning scheme

decentral1se commented

2021-03-13 17:37:21 +00:00

➜  ~ docker service inspect -f '{{index .Spec.Labels "coop-cloud.gitea.app.version" }}' gitea_app
1.13.4-078f8918
➜  ~ docker service inspect -f '{{index .Spec.Labels "coop-cloud.gitea.db.version" }}' gitea_db  
10.5-e27cf5bc

I will try to work this into abra handling now.

``` ➜ ~ docker service inspect -f '{{index .Spec.Labels "coop-cloud.gitea.app.version" }}' gitea_app 1.13.4-078f8918 ➜ ~ docker service inspect -f '{{index .Spec.Labels "coop-cloud.gitea.db.version" }}' gitea_db 10.5-e27cf5bc ``` I will try to work this into `abra` handling now.

decentral1se commented

2021-03-13 19:10:21 +00:00

OK, I am finally sold on the idea that we should vendor yq because I can list all services from a compose.yml file like so: yq e '.services | keys | .[]' compose.yml. When we take into account that we'll need to handle multiple compose.*.yml files which overload services then it starts to complicated to read the services list and feed that into the version checks.

It seems pretty chill to programmatically download that. I might stick it in .abra/vendor using these docs https://mikefarah.gitbook.io/yq/#install to get the download logic.

OK, I am finally sold on the idea that we should vendor [yq](https://mikefarah.gitbook.io/yq/) because I can list all services from a `compose.yml` file like so: `yq e '.services | keys | .[]' compose.yml`. When we take into account that we'll need to handle multiple `compose.*.yml` files which overload services then it starts to complicated to read the services list and feed that into the version checks. It seems pretty chill to programmatically download that. I might stick it in `.abra/vendor` using these docs https://mikefarah.gitbook.io/yq/#install to get the download logic.

decentral1se commented

2021-03-13 19:20:12 +00:00

If we have an app that has two compose files (c1 and c2) and in c1 we have two services (s1 and s2) then in c2 we overload s2 and define a third service (s3) then we're into some serious edge-case handling for different users who might use c2 (and therefore an update to s3 must be included) or might only use c1 (and then we never care about an upgrade to s3).

If this makes no sense ignore me I am mostly typing this out for myself haha.

I think we need a way for abra to know exactly which compose files it is looking at and exactly which services it will be updating. This feels like a lot of extra parsing work or maybe it can be solved with some manual packager work but it feels a bit too complicated and I want to keep this stuff simple.

Gonna sit on this for a bit.

If we have an app that has two compose files (c1 and c2) and in c1 we have two services (s1 and s2) then in c2 we overload s2 and define a third service (s3) then we're into some serious edge-case handling for different users who might use c2 (and therefore an update to s3 must be included) or might only use c1 (and then we never care about an upgrade to s3). If this makes no sense ignore me I am mostly typing this out for myself haha. I think we need a way for `abra` to know exactly which compose files it is looking at and exactly which services it will be updating. This feels like a lot of extra parsing work or maybe it can be solved with some manual packager work but it feels a bit too complicated and I want to keep this stuff simple. Gonna sit on this for a bit.

decentral1se commented

2021-03-13 19:57:14 +00:00

Oh, here is a rough algorithm:

parse the COMPOSE_FILE env var in sub_app_deploy
for each file defined there, parse and gather the services list
for each service, read the deployed version via docker inspect
for each service, read the app repo defined version
draw up a comparison summary which includes all relevant services

And don't forget to handle the case where everything is updated (no-op).

Oh, here is a rough algorithm: - parse the `COMPOSE_FILE` env var in `sub_app_deploy` - for each file defined there, parse and gather the services list - for each service, read the deployed version via docker inspect - for each service, read the app repo defined version - draw up a comparison summary which includes all relevant services And don't forget to handle the case where everything is updated (no-op).

decentral1se referenced this issue from a commit

2021-03-15 17:31:12 +00:00

WIP version output summary which handles services

decentral1se commented

2021-03-15 17:32:34 +00:00

e99bedf9e4 implements #47 (comment). This is nearly there. I just need to add edge-case handling, missing versions and integrating with the --update flag. I'll track that over in coop-cloud/abra#90.

https://git.autonomic.zone/coop-cloud/abra/commit/e99bedf9e4b46f73b67a22d46998d5dc60c2af74 implements https://git.autonomic.zone/coop-cloud/organising/issues/47#issuecomment-4232. This is nearly there. I just need to add edge-case handling, missing versions and integrating with the `--update` flag. I'll track that over in https://git.autonomic.zone/coop-cloud/abra/issues/90.

decentral1se closed this issue

2021-03-15 17:32:34 +00:00

This repo is archived. You cannot comment on issues.