Make backup-bot configuration separate from each app configuration #2

Open
opened 2020-09-18 08:43:03 +00:00 by decentral1se · 4 comments
Owner

As mentioned in compose-stacks/wordpress#7 (comment):

The only downside I can see right now is that each compose-stack would have to have its own backup service there and it is already some 20 lines and another configuration. We can probably extract this to a separate compose.yml and run it separately like the renovate-bot at some point when we prove to ourselves this is a good way to go. That would just need some wrangling to get the remote connection strings of the database since the service would not be in the same compose.yml definition anymore.

In practice, this would mean we'd set it up something like https://git.autonomic.zone/autonomic-cooperative/renovate-bot where we have a separate repo with all the app connection details we want to do backups for are listed and then it runs on some fixed schedule. The main advantage of this then would be that it would live outside of the app configuration and can be easily re-worked and changed without altering the app configuration.

As mentioned in https://git.autonomic.zone/compose-stacks/wordpress/issues/7#issuecomment-1157: > The only downside I can see right now is that each compose-stack would have to have its own backup service there and it is already some 20 lines and another configuration. We can probably extract this to a separate compose.yml and run it separately like the renovate-bot at some point when we prove to ourselves this is a good way to go. That would just need some wrangling to get the remote connection strings of the database since the service would not be in the same compose.yml definition anymore. In practice, this would mean we'd set it up something like https://git.autonomic.zone/autonomic-cooperative/renovate-bot where we have a separate repo with all the app connection details we want to do backups for are listed and then it runs on some fixed schedule. The main advantage of this then would be that it would live outside of the app configuration and can be easily re-worked and changed without altering the app configuration.
decentral1se self-assigned this 2020-09-18 08:43:03 +00:00
Author
Owner

Think this is maybe not possible but I am not sure. It will require quite a deep dive into docker networking, it seems. Looks like the place to start digging is over in https://docs.docker.com/network/network-tutorial-overlay/. Basically, the bot, running independently in its own swarm service would need to peak out of that service scope and into other services (the ones it wants to backup). It would also need additional secret loading on a per-service basis. I am not sure how this would look in practice but it seems worth investigating because otherwise, we'll need to copy/pasta 20 lines into every application compose.yml.

Think this is maybe not possible but I am not sure. It will require quite a deep dive into docker networking, it seems. Looks like the place to start digging is over in https://docs.docker.com/network/network-tutorial-overlay/. Basically, the bot, running independently in its own swarm service would need to peak out of that service scope and into other services (the ones it wants to backup). It would also need additional secret loading on a per-service basis. I am not sure how this would look in practice but it seems worth investigating because otherwise, we'll need to copy/pasta 20 lines into every application compose.yml.
Author
Owner

Starting point is compose-stacks/wordpress#7 (comment) which shows a working backup-bot configuration and all the bells and whistles required to get it running.

Starting point is https://git.autonomic.zone/compose-stacks/wordpress/issues/7#issuecomment-1197 which shows a working backup-bot configuration and all the bells and whistles required to get it running.
Author
Owner

Ooooor we just do this!

https://git.autonomic.zone/compose-stacks/wordpress/src/branch/master/compose.backup.yml

And then let the deploy take care of picking that up.

Much easier!

Ooooor we just do this! > https://git.autonomic.zone/compose-stacks/wordpress/src/branch/master/compose.backup.yml And then let the deploy take care of picking that up. Much easier!
Author
Owner

OK, re-opening because we discussed and it seems needed. I think we need a docker bridge network in the end following https://docs.docker.com/network/bridge/#differences-between-user-defined-bridges-and-the-default-bridge because "On a user-defined bridge network, containers can resolve each other by name or alias.". So, we'd need to make a user-defined bridge network that all apps sit on and then we could access mysql databases from isolated, one-time-run back-up containers run outside of the context of a stack/compose.yml. At least this is what I undestand now...

OK, re-opening because we discussed and it seems needed. I think we need a docker bridge network in the end following https://docs.docker.com/network/bridge/#differences-between-user-defined-bridges-and-the-default-bridge because "On a user-defined bridge network, containers can resolve each other by name or alias.". So, we'd need to make a user-defined bridge network that all apps sit on and then we could access mysql databases from isolated, one-time-run back-up containers run outside of the context of a stack/compose.yml. At least this is what I undestand now...
decentral1se removed their assignment 2020-10-23 14:21:05 +00:00
This repo is archived. You cannot comment on issues.
No Label
No Milestone
No Assignees
1 Participants
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: coop-cloud/backup-bot#2
No description provided.