capsul-flask/docs/testing.md

automated testing

Automated tests could make it safer to contribute code, easier to review new code, and much easier to refactor, or upgrade Python dependencies.

To run tests:

1. create a Postgres database called capsulflask_test

• e.g.: docker exec -it 98e1ddfbbffb createdb -U postgres -O postgres capsulflask_test
  • (98e1ddfbbffb is the docker container ID of the postgres container)

2. run python3 -m unittest; cat unittest-log-output.log; rm unittest-log-output.log

NOTE that right now we can't figure out how to get the tests to properly output the log messages that happened when they failed, (or passed), so for now we have hacked it to write to a file.

Architecture

I tried to make the absolute minimal changes to be able to override settings in tests – possible alternative approaches include accepting an argument to create_app() to define which env file to load, or adding conditional logic to create_app() to pre-load specific settings before running load_dotenv() – but allowing env vars to override dotenv vars seemed cleanest. (Thanks @forest for improving on this approach)

Creating test databases

One outstanding question is how to initialise/reinitialise the test database.

Currently, the tests rely on the existence of a capsulflask_test database on localhost, accessible by the postgres user with password dev.

I create this manually using: docker exec -it 98e1ddfbbffb createdb -U postgres -O postgres capsulflask_test

where 98e1ddfbbffb is the docker container ID of the postgres container.

In between test runs, you can either drop and recreate that database, or manually clear data using:

docker exec -it 98e1ddfbbffb psql -U postgres capsulflask_test -c "DELETE FROM vm_ssh_authorized_key; DELETE FROM vm_ssh_host_key; DELETE FROM vms; DELETE FROM login_tokens; DELETE FROM ssh_public_keys; DELETE FROM unresolved_btcpay_invoices; DELETE FROM payments; DELETE FROM payment_sessions; DELETE FROM host_operation; DELETE FROM operations; DELETE FROM api_tokens; DELETE FROM accounts;"

Test coverage

This tests the "landing" (public) pages, login, capsul index and creation. I didn't add automated coverage reporting yet, unclear if that seems useful.