Add preliminary automated tests #1

3wordchant · 2021-07-19T21:09:26Z

3wordchant commented

2021-07-19 21:09:26 +00:00

As discussed in #services:cyberia.club — automated tests could make it safer to contribute code, easier to review new code, and much easier to refactor, or upgrade Python dependencies.

To run tests:

create a Postgres database called capsulflask_test
run python -m unittest

Architecture

I tried to make the absolute minimal changes to be able to override settings in tests – possible alternative approaches include accepting an argument to create_app() to define which env file to load, or adding conditional logic to create_app() to pre-load specific settings before running load_dotenv() – but allowing env vars to override dotenv vars seemed cleanest. (Thanks @forest for improving on this approach)

Creating test databases

One outstanding question is how to initialise/reinitialise the test database.

Currently, the tests rely on the existence of a capsulflask_test database on localhost, accessible by the postgres user with password dev.

I create this manually using:
docker exec -it d1702306f409 psql -U postgres createdb -O postgres capsulflask_test;

In between test runs, you can either drop and recreate that database, or manually clear data using:

docker exec -it d1702306f409 psql -U postgres capsulflask_test -c "DELETE FROM vms; DELETE FROM login_tokens; DELETE FROM ssh_public_keys; DELETE FROM api_tokens; DELETE FROM accounts;

Test coverage

This tests the "landing" (public) pages, login, capsul index and creation. I didn't add automated coverage reporting yet, unclear if that seems useful.

As discussed in #services:cyberia.club — automated tests could make it safer to contribute code, easier to review new code, and much easier to refactor, or upgrade Python dependencies. To run tests: 1. create a Postgres database called `capsulflask_test` 2. run `python -m unittest` ## Architecture I tried to make the absolute minimal changes to be able to override settings in tests – possible alternative approaches include accepting an argument to create_app() to define which env file to load, or adding conditional logic to create_app() to pre-load specific settings before running load_dotenv() – but allowing env vars to override dotenv vars seemed cleanest. (Thanks @forest for improving on this approach) ## Creating test databases One outstanding question is how to initialise/reinitialise the test database. Currently, the tests rely on the existence of a capsulflask_test database on localhost, accessible by the postgres user with password dev. I create this manually using: ` docker exec -it d1702306f409 psql -U postgres createdb -O postgres capsulflask_test;` In between test runs, you can either drop and recreate that database, or manually clear data using: `docker exec -it d1702306f409 psql -U postgres capsulflask_test -c "DELETE FROM vms; DELETE FROM login_tokens; DELETE FROM ssh_public_keys; DELETE FROM api_tokens; DELETE FROM accounts;` ## Test coverage This tests the "landing" (public) pages, login, capsul index and creation. I didn't add automated coverage reporting yet, unclear if that seems useful.

3wordchant added 4 commits 2021-07-19 21:09:27 +00:00

665e087bd4 Basic testing using flask-testing

This commit makes it possible to override settings during tests, by
switching capsulflask/__init__.py to a "create_app" pattern, and using
`dotenv_values` instead of `load_dotenv`.

The create_app() method returns a Flask app instance, to give
more control over when to initialise the app. This allows setting
environment variables in test files.

Then, use dotenv_values to override loaded .env variables with ones from
the environment, so that tests can set `POSTGRES_CONNECTION_PARAMETERS`
and `SPOKE_MODEL` (possibly others in future..).

Inital tests for the "landing" pages, and login / activation, are
included.

4c7caa1a38 Initial console tests

NB capsul create isn't working properly, see #83

9823bda4a2 Add SSH key tests

7b9e9debab Disable VM creation check for the moment

decentral1se approved these changes 2021-07-20 21:09:08 +00:00

decentral1se left a comment

LGTM! Super nice to get some tests in.

FYI, I will note that Pytest is the de facto standard for testing which is often picked up instead of good 'ol nosetest. It has a plugin in https://pytest-flask.readthedocs.io/en/latest/ and there is a rich ecosystem of plugins in general in https://docs.pytest.org/en/latest/reference/plugin_list.html. For example, there is a plugin to help with setting up + wiping the database between tests: https://pypi.org/project/pytest-postgresql/.

LGTM! Super nice to get some tests in. FYI, I will note that Pytest is the de facto standard for testing which is often picked up instead of good 'ol nosetest. It has a plugin in https://pytest-flask.readthedocs.io/en/latest/ and there is a rich ecosystem of plugins in general in https://docs.pytest.org/en/latest/reference/plugin_list.html. For example, there is a plugin to help with setting up + wiping the database between tests: https://pypi.org/project/pytest-postgresql/.

3wordchant commented

2021-07-20 21:54:30 +00:00

Ha, I kept flip flopping between unittest and pytest and ended up going with unittest (via flask-testing) copypastad from a previous project.

Do you think it's better to go with pytest straight from the start, if we're going to? i.e. is it worth merging this as-is or should we switch first?

Ha, I kept flip flopping between `unittest` and `pytest` and ended up going with `unittest` (via flask-testing) copypastad from a previous project. Do you think it's better to go with pytest straight from the start, if we're going to? i.e. is it worth merging this as-is or should we switch first?

decentral1se commented

2021-07-20 22:09:47 +00:00

You can get Pytest to run your Nosetests so I think you can just merge this as-is. See https://docs.pytest.org/en/6.2.x/nose.html for more.

👀 1

3wordchant force-pushed tests from 0268e4c97d to 62c7355b4c

2021-07-23 01:05:12 +00:00

Compare

3wordchant added 1 commit 2021-07-23 16:01:07 +00:00

continuous-integration/drone/pr Build is failing Details