Add libvirt-client to Docker image

.. using server-side API tokens

.drone.yml

@@ -0,0 +1,14 @@
+---
+kind: pipeline
+name: publish docker image
+steps:
+  - name: build and publish
+    image: plugins/docker
+    settings:
+      username:
+        from_secret: docker_reg_username_3wc
+      password:
+        from_secret: docker_reg_passwd_3wc
+      repo: 3wordchant/capsul-flask
+      tags: latest
+

.gitignore

@@ -1,6 +1,5 @@
 notes.txt
 .env
-.env.bak
 .vscode
 
 *.pyc

Dockerfile

@@ -0,0 +1,32 @@
+FROM python:3.8-alpine as build
+
+RUN apk add gettext git gcc python3-dev musl-dev \
+    libffi-dev zlib-dev jpeg-dev libjpeg postgresql-dev build-base \
+    --virtual .build-dependencies
+
+RUN mkdir -p /app/{code,venv}
+WORKDIR /app/code
+COPY Pipfile Pipfile.lock /app/code/
+
+RUN python3 -m venv /app/venv
+RUN pip install pipenv setuptools
+ENV PATH="/app/venv/bin:$PATH" VIRTUAL_ENV="/app/venv"
+RUN pip install wheel cppy
+# Install dependencies into the virtual environment with Pipenv
+RUN pipenv install --deploy --verbose
+
+FROM python:3.8-alpine
+
+RUN apk add --no-cache libpq libstdc++ libjpeg libvirt-client
+
+COPY . /app/code/
+WORKDIR /app/code
+
+COPY --from=build /app/venv /app/venv
+ENV PATH="/app/venv/bin:$PATH" VIRTUAL_ENV="/app/venv"
+
+CMD ["gunicorn", "--bind", "0.0.0.0:5000", "-k", "gevent", "--worker-connections", "1000", "app:app"]
+
+VOLUME /app/code
+
+EXPOSE 5000

README.md

@@ -1,51 +1,227 @@
-# capsul-flask
+# capsulflask
 
-![screenshot of capsul.org home page](./docs/capsul.webp)
+Python Flask web application for capsul.org
 
-Python Flask web application implementing user accounts, payment, and virtual machine management for a smol "virtual machine (vm) as a service" aka "cloud compute" provider.  Originally developed by [Cyberia Computer Club](https://cyberia.club) for https://capsul.org
 
-`capsul-flask` integrates with [Stripe](https://stripe.com/) as a credit card processor, and [BTCPay Server](https://github.com/btcpayserver/btcpayserver-docker) as a cryptocurrency payment processor.
+## how to run locally
 
-`capsul-flask` invokes [shell-scripts](./capsulflask/shell_scripts/) to create/manage [libvirt/qemu](https://www.libvirt.org/manpages/virsh.html) vms, and it depends on `dnsmasq` to act as the DHCP server for the vms.
+Ensure you have the pre-requisites for the psycopg2 Postgres database adapter package
-
-`capsul-flask` has a ["hub and spoke" architecture](./architecture.md). The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"(s) are responsible for creating/managing virtual machines. In this way, capsul can be scaled to span more than one machine. One instance of the capsul-flask application can run in both hub mode and spoke mode at the same time, however there must only be one instance of the app running in "Hub" mode at any given time.
-
-## Quickstart (run capsul-flask on your computer in development mode)
 
 ```
-# get an instance of postgres running locally on port 5432
+sudo apt install python3-dev libpq-dev
-# (you don't have to use docker, but we thought this might be the easiest for a how-to example)
+pg_config --version
-docker run --rm -it -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres &
+```
 
-# install dependencies 
+Ensure you have the wonderful `pipenv` python package management and virtual environment cli
-sudo apt install pipenv python3-dev libpq-dev
 
-# download and run
+```
-git clone https://giit.cyberia.club/~forest/capsul-flask
+sudo apt install pipenv
-cd capsul-flask
+```
+
+Create python virtual environment and install packages
+
+```
+# install deps
 pipenv install
+```
+
+Run an instance of Postgres (I used docker for this, you can use whatever you want, point is its listening on localhost:5432)
+
+```
+docker run --rm -it -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres
+```
+
+Run the app 
+
+```
 pipenv run flask run
 ```
 
-Interested in learning more? How about a trip to the the `docs/` folder:
+Run the app in gunicorn:
 
-- [**Setting up capsul-flask locally**](./docs/local-set-up.md)
+```
-  - [Manually](./docs/local-set-up.md#manually)
+pipenv run gunicorn --bind 127.0.0.1:5000 -k gevent --worker-connections 1000 app:app
-  - [With docker-compose](./docs/local-set-up.md#docker_compose)
+```
-- [**Configuring `capsul-flask`**](./docs/configuration.md)
+
-  - [Example configuration from capsul.org (production)](./docs/configuration.md#example)
+Once you log in for the first time, you will want to give yourself some free capsulbux so you can create fake capsuls for testing. 
-  - [Configuration-type-stuff that lives in the database ](./docs/configuration.md#config_that_lives_in_db)
+
-  - [Loading variables from files (docker secrets)](./docs/configuration.md#docker_secrets)
+Note that by default when running locally, the `SPOKE_MODEL` is set to `mock`, meaning that it won't actually try to spawn vms.
-- [**`capsul-flask`'s relationship to its Database Server**](./docs/database.md)
+
-  - [Database schema management (schema versions)](./docs/database.md#schema_management)
+```
-  - [Running manual database queries](./docs/database.md#manual_queries)
+pipenv run flask cli sql -c "INSERT INTO payments (email, dollars) VALUES ('<your email address here>', 20.00)"
-- [**`capsul-flask`'s hub-and-spoke architecture**](./docs/architecture.md)
+```
-- [**Deploying capsul-flask on a server**](./docs/deployment.md)
+
-  - [Installing prerequisites for Spoke Mode](./docs/deployment.md#spoke_mode_prerequisites)
+## configuration:
-  - [Deploying capsul-flask manually](./docs/deployment.md#deploy_manually)
+
-  - [Deploying capsul-flask with coop-cloud's docker-swarm configuration](./docs/deployment.md#coop_cloud_docker)
+Create a `.env` file to set up the application configuration:
-  - [Deploying capsul-flask with coop-cloud's `abra` deployment tool](./docs/deployment.md#coop_cloud_abra)
+
-- [**Accepting cryptocurrency payments with BTCPay Server**](./docs/btcpay.md)
+```
-  - [Setting up the BTCPAY_PRIVATE_KEY](./docs/btcpay.md#BTCPAY_PRIVATE_KEY)
+nano .env
-  - [Testing cryptocurrency payments](./docs/btcpay.md#testing)
+```
-  - [Sequence diagram explaining how BTC payment process works (how we accept 0-confirmation transactions 😀)](./docs/btcpay.md#0_conf_diagram)
+
+You can enter any environment variables referenced in `__init__.py` to this file.
+
+For example you may enter your SMTP credentials like this:
+```
+MAIL_USERNAME=forest@nullhex.com
+MAIL_DEFAULT_SENDER=forest@nullhex.com
+MAIL_PASSWORD=**************
+```
+
+## how to view the logs on the database server (legion.cyberia.club)
+
+`sudo -u postgres pg_dump capsul-flask | gzip -9 > capsul-backup-2021-02-15.gz`
+
+-----
+
+## cli 
+
+You can manually mess around with the database like this:
+
+```
+pipenv run flask cli sql -f test.sql
+```
+
+```
+pipenv run flask cli sql -c 'SELECT * FROM vms'
+```
+
+This one selects the vms table with the column name header:
+
+```
+pipenv run flask cli sql -c "SELECT string_agg(column_name::text, ', ') from information_schema.columns WHERE table_name='vms'; SELECT * from vms"
+```
+
+How to modify a payment manually, like if you get a chargeback or to fix customer payment issues:
+
+```
+$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
+1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, FALSE
+
+$ pipenv run flask cli sql -c "UPDATE payments SET invalidated = True WHERE id = 1"
+1 rows affected.
+
+$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
+1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, TRUE
+```
+
+
+How you would kick off the scheduled task:
+
+```
+pipenv run flask cli cron-task
+```
+
+-----
+
+## postgres database schema management
+
+capsulflask has a concept of a schema version. When the application starts, it will query the database for a table named
+`schemaversion` that has one row and one column (`version`). If the `version` it finds is not equal to the `desiredSchemaVersion` variable set in `db.py`, it will run  migration scripts from the `schema_migrations` folder one by one until the `schemaversion` table shows the correct version.
+
+For example, the script named `02_up_xyz.sql` should contain code that migrates the database from schema version 1 to schema version 2. Likewise, the script `02_down_xyz.sql` should contain code that migrates from schema version 2 back to schema version 1.
+
+**IMPORTANT: if you need to make changes to the schema, make a NEW schema version. DO NOT EDIT the existing schema versions.**
+
+In general, for safety, schema version upgrades should not delete data. Schema version downgrades will simply throw an error and exit for now. 
+
+-----
+
+## hub-and-spoke architecture
+
+![](readme/hub-and-spoke1.png)
+
+This diagram was created with https://app.diagrams.net/.
+To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
+
+right now I have 2 types of operations, immediate mode and async.
+
+both types of operations do assignment synchronously. so if the system cant assign the operation to one or more hosts (spokes),
+or whatever the operation requires, then it will fail. 
+
+some operations tolerate partial failures, like, `capacity_avaliable` will succeed if at least one spoke succeeds.
+for immediate mode requests (like `list`, `capacity_avaliable`, `destroy`), assignment and completion of the operation are the same thing.
+
+for async ones, they can be assigned without knowing whether or not they succeeded (`create`).
+
+![](readme/hub-and-spoke2.png)
+
+This diagram was created with https://app.diagrams.net/.
+To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
+
+if you issue a create, and it technically could go to any number of hosts, but only one host responds, it will succeed 
+but if you issue a create and somehow 2 hosts both think they own that task, it will fail and throw a big error. cuz it expects exactly 1 to own the create task
+
+currently its not set up to do any polling.  its not really like a queue at all. It's all immediate for the most part 
+
+-----
+
+## how to setup btcpay server
+
+Generate a private key and the accompanying bitpay SIN for the btcpay API client. 
+
+  I used this code as an example: https://github.com/bitpay/bitpay-python/blob/master/bitpay/key_utils.py#L6
+
+```
+$ pipenv run python ./readme/generate_btcpay_keys.py
+```
+
+It should output something looking like this:
+
+```
+-----BEGIN EC PRIVATE KEY-----
+EXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK
+oUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq
+NQ+cpBvPDbyrDk9+Uf/sEaRCma094g==
+-----END EC PRIVATE KEY-----
+
+
+EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE
+```
+
+In order to register the key with the btcpay server, you have to first generate a pairing token using the btcpay server interface.
+This requires your btcpay server account to have access to the capsul store. Ask Cass about this.
+
+Navigate to `Manage store: Access Tokens` at: `https://btcpay.cyberia.club/stores/<store-id>/Tokens`
+
+![](readme/btcpay_sin_pairing.jpg)
+
+
+Finally, send an http request to the btcpay server to complete the pairing:
+
+```
+curl -H "Content-Type: application/json" https://btcpay.cyberia.club/tokens -d "{'id': 'EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE', 'pairingCode': 'XXXXXXX'}"
+```
+
+It should respond with a token: 
+
+```
+{"data":[{"policies":[],"pairingCode":"XXXXXXX","pairingExpiration":1589473817597,"dateCreated":1589472917597,"facade":"merchant","token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","label":"capsulflask"}]}
+```
+
+And you should see the token in the btcpay server UI:
+
+![](readme/paired.jpg)
+
+Now simply set your `BTCPAY_PRIVATE_KEY` variable in `.env`
+
+NOTE: make sure to use single quotes and replace the new lines with \n.
+
+```
+BTCPAY_PRIVATE_KEY='-----BEGIN EC PRIVATE KEY-----\nEXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK\noUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq\nNQ+cpBvPDbyrDk9+Uf/sEaRCma094g==\n-----END EC PRIVATE KEY-----'
+```
+
+
+-----
+
+## testing cryptocurrency payments
+
+I used litecoin to test cryptocurrency payments, because its the simplest & lowest fee cryptocurrency that BTCPay server supports. You can download the easy-to-use litecoin SPV wallet `electrum-ltc` from [github.com/pooler/electrum-ltc](https://github.com/pooler/electrum-ltc) or [electrum-ltc.org](https://electrum-ltc.org/), set up a wallet, and then either purchase some litecoin from an exchange, or ask Forest for some litecoin to use for testing.
+
+
+## sequence diagram explaining how BTC payment process works
+
+![btcpayment_process](readme/btcpayment_process.png)
+
+This diagram was created with https://app.diagrams.net/.
+To edit it, download the <a download href="readme/btcpayment_process.drawio">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.

capsulflask/__init__.py

@@ -31,6 +31,7 @@ load_dotenv(find_dotenv())
 app = Flask(__name__)
 
 app.config.from_mapping(
+  
   BASE_URL=os.environ.get("BASE_URL", default="http://localhost:5000"),
   SECRET_KEY=os.environ.get("SECRET_KEY", default="dev"),
   HUB_MODE_ENABLED=os.environ.get("HUB_MODE_ENABLED", default="True").lower() in ['true', '1', 't', 'y', 'yes'],
@@ -71,7 +72,7 @@ app.config.from_mapping(
   #STRIPE_WEBHOOK_SECRET=os.environ.get("STRIPE_WEBHOOK_SECRET", default="")
 
   BTCPAY_PRIVATE_KEY=os.environ.get("BTCPAY_PRIVATE_KEY", default="").replace("\\n", "\n"),
-  BTCPAY_URL=os.environ.get("BTCPAY_URL", default="")
+  BTCPAY_URL=os.environ.get("BTCPAY_URL", default="https://btcpay.cyberia.club")
 )
 
 app.config['HUB_URL'] = os.environ.get("HUB_URL", default=app.config['BASE_URL'])
@@ -139,17 +140,10 @@ else:
 
 app.config['HTTP_CLIENT'] = MyHTTPClient(timeout_seconds=int(app.config['INTERNAL_HTTP_TIMEOUT_SECONDS']))
 
-app.config['BTCPAY_ENABLED'] = False
+try:
-if app.config['BTCPAY_URL'] != "":
+  app.config['BTCPAY_CLIENT'] = btcpay.Client(api_uri=app.config['BTCPAY_URL'], pem=app.config['BTCPAY_PRIVATE_KEY'])
-  try:
+except:
-    response = requests.get(app.config['BTCPAY_URL'])
+  app.logger.warning("unable to create btcpay client. Capsul will work fine except cryptocurrency payments will not work. The error was: " + my_exec_info_message(sys.exc_info()))
-    if response.status_code == 200:
-      app.config['BTCPAY_CLIENT'] = btcpay.Client(api_uri=app.config['BTCPAY_URL'], pem=app.config['BTCPAY_PRIVATE_KEY'])
-      app.config['BTCPAY_ENABLED'] = True
-    else:
-      app.logger.warning(f"Can't reach BTCPAY_URL {app.config['BTCPAY_URL']}: Response status code: {response.status_code}. Capsul will work fine except cryptocurrency payments will not work.")
-  except:
-    app.logger.warning("unable to create btcpay client. Capsul will work fine except cryptocurrency payments will not work. The error was: " + my_exec_info_message(sys.exc_info()))
 
 # only start the scheduler and attempt to migrate the database if we are running the app.
 # otherwise we are running a CLI command.
@@ -159,7 +153,6 @@ is_running_server = ('flask run' in command_line) or ('gunicorn' in command_line
 app.logger.info(f"is_running_server: {is_running_server}")
 
 if app.config['HUB_MODE_ENABLED']:
-
   if app.config['HUB_MODEL'] == "capsul-flask":
     app.config['HUB_MODEL'] = hub_model.CapsulFlaskHub()
 
@@ -181,7 +174,9 @@ if app.config['HUB_MODE_ENABLED']:
   from capsulflask import db
   db.init_app(app, is_running_server)
 
-  from capsulflask import auth, landing, console, payment, metrics, cli, hub_api, admin
+  from capsulflask import (
+    auth, landing, console, payment, metrics, cli, hub_api, publicapi, admin
+  )
 
   app.register_blueprint(landing.bp)
   app.register_blueprint(auth.bp)
@@ -191,13 +186,13 @@ if app.config['HUB_MODE_ENABLED']:
   app.register_blueprint(cli.bp)
   app.register_blueprint(hub_api.bp)
   app.register_blueprint(admin.bp)
+  app.register_blueprint(publicapi.bp)
 
   app.add_url_rule("/", endpoint="index")
 
 
 
 if app.config['SPOKE_MODE_ENABLED']:
-
   if app.config['SPOKE_MODEL'] == "shell-scripts":
     app.config['SPOKE_MODEL'] = spoke_model.ShellScriptSpoke()
   else:

capsulflask/auth.py

@@ -1,3 +1,4 @@
+from base64 import b64decode
 import functools
 import re
 
@@ -24,6 +25,15 @@ def account_required(view):
 
     @functools.wraps(view)
     def wrapped_view(**kwargs):
+        api_token = request.headers.get('authorization', None)
+        if api_token is not None:
+            email = get_model().authenticate_token(b64decode(api_token).decode('utf-8'))
+
+            if email is not None:
+                session.clear()
+                session["account"] = email
+                session["csrf-token"] = generate()
+
         if session.get("account") is None or session.get("csrf-token") is None :
             return redirect(url_for("auth.login"))
 
@@ -56,7 +66,7 @@ def login():
         if not email:
             errors.append("email is required")
         elif len(email.strip()) < 6 or email.count('@') != 1 or email.count('.') == 0: 
-	        errors.append("enter a valid email address")
+            errors.append("enter a valid email address")
 
         if len(errors) == 0:
             result = get_model().login(email)

capsulflask/cli.py

@@ -62,18 +62,6 @@ def sql_script(f, c):
 
     model.connection.commit()
 
-@bp.cli.command('account-balance')
-@click.option('-u', help='users email address')
-@with_appcontext
-def account_balance(u):
-  vms = get_model().list_vms_for_account(u)
-  payments = get_model().list_payments_for_account(u)
-  click.echo(".")
-  click.echo(".")
-  click.echo(get_account_balance(vms, payments, datetime.utcnow()))
-  click.echo(".")
-
-
 
 @bp.cli.command('cron-task')
 @with_appcontext

capsulflask/console.py

@@ -1,7 +1,9 @@
+from base64 import b64encode
+from datetime import datetime, timedelta
+import json
 import re
 import sys
-import json
+
-from datetime import datetime, timedelta
 from flask import Blueprint
 from flask import flash
 from flask import current_app
@@ -98,7 +100,6 @@ def index():
 @bp.route("/<string:id>", methods=("GET", "POST"))
 @account_required
 def detail(id):
-
   duration=request.args.get('duration')
   if not duration:
     duration = "5m"
@@ -188,6 +189,72 @@ def detail(id):
       duration=duration
     )
 
+def _create(vm_sizes, operating_systems, public_keys_for_account, server_data):
+  errors = list()
+
+  size = server_data.get("size")
+  os = server_data.get("os")
+  posted_keys_count = int(server_data.get("ssh_authorized_key_count"))
+
+  if not size:
+    errors.append("Size is required")
+  elif size not in vm_sizes:
+    errors.append(f"Invalid size {size}")
+
+  if not os:
+    errors.append("OS is required")
+  elif os not in operating_systems:
+    errors.append(f"Invalid os {os}")
+
+  posted_keys = list()
+
+  if posted_keys_count > 1000:
+    errors.append("something went wrong with ssh keys")
+  else:
+    for i in range(0, posted_keys_count):
+      if f"ssh_key_{i}" in server_data:
+        posted_name = server_data.get(f"ssh_key_{i}")
+        key = None
+        for x in public_keys_for_account:
+          if x['name'] == posted_name:
+            key = x
+        if key:
+          posted_keys.append(key)
+        else:
+          errors.append(f"SSH Key \"{posted_name}\" doesn't exist")
+
+  if len(posted_keys) == 0:
+    errors.append("At least one SSH Public Key is required")
+
+  capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(
+    vm_sizes[size]['memory_mb']*1024*1024
+  )
+
+  if not capacity_avaliable:
+    errors.append("""
+      host(s) at capacity. no capsuls can be created at this time. sorry. 
+    """)
+
+  if len(errors) == 0:
+    id = make_capsul_id()
+    get_model().create_vm(
+      email=session["account"], 
+      id=id, 
+      size=size, 
+      os=os,
+      ssh_authorized_keys=list(map(lambda x: x["name"], posted_keys))
+    )
+    current_app.config["HUB_MODEL"].create(
+      email = session["account"],
+      id=id,
+      template_image_file_name=operating_systems[os]['template_image_file_name'],
+      vcpus=vm_sizes[size]['vcpus'],
+      memory_mb=vm_sizes[size]['memory_mb'],
+      ssh_authorized_keys=list(map(lambda x: x["content"], posted_keys))
+    )
+    return id, errors
+
+  return None, errors
 
 @bp.route("/create", methods=("GET", "POST"))
 @account_required
@@ -197,8 +264,20 @@ def create():
   public_keys_for_account = get_model().list_ssh_public_keys_for_account(session["account"])
   account_balance = get_account_balance(get_vms(), get_payments(), datetime.utcnow())
   capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(512*1024*1024)
-  errors = list()
 
+  if request.method == "POST":
+    if "csrf-token" not in request.form or request.form['csrf-token'] != session['csrf-token']:
+      return abort(418, f"u want tea")
+    id, errors = _create(
+      vm_sizes, 
+      operating_systems,
+      public_keys_for_account,
+      request.form)
+    if len(errors) == 0: 
+      for error in errors:
+        flash(error)
+      return redirect(f"{url_for('console.index')}?created={id}")
+  
   affordable_vm_sizes = dict()
   for key, vm_size in vm_sizes.items():
     # if a user deposits $7.50 and then creates an f1-s vm which costs 7.50 a month, 
@@ -207,73 +286,6 @@ def create():
     if vm_size["dollars_per_month"] <= account_balance+0.25:
       affordable_vm_sizes[key] = vm_size
 
-  if request.method == "POST":
-    if "csrf-token" not in request.form or request.form['csrf-token'] != session['csrf-token']:
-      return abort(418, f"u want tea")
-
-    size = request.form["size"]
-    os = request.form["os"]
-    if not size:
-      errors.append("Size is required")
-    elif size not in vm_sizes:
-      errors.append(f"Invalid size {size}")
-    elif size not in affordable_vm_sizes:
-      errors.append(f"Your account must have enough credit to run an {size} for 1 month before you will be allowed to create it")
-
-    if not os:
-      errors.append("OS is required")
-    elif os not in operating_systems:
-      errors.append(f"Invalid os {os}")
-
-    posted_keys_count = int(request.form["ssh_authorized_key_count"])
-    posted_keys = list()
-
-    if posted_keys_count > 1000:
-      errors.append("something went wrong with ssh keys")
-    else:
-      for i in range(0, posted_keys_count):
-        if f"ssh_key_{i}" in request.form:
-          posted_name = request.form[f"ssh_key_{i}"]
-          key = None
-          for x in public_keys_for_account:
-            if x['name'] == posted_name:
-              key = x
-          if key:
-            posted_keys.append(key)
-          else:
-            errors.append(f"SSH Key \"{posted_name}\" doesn't exist")
-
-    if len(posted_keys) == 0:
-      errors.append("At least one SSH Public Key is required")
-
-    capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(vm_sizes[size]['memory_mb']*1024*1024)
-
-    if not capacity_avaliable:
-      errors.append("""
-        host(s) at capacity. no capsuls can be created at this time. sorry. 
-      """)
-
-    if len(errors) == 0:
-      id = make_capsul_id()
-      # we can't create the vm record in the DB yet because its IP address needs to be allocated first.
-      # so it will be created when the allocation happens inside the hub_api.
-      current_app.config["HUB_MODEL"].create(
-        email = session["account"],
-        id=id,
-        os=os,
-        size=size,
-        template_image_file_name=operating_systems[os]['template_image_file_name'],
-        vcpus=vm_sizes[size]['vcpus'],
-        memory_mb=vm_sizes[size]['memory_mb'],
-        ssh_authorized_keys=list(map(lambda x: dict(name=x['name'], content=x['content']), posted_keys)) 
-      )
-      
-      return redirect(f"{url_for('console.index')}?created={id}")
-  
-
-  for error in errors:
-    flash(error)
-
   if not capacity_avaliable:
     current_app.logger.warning(f"when capsul capacity is restored, send an email to {session['account']}")
 
@@ -290,23 +302,25 @@ def create():
     vm_sizes=affordable_vm_sizes
   )
 
-@bp.route("/ssh", methods=("GET", "POST"))
+@bp.route("/keys", methods=("GET", "POST"))
 @account_required
-def ssh_public_keys():
+def ssh_api_keys():
   errors = list()
 
+  token = None
+
   if request.method == "POST":
     if "csrf-token" not in request.form or request.form['csrf-token'] != session['csrf-token']:
       return abort(418, f"u want tea")
       
-    method = request.form["method"]
+    action = request.form["action"]
-    content = None
+
-    if method == "POST":
+    if action == 'upload_ssh_key':
+      content = None
       content = request.form["content"].replace("\r", " ").replace("\n", " ").strip()
-      
+        
-    name = request.form["name"]
+      name = request.form["name"]
-    if not name or len(name.strip()) < 1:
+      if not name or len(name.strip()) < 1:
-      if method == "POST":
         parts = re.split(" +", content)
         if len(parts) > 2 and len(parts[2].strip()) > 0:
           name = parts[2].strip()
@@ -314,10 +328,9 @@ def ssh_public_keys():
           name = parts[0].strip()
       else:
         errors.append("Name is required")
-    if not re.match(r"^[0-9A-Za-z_@:. -]+$", name):
+      if not re.match(r"^[0-9A-Za-z_@:. -]+$", name):
-      errors.append(f"Key name '{name}' must match \"^[0-9A-Za-z_@:. -]+$\"")
+        errors.append(f"Key name '{name}' must match \"^[0-9A-Za-z_@:. -]+$\"")
 
-    if method == "POST":
       if not content or len(content.strip()) < 1:
         errors.append("Content is required")
       else:
@@ -330,24 +343,36 @@ def ssh_public_keys():
       if len(errors) == 0:
         get_model().create_ssh_public_key(session["account"], name, content)
 
-    elif method == "DELETE":
+    elif action == "delete_ssh_key":
+      get_model().delete_ssh_public_key(session["account"], name)
 
-      if len(errors) == 0:
+    elif action == "generate_api_token":
-        get_model().delete_ssh_public_key(session["account"], name)
+      name = request.form["name"]
+      if name == '':
+          name = datetime.utcnow().strftime('%y-%m-%d %H:%M:%S')
+      token = b64encode(
+        get_model().generate_api_token(session["account"], name).encode('utf-8')
+      ).decode('utf-8')
+
+    elif action == "delete_api_token":
+      get_model().delete_api_token(session["account"], request.form["id"])
 
   for error in errors:
     flash(error)
 
-  keys_list=list(map(
+  ssh_keys_list=list(map(
     lambda x: dict(name=x['name'], content=f"{x['content'][:20]}...{x['content'][len(x['content'])-20:]}"), 
     get_model().list_ssh_public_keys_for_account(session["account"])
   ))
 
+  api_tokens_list = get_model().list_api_tokens(session["account"])
+
   return render_template(
-    "ssh-public-keys.html", 
+    "keys.html", 
     csrf_token = session["csrf-token"],
-    ssh_public_keys=keys_list, 
+    api_tokens=api_tokens_list, 
-    has_ssh_public_keys=len(keys_list) > 0
+    ssh_public_keys=ssh_keys_list,
+    generated_api_token=token,
   )
 
 def get_vms():
@@ -371,7 +396,6 @@ def get_vm_months_float(vm, as_of):
   return days / average_number_of_days_in_a_month
 
 def get_account_balance(vms, payments, as_of):
-
   vm_cost_dollars = 0.0
   for vm in vms:
     vm_months = get_vm_months_float(vm, as_of)
@@ -384,7 +408,6 @@ def get_account_balance(vms, payments, as_of):
 @bp.route("/account-balance")
 @account_required
 def account_balance():
-
   payment_sessions = get_model().list_payment_sessions_for_account(session['account'])
   for payment_session in payment_sessions:
     if payment_session['type'] == 'btcpay':
@@ -426,7 +449,6 @@ def account_balance():
     has_vms=len(vms_billed)>0, 
     vms_billed=vms_billed,
     warning_text=warning_text,
-    btcpay_enabled=current_app.config["BTCPAY_ENABLED"],
     payments=list(map(
       lambda x: dict(
         dollars=x["dollars"], 

capsulflask/db.py

@@ -33,7 +33,7 @@ def init_app(app, is_running_server):
     result = re.search(r"^\d+_(up|down)", filename)
     if not result:
       app.logger.error(f"schemaVersion {filename} must match ^\\d+_(up|down). exiting.")
-      exit(1)
+      continue
     key = result.group()
     with open(join(schemaMigrationsPath, filename), 'rb') as file:
       schemaMigrations[key] = file.read().decode("utf8")
@@ -43,7 +43,7 @@ def init_app(app, is_running_server):
   hasSchemaVersionTable = False
   actionWasTaken = False
   schemaVersion = 0
-  desiredSchemaVersion = 20
+  desiredSchemaVersion = 18
 
   cursor = connection.cursor()
 
@@ -128,4 +128,3 @@ def close_db(e=None):
   if db_model is not None:
     db_model.cursor.close()
     current_app.config['PSYCOPG2_CONNECTION_POOL'].putconn(db_model.connection)
-

capsulflask/db_model.py

@@ -1,8 +1,8 @@
-
 import re
 
 # I was never able to get this type hinting to work correctly 
 # from psycopg2.extensions import connection as Psycopg2Connection, cursor as Psycopg2Cursor
+import hashlib
 from nanoid import generate
 from flask import current_app
 from typing import List
@@ -17,7 +17,6 @@ class DBModel:
     self.cursor = cursor
 
 
-
   #     ------    LOGIN     ---------     
 
 
@@ -43,6 +42,16 @@ class DBModel:
     self.connection.commit()
 
     return (token, ignoreCaseMatches)
+
+  def authenticate_token(self, token):
+    m = hashlib.md5()
+    m.update(token.encode('utf-8'))
+    hash_token = m.hexdigest()
+    self.cursor.execute("SELECT email FROM api_tokens WHERE token = %s", (hash_token, ))
+    result = self.cursor.fetchall()
+    if len(result) == 1:
+      return result[0]
+    return None
     
   def consume_token(self, token):
     self.cursor.execute("SELECT email FROM login_tokens WHERE token = %s and created > (NOW() - INTERVAL '20 min')", (token, ))
@@ -132,6 +141,32 @@ class DBModel:
     self.cursor.execute( "DELETE FROM ssh_public_keys where email = %s AND name = %s", (email, name) )
     self.connection.commit()
 
+  def list_api_tokens(self, email):
+    self.cursor.execute(
+      "SELECT id, token, name, created FROM api_tokens WHERE email = %s", 
+      (email, )
+    )
+    return list(map(
+      lambda x: dict(id=x[0], token=x[1], name=x[2], created=x[3]), 
+      self.cursor.fetchall()
+    ))
+
+  def generate_api_token(self, email, name):
+    token = generate()
+    m = hashlib.md5()
+    m.update(token.encode('utf-8'))
+    hash_token = m.hexdigest()
+    self.cursor.execute(
+      "INSERT INTO api_tokens (email, name, token) VALUES (%s, %s, %s)", 
+      (email, name, hash_token)
+    )
+    self.connection.commit()
+    return token
+
+  def delete_api_token(self, email, id_):
+    self.cursor.execute( "DELETE FROM api_tokens where email = %s AND id = %s", (email, id_))
+    self.connection.commit()
+
   def list_vms_for_account(self, email):
     self.cursor.execute(""" 
       SELECT vms.id, vms.public_ipv4, vms.public_ipv6, vms.size, vms.os, vms.created, vms.deleted, vm_sizes.dollars_per_month
@@ -479,8 +514,3 @@ class DBModel:
     #cursor.close()
 
     return to_return
-      
-
-
-
-

capsulflask/hub_api.py

@@ -160,10 +160,7 @@ def can_claim_create(payload, host_id) -> (str, str):
   if allocated_network_name is None or allocated_ipv4_address is None:
     return "", f"host \"{host_id}\" does not have any avaliable IP addresses on any of its networks."
 
-  # payload["network_name"] = allocated_network_name
+  payload["network_name"] = allocated_network_name
-  # hard-code the network name for now until we can fix the phantom dhcp lease issues.
-
-  payload["network_name"] = 'public3'
   payload["public_ipv4"] = allocated_ipv4_address
 
   return payload, ""

capsulflask/hub_model.py

@@ -17,10 +17,6 @@ from capsulflask.http_client import HTTPResult
 from capsulflask.shared import VirtualizationInterface, VirtualMachine, OnlineHost, validate_capsul_id, my_exec_info_message
 
 class MockHub(VirtualizationInterface):
-  def __init__(self):
-    self.default_network = "public1"
-    self.default_ipv4 = "1.1.1.1"
-
   def capacity_avaliable(self, additional_ram_bytes):
     return True
 
@@ -33,9 +29,9 @@ class MockHub(VirtualizationInterface):
         {"key_type":"RSA", "content":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvotgzgEP65JUQ8S8OoNKy1uEEPEAcFetSp7QpONe6hj4wPgyFNgVtdoWdNcU19dX3hpdse0G8OlaMUTnNVuRlbIZXuifXQ2jTtCFUA2mmJ5bF+XjGm3TXKMNGh9PN+wEPUeWd14vZL+QPUMev5LmA8cawPiU5+vVMLid93HRBj118aCJFQxLgrdP48VPfKHFRfCR6TIjg1ii3dH4acdJAvlmJ3GFB6ICT42EmBqskz2MPe0rIFxH8YohCBbAbrbWYcptHt4e48h4UdpZdYOhEdv89GrT8BF2C5cbQ5i9qVpI57bXKrj8hPZU5of48UHLSpXG8mbH0YDiOQOfKX/Mt", "sha256":"ghee6KzRnBJhND2kEUZSaouk7CD6o6z2aAc8GPkV+GQ"},
         {"key_type":"ECDSA", "content":"ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLLgOoATz9R4aS2kk7vWoxX+lshK63t9+5BIHdzZeFE1o+shlcf0Wji8cN/L1+m3bi0uSETZDOAWMP3rHLJj9Hk=", "sha256":"aCYG1aD8cv/TjzJL0bi9jdabMGksdkfa7R8dCGm1yYs"}
       ]""")
-      return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4=self.default_ipv4, ssh_host_keys=ssh_host_keys)
+      return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4="1.1.1.1", ssh_host_keys=ssh_host_keys)
 
-    return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4=self.default_ipv4)
+    return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4="1.1.1.1")
 
   def list_ids(self) -> list:
     return get_model().all_non_deleted_vm_ids()
@@ -44,16 +40,6 @@ class MockHub(VirtualizationInterface):
     validate_capsul_id(id)
     current_app.logger.info(f"mock create: {id} for {email}")
     sleep(1)
-    get_model().create_vm(
-      email=email, 
-      id=id, 
-      size=size, 
-      os=os,
-      host=current_app.config["SPOKE_HOST_ID"],
-      network_name=self.default_network,
-      public_ipv4=self.default_ipv4,
-      ssh_authorized_keys=list(map(lambda x: x["name"], ssh_authorized_keys)),
-    )
 
   def destroy(self, email: str, id: str):
     current_app.logger.info(f"mock destroy: {id} for {email}")
@@ -63,6 +49,7 @@ class MockHub(VirtualizationInterface):
 
 
 class CapsulFlaskHub(VirtualizationInterface):
+
   def synchronous_operation(self, hosts: List[OnlineHost], email: str, payload: str) -> List[HTTPResult]:
     return self.generic_operation(hosts, email, payload, True)[1]
   
@@ -228,8 +215,8 @@ class CapsulFlaskHub(VirtualizationInterface):
         # no need to do anything here since if it cant be parsed then generic_operation will handle it.
         pass
       
-    assigned_hosts_string = ", ".join(assigned_hosts)
     if number_of_assigned != 1:
+      assigned_hosts_string = ", ".join(assigned_hosts)
       raise ValueError(f"expected create capsul operation {operation_id} to be assigned to one host, it was assigned to {number_of_assigned} ({assigned_hosts_string})")
     if error_message != "":
       raise ValueError(f"create capsul operation {operation_id} on {assigned_hosts_string} failed with {error_message}")
@@ -275,3 +262,4 @@ class CapsulFlaskHub(VirtualizationInterface):
 
     if not result_status == "success":
       raise ValueError(f"""failed to {command} vm "{id}" on host "{host.id}" for {email}: {result_json_string}""")
+

capsulflask/landing.py

@@ -12,11 +12,9 @@ def index():
 
 @bp.route("/pricing")
 def pricing():
-  vm_sizes = get_model().vm_sizes_dict()
   operating_systems = get_model().operating_systems_dict()
   return render_template(
     "pricing.html", 
-    vm_sizes=vm_sizes,
     operating_systems=operating_systems
   )
 

capsulflask/payment.py

@@ -48,29 +48,20 @@ def validate_dollars():
 def btcpay_payment():
   errors = list()
 
-  if not current_app.config['BTCPAY_ENABLED']:
-    flash("BTCPay is not enabled on this server")
-    return redirect(url_for("console.account_balance"))
-
   if request.method == "POST":
     result = validate_dollars()
     errors = result[0]
     dollars = result[1]
 
     if len(errors) == 0:
-      try:
+      invoice = current_app.config['BTCPAY_CLIENT'].create_invoice(dict(
-        invoice = current_app.config['BTCPAY_CLIENT'].create_invoice(dict(
+        price=float(dollars), 
-          price=float(dollars), 
+        currency="USD",
-          currency="USD",
+        itemDesc="Capsul Cloud Compute",
-          itemDesc="Capsul Cloud Compute",
+        transactionSpeed="high",
-          transactionSpeed="high",
+        redirectURL=f"{current_app.config['BASE_URL']}/console/account-balance",
-          redirectURL=f"{current_app.config['BASE_URL']}/console/account-balance",
+        notificationURL=f"{current_app.config['BASE_URL']}/payment/btcpay/webhook"
-          notificationURL=f"{current_app.config['BASE_URL']}/payment/btcpay/webhook"
+      ))
-        ))
-      except:
-        current_app.logger.error(f"An error occurred while attempting to reach BTCPay Server: {my_exec_info_message(sys.exc_info())}")
-        flash("An error occurred while attempting to reach BTCPay Server.")
-        return redirect(url_for("console.account_balance"))
 
       current_app.logger.info(f"created btcpay invoice: {invoice}")
 
@@ -298,4 +289,4 @@ def success():
 #   except stripe.error.SignatureVerificationError:
 #     print("/payment/stripe/webhook returned 400: invalid signature")
 #     abort(400, "invalid signature")
-      
+      

capsulflask/publicapi.py

@@ -0,0 +1,40 @@
+import datetime
+
+from flask import Blueprint
+from flask import current_app
+from flask import jsonify
+from flask import request
+from flask import session
+from nanoid import generate
+
+from capsulflask.auth import account_required
+from capsulflask.db import get_model
+
+bp = Blueprint("publicapi", __name__, url_prefix="/api")
+
+@bp.route("/capsul/create", methods=["POST"])
+@account_required
+def capsul_create():
+    email = session["account"]
+    
+    from .console import _create,get_account_balance, get_payments, get_vms
+
+    vm_sizes = get_model().vm_sizes_dict()
+    operating_systems = get_model().operating_systems_dict()
+    public_keys_for_account = get_model().list_ssh_public_keys_for_account(session["account"])
+    account_balance = get_account_balance(get_vms(), get_payments(), datetime.datetime.utcnow())
+    capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(512*1024*1024)
+
+    request.json['ssh_authorized_key_count'] = 1
+
+    id, errors = _create(
+      vm_sizes, 
+      operating_systems,
+      public_keys_for_account,
+      request.json)
+
+    if id is not None:
+        return jsonify(
+            id=id,
+        )
+    return jsonify(errors=errors)

capsulflask/schema_migrations/17_down_api_tokens.py

@@ -0,0 +1,2 @@
+DROP TABLE api_keys;
+UPDATE schemaversion SET version = 16;

capsulflask/schema_migrations/17_up_api_tokens.py

@@ -0,0 +1,9 @@
+CREATE TABLE api_tokens (
+  id                 SERIAL PRIMARY KEY,
+  email              TEXT REFERENCES accounts(email) ON DELETE RESTRICT,
+  name               TEXT NOT NULL,
+  created            TIMESTAMP NOT NULL DEFAULT NOW(),
+  token              TEXT NOT NULL
+);
+
+UPDATE schemaversion SET version = 17;

capsulflask/schema_migrations/19_down_api_tokens.py

@@ -0,0 +1,2 @@
+DROP TABLE api_keys;
+UPDATE schemaversion SET version = 16;

capsulflask/schema_migrations/19_down_os_updates.sql

@@ -1,8 +0,0 @@
-DELETE FROM os_images WHERE id = 'guixsystem130';
-DELETE FROM os_images WHERE id = 'archlinux';
-UPDATE os_images SET deprecated = FALSE WHERE id = 'guixsystem120';
-UPDATE os_images SET deprecated = FALSE WHERE id = 'centos7';
-UPDATE os_images SET deprecated = FALSE WHERE id = 'centos8';
-UPDATE os_images SET description = 'Ubuntu 20.04 LTS (Fossa)' WHERE id = 'ubuntu20';
-
-UPDATE schemaversion SET version = 18;

capsulflask/schema_migrations/19_up_api_tokens.py

@@ -0,0 +1,9 @@
+CREATE TABLE api_tokens (
+  id                 SERIAL PRIMARY KEY,
+  email              TEXT REFERENCES accounts(email) ON DELETE RESTRICT,
+  name               TEXT NOT NULL,
+  created            TIMESTAMP NOT NULL DEFAULT NOW(),
+  token              TEXT NOT NULL
+);
+
+UPDATE schemaversion SET version = 17;

capsulflask/schema_migrations/19_up_os_updates.sql

@@ -1,12 +0,0 @@
-INSERT INTO os_images (id, template_image_file_name, description, deprecated)
-VALUES  ('guixsystem130', 'guixsystem/1.3.0/root.img.qcow2', 'Guix System 1.3.0', FALSE);
-INSERT INTO os_images (id, template_image_file_name, description, deprecated)
-VALUES  ('archlinux', 'archlinux/root.img.qcow2', 'Arch Linux', FALSE);
-
-UPDATE os_images SET deprecated = TRUE WHERE id = 'guixsystem120';
-UPDATE os_images SET deprecated = TRUE WHERE id = 'centos7';
-UPDATE os_images SET deprecated = TRUE WHERE id = 'centos8';
-UPDATE os_images SET description = 'Ubuntu 20.04 (Focal)' WHERE id = 'ubuntu20';
-
-
-UPDATE schemaversion SET version = 19;

capsulflask/schema_migrations/20_down_deferrable_constraints.sql

@@ -1,21 +0,0 @@
-
-ALTER TABLE host_network ALTER CONSTRAINT host_network_host_fkey NOT DEFERRABLE;
-ALTER TABLE host_operation ALTER CONSTRAINT host_operation_host_fkey NOT DEFERRABLE;
-ALTER TABLE host_operation ALTER CONSTRAINT host_operation_operation_fkey NOT DEFERRABLE;
-ALTER TABLE login_tokens ALTER CONSTRAINT login_tokens_email_fkey NOT DEFERRABLE;
-ALTER TABLE operations ALTER CONSTRAINT operations_email_fkey NOT DEFERRABLE;
-ALTER TABLE payment_sessions ALTER CONSTRAINT payment_sessions_email_fkey NOT DEFERRABLE;
-ALTER TABLE payments ALTER CONSTRAINT payments_email_fkey NOT DEFERRABLE;
-ALTER TABLE ssh_public_keys ALTER CONSTRAINT ssh_public_keys_email_fkey NOT DEFERRABLE;
-ALTER TABLE unresolved_btcpay_invoices ALTER CONSTRAINT unresolved_btcpay_invoices_email_fkey NOT DEFERRABLE;
-ALTER TABLE unresolved_btcpay_invoices ALTER CONSTRAINT unresolved_btcpay_invoices_email_payment_id_fkey NOT DEFERRABLE;
-ALTER TABLE vm_ssh_authorized_key ALTER CONSTRAINT vm_ssh_public_key_email_ssh_public_key_name_fkey NOT DEFERRABLE;
-ALTER TABLE vm_ssh_authorized_key ALTER CONSTRAINT vm_ssh_public_key_email_vm_id_fkey NOT DEFERRABLE;
-ALTER TABLE vm_ssh_host_key ALTER CONSTRAINT vm_ssh_host_key_email_vm_id_fkey NOT DEFERRABLE;
-ALTER TABLE vms ALTER CONSTRAINT vms_email_fkey NOT DEFERRABLE;
-ALTER TABLE vms ALTER CONSTRAINT vms_host_network_name_fkey NOT DEFERRABLE;
-ALTER TABLE vms ALTER CONSTRAINT vms_host_fkey NOT DEFERRABLE;
-ALTER TABLE vms ALTER CONSTRAINT vms_os_fkey NOT DEFERRABLE;
-ALTER TABLE vms ALTER CONSTRAINT vms_size_fkey NOT DEFERRABLE;
-
-UPDATE schemaversion SET version = 19;

capsulflask/schema_migrations/20_up_deferrable_constraints.sql

@@ -1,21 +0,0 @@
-
-ALTER TABLE host_network ALTER CONSTRAINT host_network_host_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE host_operation ALTER CONSTRAINT host_operation_host_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE host_operation ALTER CONSTRAINT host_operation_operation_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE login_tokens ALTER CONSTRAINT login_tokens_email_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE operations ALTER CONSTRAINT operations_email_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE payment_sessions ALTER CONSTRAINT payment_sessions_email_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE payments ALTER CONSTRAINT payments_email_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE ssh_public_keys ALTER CONSTRAINT ssh_public_keys_email_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE unresolved_btcpay_invoices ALTER CONSTRAINT unresolved_btcpay_invoices_email_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE unresolved_btcpay_invoices ALTER CONSTRAINT unresolved_btcpay_invoices_email_payment_id_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vm_ssh_authorized_key ALTER CONSTRAINT vm_ssh_public_key_email_ssh_public_key_name_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vm_ssh_authorized_key ALTER CONSTRAINT vm_ssh_public_key_email_vm_id_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vm_ssh_host_key ALTER CONSTRAINT vm_ssh_host_key_email_vm_id_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vms ALTER CONSTRAINT vms_email_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vms ALTER CONSTRAINT vms_host_network_name_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vms ALTER CONSTRAINT vms_host_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vms ALTER CONSTRAINT vms_os_fkey DEFERRABLE INITIALLY DEFERRED;
-ALTER TABLE vms ALTER CONSTRAINT vms_size_fkey DEFERRABLE INITIALLY DEFERRED;
-
-UPDATE schemaversion SET version = 20;

capsulflask/spoke_api.py

@@ -182,10 +182,7 @@ def handle_destroy(operation_id, request_body):
     return abort(400, f"bad request; email is required for destroy")
 
   try:
-    vm = current_app.config['SPOKE_MODEL'].get(request_body['id'], False)
+    current_app.config['SPOKE_MODEL'].destroy(id=request_body['id'], email=request_body['email'])
-    current_app.logger.warning(f"destroy {request_body['id']} was called for {request_body['email']}, however the vm does not exist. returning success. ")
-    if vm is not None:
-      current_app.config['SPOKE_MODEL'].destroy(id=request_body['id'], email=request_body['email'])
   except:
     error_message = my_exec_info_message(sys.exc_info())
     params =          f"email='{request_body['email'] if 'email' in request_body else 'KeyError'}', "

capsulflask/static/style.css

@@ -241,6 +241,7 @@ thead {
   background: #bdc7b812;
 }
 td, th {
+
   padding: 0.1em 1em;
 }
 table.small td, table.small th {
@@ -377,4 +378,4 @@ footer {
   border: 1px solid rgba(255, 223, 155, 0.8);
   box-sizing: border-box;
   position: relative;
-}
+}

capsulflask/templates/account-balance.html

@@ -46,9 +46,7 @@
             <a href="/payment/stripe">Add funds with Credit/Debit (stripe)</a>
             <ul><li>notice: stripe will load nonfree javascript </li></ul>
           </li>
-          {% if btcpay_enabled %}
           <li><a href="/payment/btcpay">Add funds with Bitcoin/Litecoin/Monero (btcpay)</a></li>
-          {% endif %}
     
           <li>Cash: email <a href="mailto:treasurer@cyberia.club">treasurer@cyberia.club</a></li>
         </ul>

capsulflask/templates/base.html

@@ -31,7 +31,7 @@
 
     {% if session["account"] %} 
       <a href="/console">Capsuls</a>
-      <a href="/console/ssh">SSH Public Keys</a>
+      <a href="/console/keys">SSH &amp; API Keys</a>
       <a href="/console/account-balance">Account Balance</a>
     {% endif %}
 

capsulflask/templates/capsul-detail.html

@@ -101,7 +101,7 @@
       </div>
       <div class="row justify-start">
         <label class="align" for="ssh_authorized_keys">SSH Authorized Keys</label>
-        <a id="ssh_authorized_keys" href="/console/ssh">{{ vm['ssh_authorized_keys'] }}</a>
+        <a id="ssh_authorized_keys" href="/console/keys">{{ vm['ssh_authorized_keys'] }}</a>
       </div>
       
     </div>

capsulflask/templates/create-capsul.html

@@ -31,7 +31,7 @@
     <p>(At least one month of funding is required)</p>
   {% elif no_ssh_public_keys %}
     <p>You don't have any ssh public keys yet.</p> 
-    <p>You must <a href="/console/ssh">upload one</a> before you can create a Capsul.</p>
+    <p>You must <a href="/console/keys">upload one</a> before you can create a Capsul.</p>
   {% elif not capacity_avaliable %}
     <p>Host(s) at capacity. No capsuls can be created at this time. sorry. </p> 
   {% else %}

capsulflask/templates/faq.html

@@ -23,7 +23,6 @@
      How do I log in?
      <p>ssh to the ip provided to you using the cyberian user.</p>
      <pre class='code'>$ ssh cyberian@1.2.3.4</pre>
-     <p>For more information, see <a href="/about-ssh">Understanding the Secure Shell Protocol (SSH)</a>.</p>
    </li>
    <li>
      How do I change to the root user?

capsulflask/templates/keys.html

@@ -1,17 +1,18 @@
 {% extends 'base.html' %}
 
-{% block title %}SSH Public Keys{% endblock %}
+{% block title %}SSH & API Keys{% endblock %}
 
 {% block content %}
 <div class="row third-margin">
   <h1>SSH PUBLIC KEYS</h1>
 </div>
 <div class="row third-margin"><div>
-  {% if has_ssh_public_keys %} <hr/> {% endif %}
+    {% if ssh_public_keys|length > 0 %} <hr/> {% endif %}
   
   {% for ssh_public_key in ssh_public_keys %}
     <form method="post">
       <input type="hidden" name="method" value="DELETE"></input> 
+      <input type="hidden" name="action" value="delete_ssh_key"></input>
       <input type="hidden" name="name" value="{{ ssh_public_key['name'] }}"></input> 
       <input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
       <div class="row">
@@ -22,13 +23,14 @@
     </form>
   {% endfor %}
 
-  {% if has_ssh_public_keys %} <hr/> {% endif %}
+  {% if ssh_public_keys|length > 0 %} <hr/> {% endif %}
 
   <div class="third-margin">
     <h1>UPLOAD A NEW SSH PUBLIC KEY</h1>
   </div>
   <form method="post">
     <input type="hidden" name="method" value="POST"></input>
+    <input type="hidden" name="action" value="upload_ssh_key"></input>
     <input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
     <div class="row justify-start">
       <label class="align" for="content">File Contents</label>
@@ -54,6 +56,51 @@
     </div>
   </form>
 </div></div>
+<hr/>
+<div class="row third-margin">
+  <h1>API KEYS</h1>
+</div>
+<div class="row third-margin"><div>
+  {% if generated_api_token %}
+    <hr/>
+    Generated key:
+    <span class="code">{{ generated_api_token }}</span>
+  {% endif %}
+  {% if api_tokens|length >0 %} <hr/>{% endif %}
+  {% for api_token in api_tokens %}
+    <form method="post">
+      <input type="hidden" name="method" value="DELETE"></input> 
+      <input type="hidden" name="action" value="delete_api_token"></input>
+      <input type="hidden" name="id" value="{{ api_token['id'] }}"></input> 
+      <input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
+      <div class="row">
+        <span class="code">{{ api_token['name'] }}</span>
+        created {{ api_token['created'].strftime("%b %d %Y") }}
+        <input type="submit" value="Delete">
+      </div>
+    </form>
+  {% endfor %}
+  {% if api_tokens|length >0 %} <hr/>{% endif %}
+
+  <div class="third-margin">
+    <h1>GENERATE A NEW API KEY</h1>
+  </div>
+  <form method="post">
+    <input type="hidden" name="method" value="POST"></input>
+    <input type="hidden" name="action" value="generate_api_token"></input>
+    <input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
+    <div class="smalltext">
+      <p>Generate a new API key, to integrate with other systems.</p>
+    </div>
+    <div class="row justify-start">
+      <label class="align" for="name">Key Name</label>
+      <input type="text" id="name" name="name"></input> (defaults to creation time)
+    </div>
+    <div class="row justify-end">
+      <input type="submit" value="Generate">
+    </div>
+  </form>
+</div></div>
 {% endblock %}
 
 {% block pagesource %}/templates/ssh-public-keys.html{% endblock %}

capsulflask/templates/pricing.html

@@ -6,37 +6,21 @@
   <div class="row third-margin">
     <h1>CAPSUL TYPES & PRICING</h1>
   </div>
-  <div class="row half-margin">
-    <table>
-      <thead>
-        <tr>
-          <th>type</th>
-          <th>monthly*</th>
-          <th>cpus</th>
-          <th>mem</th>
-          <th>ssd</th>
-          <th>net</th>
-        </tr>
-      </thead>
-      <tbody>
-      {% for vm_size_key, vm_size in vm_sizes.items() %}
-        <tr>
-          <td>{{ vm_size_key }}</td>
-          <td>${{ vm_size['dollars_per_month'] }}</td>
-          <td>{{ vm_size['vcpus'] }}</td>
-          <td>{{ vm_size['memory_mb'] }}</td>
-          <td>25G</td>
-          <td>{{ vm_size['bandwidth_gb_per_month'] }}</td>
-        </tr>
-      {% endfor %}
-      </tbody>
-    </table>
-  </div>
   <div class="row half-margin">
     <pre>
+    type     monthly*  cpus  mem     ssd   net*
+    -----    -------   ----  ---     ---   ---
+    f1-xs    $5.00     1     512M    25G   .5TB
+    f1-s     $7.50     1     1024M   25G   1TB
+    f1-m     $12.50    1     2048M   25G   2TB
+    f1-l     $20.00    2     3072M   25G   3TB
+    f1-x     $27.50    3     4096M   25G   4TB
+    f1-xx    $50.00    4     8192M   25G   5TB
+
     * net is calculated as a per-month average
     * vms are billed for a minimum of 24 hours upon creation
     * all VMs come standard with one public IPv4 address
+  
     
     SUPPORTED OPERATING SYSTEMS:
 

capsulflask/templates/support.html

@@ -7,11 +7,18 @@
     <h1>SUPPORT</h1>
   </div>
   <div class="row half-margin">
-    <a href="mailto:support@cyberia.club?subject=capsul%20support%20request">support@cyberia.club</a>
+    <a href="mailto:support@cyberia.club?subject=Please%20help!">support@cyberia.club</a> 
   </div>
 {% endblock %}
 
 {% block subcontent %}
+<p>
+  Note: We maintain a searchable archive of all support emails at 
+  <a href="https://lists.cyberia.club/~cyberia/support">https://lists.cyberia.club/~cyberia/support</a>
+</p>
+<p>
+  If you do not want your mail to appear in a public archive, email <a href="mailto:capsul@cyberia.club?subject=Please%20help!">capsul@cyberia.club</a> instead.
+</p>
 <p>
   Please describe your problem or feature request, and we will do our best to get back to you promptly. Thank you very much.
 </p>

docker-compose.yml

@@ -0,0 +1,29 @@
+---
+version: "3.8"
+
+services:
+  app:
+    image: 3wordchant/capsul-flask:latest
+    build: .
+    volumes:
+      - "./:/app/code"
+    depends_on:
+      - db
+    ports:
+      - "5000:5000"
+    environment:
+      - "POSTGRES_CONNECTION_PARAMETERS=host=db port=5432 user=capsul password=capsul dbname=capsul"
+    # The image uses gunicorn by default, let's override it with Flask's
+    # built-in development server
+    command: ["flask", "run", "-h", "0.0.0.0", "-p", "5000"]
+  db:
+    image: "postgres:9.6.5"
+    volumes:
+      - "postgres:/var/lib/postgresql/data"
+    environment:
+      POSTGRES_USER: capsul
+      POSTGRES_PASSWORD: capsul
+      POSTGRES_DB: capsul
+
+volumes:
+  postgres:

docs/architecture.md

@@ -1,30 +0,0 @@
-# hub-and-spoke architecture
-
-The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"s are responsible for creating/managing virtual machines. One instance of the capsul-flask application can run in hub mode and spoke mode at the same time. 
-
-The Hub and the Spoke must be configured to communicate securely with each-other over HTTPS. They both have to be able to dial each-other directly.  The URLs / auth tokens they use are configured both in the config file (`HUB_URL`, `SPOKE_HOST_ID`, `SPOKE_HOST_TOKEN` and `HUB_TOKEN`) and in the database (the `id`, `https_url`, and `token` columns in the `hosts` table).
-
-![](images/hub-and-spoke1.png)
-
-This diagram was created with https://app.diagrams.net/.
-To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
-
-right now I have 2 types of operations, immediate mode and async.
-
-both types of operations do assignment synchronously. so if the system cant assign the operation to one or more hosts (spokes),
-or whatever the operation requires, then it will fail. 
-
-some operations tolerate partial failures, like, `capacity_avaliable` will succeed if at least one spoke succeeds.
-for immediate mode requests (like `list`, `capacity_avaliable`, `destroy`), assignment and completion of the operation are the same thing.
-
-for async ones, they can be assigned without knowing whether or not they succeeded (`create`).
-
-![](images/hub-and-spoke2.png)
-
-This diagram was created with https://app.diagrams.net/.
-To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
-
-if you issue a create, and it technically could go to any number of hosts, but only one host responds, it will succeed 
-but if you issue a create and somehow 2 hosts both think they own that task, it will fail and throw a big error. cuz it expects exactly 1 to own the create task
-
-currently its not set up to do any polling.  its not really like a queue at all. It's all immediate for the most part 

docs/btcpay.md

@@ -1,68 +0,0 @@
-## <a name="BTCPAY_PRIVATE_KEY"></a>Setting up the BTCPAY_PRIVATE_KEY 
-
-Generate a private key and the accompanying bitpay SIN for the btcpay API client. 
-
-I used this code as an example: https://github.com/bitpay/bitpay-python/blob/master/bitpay/key_utils.py#L6
-
-```
-$ pipenv run python ./readme/generate_btcpay_keys.py
-```
-
-It should output something looking like this:
-
-```
------BEGIN EC PRIVATE KEY-----
-EXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK
-oUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq
-NQ+cpBvPDbyrDk9+Uf/sEaRCma094g==
------END EC PRIVATE KEY-----
-
-
-EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE
-```
-
-In order to register the key with the btcpay server, you have to first generate a pairing token using the btcpay server interface.
-This requires your btcpay server account to have access to the capsul store. Ask Cass about this.
-
-Navigate to `Manage store: Access Tokens` at: `https://btcpay.cyberia.club/stores/<store-id>/Tokens`
-
-![](images/btcpay_sin_pairing.jpg)
-
-
-Finally, send an http request to the btcpay server to complete the pairing:
-
-```
-curl -H "Content-Type: application/json" https://btcpay.cyberia.club/tokens -d "{'id': 'EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE', 'pairingCode': 'XXXXXXX'}"
-```
-
-It should respond with a token: 
-
-```
-{"data":[{"policies":[],"pairingCode":"XXXXXXX","pairingExpiration":1589473817597,"dateCreated":1589472917597,"facade":"merchant","token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","label":"capsulflask"}]}
-```
-
-And you should see the token in the btcpay server UI:
-
-![](images/paired.jpg)
-
-Now simply set your `BTCPAY_PRIVATE_KEY` variable in `.env`
-
-NOTE: make sure to use single quotes and replace the new lines with \n.
-
-```
-BTCPAY_PRIVATE_KEY='-----BEGIN EC PRIVATE KEY-----\nEXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK\noUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq\nNQ+cpBvPDbyrDk9+Uf/sEaRCma094g==\n-----END EC PRIVATE KEY-----'
-```
-
------
-
-## <a name="testing"></a>testing cryptocurrency payments
-
-I used litecoin to test cryptocurrency payments, because its the simplest & lowest fee cryptocurrency that BTCPay server supports. You can download the easy-to-use litecoin SPV wallet `electrum-ltc` from [github.com/pooler/electrum-ltc](https://github.com/pooler/electrum-ltc) or [electrum-ltc.org](https://electrum-ltc.org/), set up a wallet, and then either purchase some litecoin from an exchange, or [ask Forest for some litecoin](https://sequentialread.com/capsul-rollin-onwards-with-a-web-application/#sqr-comment-container) to use for testing.
-
-
-## <a name="0_conf_diagram"></a>sequence diagram explaining how BTC payment process works (how we accept 0-confirmation transactions 😀)
-
-![btcpayment_process](images/btcpayment_process.png)
-
-This diagram was created with https://app.diagrams.net/.
-To edit it, download the <a download href="readme/btcpayment_process.drawio">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.

docs/configuration.md

@@ -1,89 +0,0 @@
-# Configuring Capsul-Flask
-
-Create a `.env` file to set up the application configuration:
-
-```
-nano .env
-```
-
-You can enter any environment variables referenced in [`__init__.py`](../capsulflask/__init__.py) to this file.
-
-For example you may enter your SMTP credentials like this:
-```
-MAIL_USERNAME=forest@nullhex.com
-MAIL_DEFAULT_SENDER=forest@nullhex.com
-MAIL_PASSWORD=**************
-```
-
-## <a name="example"></a>Example configuration from capsul.org (production):
-
-```
-#LOG_LEVEL=DEBUG
-
-BASE_URL="https://capsul.org"
-
-# hub url is used by the SPOKE_MODE to contact the hub. Since this server is the hub,
-# this is fine. In fact it runs into problems (routing related?) when I set it to capsul.org.
-# similarly the baikal "spoke" (set up in the hosts table in the db) has "http://localhost:5000" as the https_url
-HUB_URL="http://localhost:5000"
-
-HUB_MODE_ENABLED="t"
-SPOKE_MODE_ENABLED="t"
-HUB_MODEL="capsul-flask"
-SPOKE_MODEL="shell-scripts"
-SPOKE_HOST_ID="baikal"
-SPOKE_HOST_TOKEN="<redacted>"
-HUB_TOKEN="<redacted>"
-
-# smtp.. see https://flask-mail.readthedocs.io/en/latest/#configuring-flask-mail
-MAIL_SERVER="smtp.nullhex.com"
-
-# MAIL_USE_SSL means SMTP with STARTTLS
-MAIL_USE_SSL=true
-
-# MAIL_USE_TLS means SMTP wrapped in TLS
-MAIL_USE_TLS=false
-
-MAIL_PORT="465"
-MAIL_USERNAME="capsul@nullhex.com"
-MAIL_PASSWORD="<redacted>"
-MAIL_DEFAULT_SENDER="capsul@nullhex.com"
-
-# stripe
-STRIPE_SECRET_KEY="sk_live_<redacted>"
-STRIPE_PUBLISHABLE_KEY="pk_live_tGDHY7kBwqC71b4F0N7LZdGl00GZOw0iNJ"
-
-# internal
-SECRET_KEY="<redacted>"
-POSTGRES_CONNECTION_PARAMETERS="sslmode=verify-full sslrootcert=letsencrypt-root-ca.crt host=postgres.cyberia.club port=5432 ...<redacted>"
-
-
-# btcpay server
-BTCPAY_URL="https://beeteeceepae2.cyberia.club"
-BTCPAY_PRIVATE_KEY='-----BEGIN EC PRIVATE KEY-----\n<redacted>\n-----END EC PRIVATE KEY-----'
-```
-
-## <a name="config_that_lives_in_db"></a>Configuration-type-stuff that lives in the database 
-
-  - `hosts` table:
-    - `id` (corresponds to `SPOKE_HOST_ID` in the config)
-    - `https_url`
-    - `token` (corresponds to `SPOKE_HOST_TOKEN` in the config)
-  - `os_images` table:
-    - `id` 
-    - `template_image_file_name`
-    - `description`
-    - `deprecated`
-  - `vm_sizes` table:
-    - `id` 
-    - `dollars_per_month`
-    - `memory_mb`
-    - `vcpus`
-    - `bandwidth_gb_per_month`
-    
-## <a name="docker_secrets"></a>Loading variables from files (docker secrets)
-
-To support [Docker Secrets](https://docs.docker.com/engine/swarm/secrets/), you can also load secret values from files – for example, to load `MAIL_PASSWORD` from `/run/secrets/mail_password`, set
-```sh
-MAIL_PASSWORD_FILE=/run/secrets/mail_password
-```

docs/database.md

@@ -1,67 +0,0 @@
-# capsul-flask's relationship to its Database Server
-
-Capsul has a ["hub and spoke" architecture](./architecture.md). The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"s are responsible for creating/managing virtual machines. One instance of the capsul-flask application can run in both hub mode and spoke mode at the same time, however there must only be one instance of the app running in "Hub" mode at any given time.
-
-The Postgres connections parameters are [configurable](./configuration.md).
-
-## <a name="schema_management"></a>Database schema management (schema versions)
-
-capsul-flask has a concept of a schema version. When the application starts, it will query the database for a table named `schemaversion` that has one row and one column (`version`). If the `version` it finds is not equal to the `desiredSchemaVersion` variable set in `db.py`, it will run  migration scripts from the `schema_migrations` folder one by one until the `schemaversion` table shows the correct version.
-
-For example, the script named `02_up_xyz.sql` should contain code that migrates the database from schema version 1 to schema version 2. Likewise, the script `02_down_xyz.sql` should contain code that migrates from schema version 2 back to schema version 1.
-
-**IMPORTANT: if you need to make changes to the schema, make a NEW schema version. DO NOT EDIT the existing schema versions.**
-
-In general, for safety, schema version upgrades should not delete data. Schema version downgrades will simply throw an error and exit for now. 
-
-## <a name="manual_queries"></a>Running manual database queries
-
-You can manually mess around with the database like this:
-
-```
-pipenv run flask cli sql -f test.sql
-```
-
-```
-pipenv run flask cli sql -c 'SELECT * FROM vms'
-```
-
-This one selects the vms table with the column name header:
-
-```
-pipenv run flask cli sql -c "SELECT string_agg(column_name::text, ', ') from information_schema.columns WHERE table_name='vms'; SELECT * from vms"
-```
-
-How to modify a payment manually, like if you get a chargeback or to fix customer payment issues:
-
-```
-$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
-1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, FALSE
-
-$ pipenv run flask cli sql -c "UPDATE payments SET invalidated = True WHERE id = 1"
-1 rows affected.
-
-$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
-1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, TRUE
-```
-
-## how to view the logs on the database server (legion.cyberia.club)
-
-`sudo -u postgres pg_dump capsul-flask | gzip -9 > capsul-backup-2021-02-15.gz`
-
-
-## changing the email address on an account
-
-```
-UPDATE accounts SET lower_case_email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE accounts SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE login_tokens SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE operations SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE payment_sessions SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE payments SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE ssh_public_keys SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE unresolved_btcpay_invoices SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE vm_ssh_authorized_key SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE vm_ssh_host_key SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-UPDATE vms SET email = 'new@email.address' WHERE email = 'old@email.address' ;
-```

docs/deployment.md

@@ -1,87 +0,0 @@
-# Deploying Capsul on a server
-
-Capsul has a ["hub and spoke" architecture](./architecture.md). The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"s are responsible for creating/managing virtual machines. One instance of the capsul-flask application can run in both hub mode and spoke mode at the same time, however there must only be one instance of the app running in "Hub" mode at any given time.
-
-## <a name="spoke_mode_prerequisites"></a>Installing prerequisites for Spoke Mode
-
-On your spoke (see [Architecture](./architecture.md) You'll need `libvirtd`, `dnsmasq`, and `qemu-kvm`, plus a `/tank` diectory with some operating system images in it:
-
-```
-sudo apt install libvirt-daemon-system virtinst git dnsmasq qemu qemu-kvm
-sudo mkdir -p /var/www /tank/{vm,img,config}
-sudo mkdir -p /tank/img/debian/10
-cd !$
-sudo wget https://cloud.debian.org/images/cloud/buster/20201023-432/debian-10-genericcloud-amd64-20201023-432.qcow2 -O root.img.qcow2
-```
-
-TODO: network set-up
-TODO: cyberia-cloudinit.yml
-
-## Deploying capsul-flask
-
-### <a name="deploy_manually"></a>Manually
-
-Follow the [local set-up instructions](./local-set-up.md) on your server. 
-
-Make sure to set `BASE_URL` correctly, generate your own secret tokens, and
-configure your own daemon management for the capsul-flask server (e.g. writing
-init scripts, or SystemD unit files).
-
-Use the suggested `gunicorn` command (with appropriately-set address and port),
-instead of `flask run`, to launch the server.
-
-For example, here is the SystemD service unit file we use in production for `capsul.org`:
-
-```
-[Unit]
-Description=capsul-flask virtual machines as a service
-After=network.target
-
-[Service]
-ExecStart=/usr/local/bin/pipenv run gunicorn --bind 127.0.0.1:5000 -k gevent --worker-connections 1000 app:app
-Restart=on-failure
-WorkingDirectory=/opt/capsul-flask
-
-[Install]
-WantedBy=multi-user.target
-
-```
-
-TODO: cron runner is required to run maintenance tasks for now, but in the future we want to build this into the python based task scheduler. 
-
-### <a name="coop_cloud_docker"></a> Using Co-op Cloud's vanilla Docker Swarm configuration
-
-Download the Co-op Cloud swarm `compose.yml`:
-
-```sh
-wget https://git.autonomic.zone/coop-cloud/capsul/src/branch/main/compose.yml
-```
-
-Optionally, download add-on compose files for Stripe, BTCPay, and Spoke Mode:
-
-```sh
-wget https://git.autonomic.zone/coop-cloud/capsul/src/branch/main/compose.{stripe,btcpay,spoke}.yml
-```
-
-Then, create a `.env` file and configure appropriately -- you probably want to
-define most settings in [the Co-op Cloud `.envrc.sample`
-file](https://git.autonomic.zone/coop-cloud/capsul/src/branch/main/.envrc.sample).
-
-Load the environment variables (using Python `direnv`, or a manual `set -a && source .env && set +a`), insert any necessary secrets, then run the deployment:
-
-```sh
-docker stack deploy -c compose.yml -c compose.stripe.yml your_capsul
-```
-
-(where you'd add an extra `-c compose.btcpay.yml` for each optional compose file
-you want, and set `your_capsul` to the "stack name" you want).
-
-TODO: cron runner
-
-### <a name="coop_cloud_abra"></a> Using Co-op Cloud's `abra` deployment tool
-
-Follow [the guide in the README for the Co-op Cloud capsul package](https://git.autonomic.zone/coop-cloud/capsul/).
-
-### Using docker-compose
-
-TODO

docs/local-set-up.md

@@ -1,68 +0,0 @@
-# How to run Capsul locally
-
-## <a name="manually"></a>Manually
-
-Ensure you have the pre-requisites for the psycopg2 Postgres database adapter package:
-
-```sh
-sudo apt install python3-dev libpq-dev
-pg_config --version
-```
-
-Ensure you have the wonderful `pipenv` python package management and virtual environment cli:
-
-```sh
-sudo apt install pipenv
-```
-
-Create python virtual environment and install packages:
-
-```sh
-pipenv install
-```
-
-Run an instance of Postgres (I used docker for this, you can use whatever you want, point is its listening on `localhost:5432`):
-
-```sh
-docker run --rm -it -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres
-```
-
-Run the app 
-
-```sh
-pipenv run flask run
-```
-
-or, using Gunicorn:
-
-```sh
-pipenv run gunicorn --bind 127.0.0.1:5000 -k gevent --worker-connections 1000 app:app
-```
-
-Note that by default when running locally, the `SPOKE_MODEL` is set to `mock`, meaning that it won't actually try to spawn vms.
-
-## Crediting your account
-
-Once you log in for the first time, you will want to give yourself some free capsulbux so you can create fake capsuls for testing. 
-
-```sh
-pipenv run flask cli sql -c "INSERT INTO payments (email, dollars) VALUES ('<your email address here>', 20.00)"
-```
-
-## Running scheduled tasks:
-
-```sh
-pipenv run flask cli cron-task
-```
-
-## <a name="docker_compose"></a>Run locally with docker-compose
-
-If you have Docker and Docker-Compose installed, you can use the
-`3wordchant/capsul-flask` Docker image to launch capsul-flask, and a Postgres
-database server, for you:
-
-```sh
-docker-compose up
-```
-
-`capsul-flask` will read settings from your `.env` file as usual; you can set any of the options mentioned in the [configuration documentation](./configuration.md).