Running the Dev Containers

WIP TODO list:

• Add launch.json files to 1Password for each project
• Flesh out the troubleshooting section
• Add a section on running and debugging the tests. (NOTIFY_ENVIRONMENT=[test|development])
• Adapt the Dump the staging DB guide into a guide on backing up and restoring your DB when the nuclear option of destroying your containers and starting from scratch is necessary.

Prerequisites

This wiki assumes you have completed the following prerequisite tasks.

• Setup 1Password and have access to the Notify folders
• Have access to your own AWS accounts for both the Staging and Production environments
• Docker Desktop is installed on your machine
• Visual Studio Code is installed on your machine
• Cloned the API , Admin, and notification-document-download-api repos

How to work with dev containers in VSCode

1. If it is not already installed, add the Dev Containers extension to VSCode
2. Open the Command Palette with ⌘ + shift + p
3. Type Dev Containers and select the Open folder in container option
4. Navigate to and open the Admin project
5. The Dev Container should begin building. You should notice output in the terminal.

What sorcery is this?

If you're familiar with Docker and Dev Containers already, feel free to skip ahead to the configuration section.

Since you're waiting for the dev container and project to build, it's a good time to learn some dev container basics.

In each project you'll notice a .devcontainer folder structured similarly to the one pictured below.

Let's briefly go over how these files relate to one another.

Dockerfile

A set of instructions used by Docker to build an image that serves as the basis for the our Dev Container environment. OS and root level operations are defined here.

Install OS level dependencies and packages (git, npm, nodejs, vim etc.), perform root-level operations (chown, chmod, set env vars etc)

Scripts

These shell scripts execute within the the container after it is built. They are geared toward user space actions and general project setup.

Set up terminal aliases and autocompletions, install python/JS dependencies, automatically kick off a project build

devcontainer.json

This file brings everything together. It instructs VSCode how to handle creation of, and access to a dev container as well as automatically configuring VSCode extensions.

Which Dockerfile is used to build the container, will we use bash or zsh, what VSCode extensions are should be installed, which scripts should run after the container is built

Configure the projects

.env configs

Each project relies on a .env Flask configuration file that should be placed in the root of the project. You can find one for each respective project in 1Password: Canadian Digital Service > CDS Platform - Notify Local (NEW).

1. Create a file named .env at the root of each project
2. Copy the contents from 1Password into each respective .env file.

launch.json configs

The last piece needed is a launch configuration to instruct VSCode how to launch the apps. The configs can be found in 1Password.

1. Add the following directory structure in each project: <project_root>/.vscode
2. In the .vscode directory, create a file called launch.json
3. Copy the contents from 1Password into each respective launch.json file

Launching the apps

TODO: Mention how you can also start the apps / celery via makefile commands

With the configurations in place and the dev containers built, running the application is as simple as:

1. Navigate to Run and Debug
2. Select Python: Flask from the drop down
3. Press the green arrow

Admin

1. Run the Admin project.
2. Once the app has started, you should see configuration output in your terminal.
3. Navigate to http://localhost:6012, you should see the following page:

You're seeing this message because we also need to run the API app to provide Admin with a means to interact with the data.

API

Start API the same way you started Admin, running both the Python: Flask and Python: Celery apps, then refresh the page in your browser. If you see the following, congrats! You've successfully setup GCNotify locally!

Explore the app:

1. Create an account
2. log in
3. Create a service and template
4. Perform a send

If all goes well, you're ready to start developing. If you ran into issues check out the troubleshooting section or hit up one of your fellow devs on Slack and we'll help you out.

Document Download API

In general, Admin and API suffice for most day to day work. However if you need to send notifications with attached files, the Document Download API app needs to be running.

Verify it's working

Once you built and ran the DD-API container and app you'll need to change a few service settings, add a new template, and create an API key to send an email notification with an attachment.

1. Log in and navigate to your service settings
2. Under the Emails settings enable the Send files by email setting
3. Navigate to your Dashboard and create a new email template

Make sure to include a variable in the template body, ((var)) will do.

4. Navigate to API Integration > API keys and create a Live API key

Take note of the API key as it is needed in the next steps

5. Use a tool like Postman to send the following request to: http://localhost:6011/v2/notifications/email Make sure you set the following headers:

• Authorization : <API KEY FROM STEP 4>
• Content-Type : application/json

{
  "email_address": "<YOUR EMAIL ADDRESS>",
  "template_id": "<TEMPLATE ID FROM STEP 3>",
  "personalisation": {
    "var": "var",
    "application_file": {
      "file": "aGkgdGhlcmU=",
      "filename": "test_file.txt",
      "sending_method": "attach"
    }
  }
}

6. Check your email and make sure you can open the attached or linked file.

Alternatively you can use a curl command:

curl -i -X POST \
   -H "Authorization:ApiKey-v1 <API KEY FROM STEP 4>" \
   -H "Content-Type:application/json" \
   -d \
'{
  "email_address": "<YOUR EMAIL ADDRESS>",
  "template_id": "<TEMPLATE ID FROM STEP 3>",
  "personalisation": {
    "var": "var",
    "application_file": {
      "file": "aGkgdGhlcmU=",
      "filename": "test_file.txt",
      "sending_method": "attach"
    }
  }
}' \
 'http://localhost:6011/v2/notifications/email'

Troubleshooting & FAQs

I see the error: "Sorry, we’re experiencing technical difficulties" when I navigate to the home page

• Have you built and ran the the API app?
• In the admin project, check that theAPI_HOST_NAME in your .env file is pointing to the correct address: http://host.docker.internal:6011.

In simple terms, host.docker.internal is like the Docker version of localhost, except with a little extra DNS magic. It is scoped to a virtual network in Docker In this case the bridge network.

If the above is in order, it's time to inspect the bridge network in Docker to ensure that the Admin and API containers are both present in the same network. Open a terminal and:

1. Identify the ADMIN and API containers: docker container ls
2. Check the bridge network: docker network inspect bridge
3. Add any missing containers to the bridge network: docker network connect bridge <container-id>

I get missing module errors when running an app

We use a tool called Poetry to manage dependencies and create Python virtual environments for the apps run in. When the dev container is built, the app's dependencies are installed in this venv. Occasionally VSCode doesn't pick up this venv and you need to manually point VSCode to it.

1. Open a Python source file in VSCode
2. Check the bottom status bar and ensure the selected Python interpreter indicates Poetry:

If this isn't present, right click the status bar and check the Selected Python Interpreter option.

3. To change the selected Interpreter click current interpreter on the status bar and select the Poetry environment in the Command Palette window that opens.

If the Poetry environment isn't listed

1. Select Enter interpreter path...
2. Navigate to /home/vscode/.cache/pypoetry/virtualenvs/<project-name-some_id-py3.10>/bin and select python3.10

Okay I fixed the virtual environment but still no dice, what gives?

Try reinstalling the project's dependencies and running the app again.

# Make sure the lock file is not out of date
poetry lock --check
# Update the lock file if inconsistencies are found
poetry lock --no-update
# Install the project dependencies
poetry install

My API unit tests suddenly stopped working

We stand up and tear down a separate DB when running unit tests in API. Sometimes this DB can become invalid during this process, leading to errors ranging from missing tables, to constraint violations.

Use pgAdmin or your favourite DB explorer to manually remove test DB instances.

test_notitification_api
test_notitification_api_gw0
test_notitification_api_gw1
...

Run the tests again to regenerate the test DB from scratch. This usually corrects most DB <-> unit test related issues.

I can run Admin but styling is wrong / translations are missing

Make sure you've built the static assets: npm run build

Make sure that Tailwind CSS styles are compiled: npm run tailwind

Addressing seemingly irreconcilable issues with dev containers

Occasionally you might run into problems with your dev containers that seem unsolvable. Everything looks correct but it doesn't make sense that the containers are malfunctioning. As a last resort you can prune existing Docker components and rebuild your containers from scratch.

IMPORTANT:

This will result in your existing database being destroyed. TODO: Refer to this guide to back up your DB before you continue if you want to avoid setting up your accounts / data locally again

Stop currently running containers

docker stop $(docker ps -a -q)

Prune Docker images, containers, networks and volumes

--volumes is optional but recommended unless you leverage volumes for specific use cases.

docker system prune --volumes

Re-open / rebuild the project's container(s) in VSCode, refer to the "How to work with dev containers in VSCode" if you are unsure how to do this.