End‐to‐end Tests with Playwright

CI

Smoke tests

Smoke tests run via a GitHub action. The action triggers on pull_request with the additional condition of the label Seen-on-prod being present (this label is added by prout once a deploy has finished and the changes have been confirmed in production).

You can view the action here.

Cron tests

Cron tests also run via a GitHub action. These run on a schedule (hourly Monday - Friday 9am - 5pm). We test PayPal checkout flows in the cron tests as we don't want to run any more frequently as we risk being rate limited by PayPal.

You can view the action here.

Running locally

Various scripts are defined in the package.json file for running the e2e tests locally against different environments (prod, code, local). Typically these can be run in a headless or UI mode (UI tests have ui in the script name).

Type checking the tests

$ yarn tsc

Possible future work

It would be nice to use Playwright for integration tests as well as E2E tests. Possibly we could split out the test directory into e2e and integration directories and modify the testDir config property in the Playwright config files to reflect whether you are running E2E or integration tests.

Rationale

Playwright tests written in Typescript have replaced the old Scala Selenium tests. The rational for this decision was primarily to allow us to write tests in the same language that the site is written in.