Skip to content

Run example apps headless tests in CI #2854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 50 commits into
base: main
Choose a base branch
from

Conversation

FranjoMindek
Copy link
Contributor

@FranjoMindek FranjoMindek commented Jun 26, 2025

Fixes #2825

Currently, the workflow is:

  1. We run the job which caches playwright browsers
  2. We run job for each of the examples apps in parallel, each one restores playwright browsers, installs wasp-cli, and runs it's headless tests
graph TD;
P[playwright] --> A1[wasp-cli]
A1 -->A2[waspello]
P --> B1[wasp-cli]
B1 --> B2[waspleau]
P --> C1[wasp-cli]
C1 --> C2[websockets-realtime-voting]
P --> D1[wasp-cli]
D1 --> D2[TodoApp]
P --> E1[wasp-cli]
E1 --> E2[TodoAppTs]
Loading

I've wanted to do:

graph TD;
P[playwright] --> Wait{AND}
W[wasp-cli] --> Wait
Wait --> A[waspello]
Wait --> B[waspleau]
Wait --> C[websockets-realtime-voting]
Wait --> D[TodoApp]
Wait --> E[TodoAppTs]
Loading

While building the wasp-cli and caching the artifact is easy, the problem is the installation.
I'm not sure what's the easiest way to do it.

@FranjoMindek FranjoMindek requested a review from Copilot June 27, 2025 10:18
Copilot

This comment was marked as outdated.

@infomiho
Copy link
Contributor

@FranjoMindek just a heads up, there was this PR that tried to tackle the same thing - but I never finished it since running all the tests took too long: #2791

…i-test

# Conflicts:
#	examples/tutorials/TodoApp/package.json
#	examples/tutorials/TodoAppTs/package.json
#	examples/waspello/package-lock.json
#	examples/waspello/package.json
#	examples/waspleau/package.json
#	examples/websockets-realtime-voting/package.json

This comment was marked as off-topic.

DATABASE_PROVIDER=$(wasp-cli info \
| grep "Database system" \
| sed 's/.*: //' \
| sed -e 's/\x1b\[[0-9;]*m//g' \
Copy link
Contributor Author

@FranjoMindek FranjoMindek Jul 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any way to add comments here?
Any variant I've tried, it got inferred as part of "bash string" and errored.

At first I read the schema.prisma file but then remembered wasp info.
Seems more robust, the only problem is that we use ANSI colors in the output.

This resulted is some debugging since 'sqlite' == 'sqlite' was false because one sqlite was colored, and this can't be seen in the logs.

I want to leave a comment to explain sed -e 's/\x1b\[[0-9;]*m//g' \ which removes ANSI coloring chars.
But can't find out how.

- main
- release
pull_request:
# paths:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Temp, or workflow won't trigger here, because the initial PR commits didn't include changes to examples/**

- name: Set up Wasp App runner
run: npm i -g @wasp.sh/wasp-app-runner

- name: Install Dependencies for ${{ matrix.example }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm following waspc/examples/todoApp convention, so if we detect .env.server.headless we use that env for running the headless tests.

This is because some apps require env vars to run properly.

Comment on lines +3 to +5
const HEADLESS_TEST_MODE = process.env.HEADLESS_TEST_MODE ?? "dev";
const WASP_CLI_CMD = process.env.WASP_CLI_CMD ?? "wasp-cli";

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm doubling down on "dev" here, so that the default is that we run headless tests in dev mode with "dev" wasp (wasp-cli).

I think this is more suitable for us.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automatically test example apps before release
2 participants