Skip to content

Commit 456c91d

Browse files
brichetbrichet
authored
Add UI tests (#97)
* Add basic test without provider * lint, clean staged files and fix installation * Add test with Ollama * Wait for the models to be ready before testing ollama * Add a basic completion test
1 parent 7ebc54a commit 456c91d

File tree

13 files changed

+4805
-1
lines changed

13 files changed

+4805
-1
lines changed

‎.github/workflows/tests.yml

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: main
6+
pull_request:
7+
branches: '*'
8+
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
ui-tests:
15+
name: ui-tests
16+
runs-on: ubuntu-latest
17+
18+
env:
19+
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/pw-browsers
20+
21+
steps:
22+
- name: Checkout
23+
uses: actions/checkout@v4
24+
25+
- name: Base Setup
26+
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
27+
28+
- name: Install the extension
29+
run: |
30+
set -eux
31+
pip install "jupyterlab>=4.4.0"
32+
pip install -e .
33+
34+
- name: Install Ollama with little model
35+
uses: pydantic/ollama-action@v3
36+
with:
37+
model: qwen2:0.5b
38+
39+
- name: Install tests dependencies
40+
working-directory: ui-tests
41+
env:
42+
YARN_ENABLE_IMMUTABLE_INSTALLS: 0
43+
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
44+
run: jlpm install
45+
46+
- name: Set up browser cache
47+
uses: actions/cache@v3
48+
with:
49+
path: |
50+
${{ github.workspace }}/pw-browsers
51+
key: ${{ runner.os }}-${{ hashFiles('ui-tests/yarn.lock') }}
52+
53+
- name: Install browser
54+
run: jlpm playwright install chromium
55+
working-directory: ui-tests
56+
57+
- name: Execute integration tests
58+
working-directory: ui-tests
59+
run: |
60+
jlpm test --retries=2
61+
62+
- name: Upload Playwright Test report
63+
if: always()
64+
uses: actions/upload-artifact@v4
65+
with:
66+
name: integration-jupyterlab
67+
path: |
68+
ui-tests/test-results
69+
ui-tests/playwright-report

‎.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,3 +130,7 @@ _output
130130

131131
# Jupyter
132132
Untitled*.ipynb
133+
134+
# Integration tests
135+
ui-tests/test-results/
136+
ui-tests/playwright-report/

‎schema/provider-registry.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"title": "AI provider",
33
"description": "Provider registry settings",
44
"jupyter.lab.setting-icon": "@jupyterlite/ai:jupyternaut-lite",
5-
"jupyter.lab.setting-icon-label": "JupyterLite AI Chat",
5+
"jupyter.lab.setting-icon-label": "JupyterLite AI provider",
66
"type": "object",
77
"properties": {
88
"UseSecretsManager": {

‎src/settings/panel.tsx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -467,6 +467,7 @@ export class AiSettings extends React.Component<
467467
formData={{ provider: this._provider }}
468468
schema={this._providerSchema}
469469
onChange={this._onProviderChanged}
470+
idPrefix={`jp-SettingsEditor-${PLUGIN_IDS.providerRegistry}`}
470471
/>
471472
{this.state.compatibilityError !== null && (
472473
<div className={ERROR_CLASS}>

‎ui-tests/README.md

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# Integration Testing
2+
3+
This folder contains the integration tests of the extension.
4+
5+
They are defined using [Playwright](https://playwright.dev/docs/intro) test runner
6+
and [Galata](https://github.com/jupyterlab/jupyterlab/tree/main/galata) helper.
7+
8+
The Playwright configuration is defined in [playwright.config.js](./playwright.config.js).
9+
10+
The JupyterLab server configuration to use for the integration test is defined
11+
in [jupyter_server_test_config.py](./jupyter_server_test_config.py).
12+
13+
The default configuration will produce video for failing tests and an HTML report.
14+
15+
> There is a UI mode that you may like; see [that video](https://www.youtube.com/watch?v=jF0yA-JLQW0).
16+
17+
## Run the tests
18+
19+
> All commands are assumed to be executed from the root directory
20+
21+
To run the tests, you need to:
22+
23+
1. Compile the extension:
24+
25+
```sh
26+
jlpm install
27+
jlpm build:prod
28+
```
29+
30+
> Check the extension is installed in JupyterLab.
31+
32+
2. Install test dependencies (needed only once):
33+
34+
```sh
35+
cd ./ui-tests
36+
jlpm install
37+
jlpm playwright install
38+
cd ..
39+
```
40+
41+
3. Execute the [Playwright](https://playwright.dev/docs/intro) tests:
42+
43+
```sh
44+
cd ./ui-tests
45+
jlpm playwright test
46+
```
47+
48+
Test results will be shown in the terminal. In case of any test failures, the test report
49+
will be opened in your browser at the end of the tests execution; see
50+
[Playwright documentation](https://playwright.dev/docs/test-reporters#html-reporter)
51+
for configuring that behavior.
52+
53+
## Update the tests snapshots
54+
55+
> All commands are assumed to be executed from the root directory
56+
57+
If you are comparing snapshots to validate your tests, you may need to update
58+
the reference snapshots stored in the repository. To do that, you need to:
59+
60+
1. Compile the extension:
61+
62+
```sh
63+
jlpm install
64+
jlpm build:prod
65+
```
66+
67+
> Check the extension is installed in JupyterLab.
68+
69+
2. Install test dependencies (needed only once):
70+
71+
```sh
72+
cd ./ui-tests
73+
jlpm install
74+
jlpm playwright install
75+
cd ..
76+
```
77+
78+
3. Execute the [Playwright](https://playwright.dev/docs/intro) command:
79+
80+
```sh
81+
cd ./ui-tests
82+
jlpm playwright test -u
83+
```
84+
85+
> Some discrepancy may occurs between the snapshots generated on your computer and
86+
> the one generated on the CI. To ease updating the snapshots on a PR, you can
87+
> type `please update playwright snapshots` to trigger the update by a bot on the CI.
88+
> Once the bot has computed new snapshots, it will commit them to the PR branch.
89+
90+
## Create tests
91+
92+
> All commands are assumed to be executed from the root directory
93+
94+
To create tests, the easiest way is to use the code generator tool of playwright:
95+
96+
1. Compile the extension:
97+
98+
```sh
99+
jlpm install
100+
jlpm build:prod
101+
```
102+
103+
> Check the extension is installed in JupyterLab.
104+
105+
2. Install test dependencies (needed only once):
106+
107+
```sh
108+
cd ./ui-tests
109+
jlpm install
110+
jlpm playwright install
111+
cd ..
112+
```
113+
114+
3. Start the server:
115+
116+
```sh
117+
cd ./ui-tests
118+
jlpm start
119+
```
120+
121+
4. Execute the [Playwright code generator](https://playwright.dev/docs/codegen) in **another terminal**:
122+
123+
```sh
124+
cd ./ui-tests
125+
jlpm playwright codegen localhost:8888
126+
```
127+
128+
## Debug tests
129+
130+
> All commands are assumed to be executed from the root directory
131+
132+
To debug tests, a good way is to use the inspector tool of playwright:
133+
134+
1. Compile the extension:
135+
136+
```sh
137+
jlpm install
138+
jlpm build:prod
139+
```
140+
141+
> Check the extension is installed in JupyterLab.
142+
143+
2. Install test dependencies (needed only once):
144+
145+
```sh
146+
cd ./ui-tests
147+
jlpm install
148+
jlpm playwright install
149+
cd ..
150+
```
151+
152+
3. Execute the Playwright tests in [debug mode](https://playwright.dev/docs/debug):
153+
154+
```sh
155+
cd ./ui-tests
156+
jlpm playwright test --debug
157+
```
158+
159+
## Upgrade Playwright and the browsers
160+
161+
To update the web browser versions, you must update the package `@playwright/test`:
162+
163+
```sh
164+
cd ./ui-tests
165+
jlpm up "@playwright/test"
166+
jlpm playwright install
167+
```

‎ui-tests/jupyter_server_test_config.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
"""Server configuration for integration tests.
2+
3+
!! Never use this configuration in production because it
4+
opens the server to the world and provide access to JupyterLab
5+
JavaScript objects through the global window variable.
6+
"""
7+
from jupyterlab.galata import configure_jupyter_server
8+
9+
configure_jupyter_server(c)
10+
11+
c.FileContentsManager.delete_to_trash = False
12+
13+
# Uncomment to set server log level to debug level
14+
# c.ServerApp.log_level = "DEBUG"

‎ui-tests/package.json

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"name": "jupyterlite-ai-ui-tests",
3+
"version": "1.0.0",
4+
"description": "jupyterlite-ai Integration Tests",
5+
"private": true,
6+
"scripts": {
7+
"start": "jupyter lab --config jupyter_server_test_config.py",
8+
"test": "jlpm playwright test",
9+
"test:update": "jlpm playwright test --update-snapshots"
10+
},
11+
"devDependencies": {
12+
"@jupyterlab/galata": "^5.0.5",
13+
"@playwright/test": "^1.37.0"
14+
}
15+
}

‎ui-tests/playwright.config.js

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
/**
2+
* Configuration for Playwright using default from @jupyterlab/galata
3+
*/
4+
const baseConfig = require('@jupyterlab/galata/lib/playwright-config');
5+
6+
module.exports = {
7+
...baseConfig,
8+
webServer: {
9+
command: 'jlpm start',
10+
url: 'http://localhost:8888/lab',
11+
timeout: 120 * 1000,
12+
reuseExistingServer: !process.env.CI
13+
}
14+
};

0 commit comments

Comments
 (0)