Upgrade WebdriverIO to v9.20.0 #1770

Yuyuutsu · 2025-11-03T11:26:41Z

What is the context of this PR?

This PR upgrades all WDIO packages to v9.20.0 and updates our functional tests to comply with the new upgrades.

Changes made

getHTML() no longer takes a boolean flag. It now accepts an options object. The default options:
({ includeSelectorTag: true, prettify: true }) were causing differences in the returned HTML, which led to assertion failures (formatting changes and missing alt attributes on logo tags). We have to explicitly set these values to false.
newWindow() now needs to explicitly have a URL value. In v8, newWindow() implicitly opened about:blank when no URL was provided. In v9 this implicit default was removed, so we now explicitly pass 'about:blank' when opening a new window.
SDS Tests in GA are failing when in the main test, but work when separated into their own job with "MAX_INSTANCE=1". Also the very first SDS launch always hits the “Sorry, there is a problem with this service” page. So a new helper function - openQuestionnaireWithRetry(). I think we need an investigation on why SDS is failing like this in GA only.
Chrome options also updated in an attempt to make tests more consistent.
Babel dependencies have also been upgraded to latest versions

How to review

Run Functional tests locally on main and then on this branch and see if they all work as expected
Check all test-functional commands work as expected
Do you agree with the changes made? There are multiple routes we can take with these changes, happy to hear out any suggestions.

Links

Checklist

New static content marked up for translation
Newly defined schema content included in eq-translations repo

Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

… for more robust testing

…stness

…conditions

…tly used in webdriverv8 Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

…on and interestital spec

…plemenetary data tests are executing before page is rendered Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

berroar

Was just wondering with:
Some tests became flaky due to reliance onbrowser.pause() to avoid race conditions. Therefore a new helper function has been added, waitForPageToLoad(), which waits for the page to fully load.
Did we test without replacing these at all? I think they were initially added before we went back to running consecutively?

Yuyuutsu · 2025-11-24T15:09:24Z

Was just wondering with: Some tests became flaky due to reliance onbrowser.pause() to avoid race conditions. Therefore a new helper function has been added, waitForPageToLoad(), which waits for the page to fully load. Did we test without replacing these at all? I think they were initially added before we went back to running consecutively?

Tested on a seperate branch and works without browser.pause. Changes have now been added to this branch 👍

berroar

Observing a lot of The data in this module is over two months old. To ensure accurate Baseline data, please update: npm i baseline-browser-mapping@latest -D` in the run logs.

…pping

Yuyuutsu · 2025-11-28T14:28:31Z

Observing a lot of The data in this module is over two months old. To ensure accurate Baseline data, please update: npm i baseline-browser-mapping@latest -D` in the run logs.

Transitive package from:

@babel/core@7.28.5
└─ @babel/helper-compilation-targets@7.27.2
   └─ browserslist@4.26.2
      └─ baseline-browser-mapping@2.8.6

I have upgraded the dependencies in babel so now baseline-brwoser-mapping is on the latest version.
Other devs are having the same issue with this warning message.

Update: new version of basline-browser-mapping has been released and resolves the issue.

tests/functional/spec/thank_you.spec.js

Yuyuutsu · 2026-01-13T14:33:04Z

tests/functional/wdio.conf.cjs

  // directory is where your package.json resides, so `wdio` will be called from there.
  //
-  specs: ["./spec/**/*.js"],
+  specs,


We now define Specs at the top, depedning on what the MODE variable equals too. So we can sepearte the SDS Specs (flakey) from the rest of the Specs

Yuyuutsu · 2026-01-13T14:36:16Z

tests/functional/wdio.conf.cjs

          booleanFlag,
        });
-        this.url(`/session?token=${token}`);
+        await this.url(`/session?token=${encodeURIComponent(token)}`);


await to prevent any race conditions - flakiness
encodeURIcomponent for an added layer of security

Yuyuutsu · 2026-01-13T14:37:44Z

Makefile

+	npm run test_functional_main
+
+test-functional-sds: generate-pages
+	EQ_FUNCTIONAL_TEST_MAX_INSTANCES=1 npm run test_functional_sds


Flakiness was persisting when Instances was higher than 1

Yuyuutsu · 2026-01-13T14:39:34Z

Just added a few comments explaining reasons for certain changes

liamtoozer

Changes look great and just wanted to call out great PR description as well 👍

I think this is more to do with the bump to wdio v9, but I wonder if some of the new flakiness we're seeing more of in Actions now is because of this. I think we might need to spend a bit of extra time to prevent introducing extra flakiness for some of these tests, because I'm seeing a few different failures that weren't cropping up as often with v8. Appreciate that you're trying to tackle this already though 👍 Not sure what the solution is yet but we might need to park this and spawn off an investigation into some of the newer intermittent failures?

Also, just a very minor suggestion, I think we might benefit from calling getHTML() from inside a central wrapper/helper function, and defaulting with the { prettify: false } object there instead. Otherwise, passing the object in multiple place might be a maintenance overhead if we need to refactor in future? Might help keep some of the assertions to look a bit cleaner?

liamtoozer · 2026-01-20T09:32:19Z

...s/functional/spec/components/checkbox/mutually_exclusive/mutually_exclusive_currency.spec.js

  beforeEach(async () => {
    await browser.openQuestionnaire("test_mutually_exclusive.json");
-    await browser.pause(100);
+


Very, very (very) minor: Not sure if the linter might have done this, but there are now blank lines where the previous browser.pause() was? I think there are a few files like this that might need to be updated too

liamtoozer · 2026-01-21T12:15:29Z

tests/functional/spec/introduction.spec.js

  });

  it("Given I start a survey, When I view the introduction page, Then I should be able to see introduction information", async () => {
-    await browser.openQuestionnaire(introductionSchema);


Good spot with these redundant calls 👀

upgrade webdriver

4ead39b

Yuyuutsu changed the title ~~upgrade webdriver~~ Upgrade Webdriver to v9.20.0 Nov 3, 2025

Yuyuutsu added 19 commits November 4, 2025 05:50

update whole webdriver libary to 9.20.0

90b938f

Set getHTML prettify to false

58f3796

Set Prettify to false in getHTML

89a9325

fix formatting error with getHTML prettify change

9ed2756

fix formatting in funcitonal tests

e74b924

update getHTML tag inlcudeSelectorTag to equal false on svg tags

2b0e71a

Remove unecessary brownser.open stratments and update wait check

291924c

update list collector summary item html structure test

613bde3

Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

Add wait browser to load helper function to replace browser.wait(100)…

e2934a8

… for more robust testing

fix linting errors in functional tests

f4d1c61

set includeselectortag to false for nhse spec

b65d862

add includeselectortags for logo checks

9d814e8

Add wait for questionnaire to load functions to tests to imporve robu…

62f545f

…stness

Add wait for questionnaire to load functions to tests to imporve robu…

a811342

…stness

fix linting and reformat tests

0e3e82f

Add waitForQuestionniareToLoad functions to flakey tests due to race …

08afdeb

…conditions

Update tests to reduce flakinesses

ed2fc10

update newwindow to explictly use about:blank which used to be implic…

9feb657

…tly used in webdriverv8 Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

fix linting errors using make format

5710a2a

Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

Yuyuutsu marked this pull request as ready for review November 5, 2025 10:57

Yuyuutsu requested a review from a team as a code owner November 5, 2025 10:57

Fix race connditioning issues in supplementary data calculated questi…

8e8d298

…on and interestital spec

Yuyuutsu changed the title ~~Upgrade Webdriver to v9.20.0~~ Upgrade WebdriverIO to v9.20.0 Nov 5, 2025

Yuyuutsu added 5 commits November 5, 2025 12:09

add waitForQuestionnaireToLoad to prevent flakiness

28c423c

add waits for questionnaires to load

56ab8d9

add a wait function before checking for elements

ca7e110

update waitForQuestionnaireToload to waitFroPageToLoad because of sup…

d2115d3

…plemenetary data tests are executing before page is rendered Signed-off-by: Yuyutsu Rai <Yuyutsu.Rai@ons.gov.uk>

update logic for waitForPageToLoad

0bd9170

Yuyuutsu added 5 commits November 20, 2025 10:23

revert debugging configs

8f33f06

remove accidental revert

bbcebb1

Revert back to 2 max instances

e53b154

re-add updated chrome options

3ae1699

remove comment in pull request yml

d3ed675

berroar reviewed Nov 20, 2025

View reviewed changes

update tests to remove browser.wait()

9ac0e96

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

e33d387

berroar reviewed Nov 27, 2025

View reviewed changes

Upgrade Babel to upgrade transistive dependency - baseline-browser-ma…

6e2b298

…pping

rmccar reviewed Dec 1, 2025

View reviewed changes

tests/functional/spec/thank_you.spec.js Show resolved Hide resolved

Yuyuutsu and others added 3 commits December 3, 2025 09:22

upgrade glob in package.lock

d60e135

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

a2ad69b

update basline-browser-mapping

7a91163

rmccar mentioned this pull request Dec 4, 2025

Bump tar-fs from 3.0.4 to 3.1.1 in the npm_and_yarn group across 1 directory #1762

Closed

Yuyuutsu and others added 6 commits December 4, 2025 14:36

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

5d7be77

remove chromium-bidi

a61f60b

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

2614a09

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

309f039

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

57b3617

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

a62b1b9

Yuyuutsu commented Jan 13, 2026

View reviewed changes

Yuyuutsu added 2 commits January 15, 2026 10:06

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

d33c2c3

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

1a4d987

liamtoozer reviewed Jan 21, 2026

View reviewed changes

Merge branch 'main' into upgrade-webdriver-to-v9.20.0

66b1588

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade WebdriverIO to v9.20.0 #1770

Upgrade WebdriverIO to v9.20.0 #1770

Yuyuutsu commented Nov 3, 2025 •

edited

Loading

Uh oh!

berroar left a comment

Uh oh!

Yuyuutsu commented Nov 24, 2025

Uh oh!

berroar left a comment

Uh oh!

Yuyuutsu commented Nov 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

Yuyuutsu Jan 13, 2026

Uh oh!

Yuyuutsu Jan 13, 2026

Uh oh!

Yuyuutsu Jan 13, 2026

Uh oh!

Yuyuutsu commented Jan 13, 2026

Uh oh!

liamtoozer left a comment

Uh oh!

liamtoozer Jan 20, 2026

Uh oh!

liamtoozer Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Upgrade WebdriverIO to v9.20.0 #1770

Are you sure you want to change the base?

Upgrade WebdriverIO to v9.20.0 #1770

Conversation

Yuyuutsu commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the context of this PR?

Changes made

How to review

Links

Checklist

Uh oh!

berroar left a comment

Choose a reason for hiding this comment

Uh oh!

Yuyuutsu commented Nov 24, 2025

Uh oh!

berroar left a comment

Choose a reason for hiding this comment

Uh oh!

Yuyuutsu commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Yuyuutsu Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Yuyuutsu Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Yuyuutsu Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Yuyuutsu commented Jan 13, 2026

Uh oh!

liamtoozer left a comment

Choose a reason for hiding this comment

Uh oh!

liamtoozer Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

liamtoozer Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Yuyuutsu commented Nov 3, 2025 •

edited

Loading

Yuyuutsu commented Nov 28, 2025 •

edited

Loading