Changes that add and integrate code coverage support into XDMoD's CI pipeline #1443
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ryanrath
requested review from
jtpalmer and
plessbd
and removed request for
jtpalmer
jpwhite4
requested changes
Sep 10, 2020
| browser.setValue(input, setting.updated); | ||
| const updatedValue = browser.getValue(input); | ||
| expect(updatedValue).to.equal(setting.updated); | ||
| for (let i = 0; i < 100; i++) { |
There was a problem hiding this comment.
This will hide test failures if the exception is thrown more than 100 times
There was a problem hiding this comment.
If you want to make multiple attempts then you'll need to (a) add a short pause between tries (e.g. 10ms), and (b) make sure to throw an exception if the max attempts is reached.
There was a problem hiding this comment.
I've ended up removing these for loops as the ended up being unnecessary with the latest updates.
| browser.click(inputTrigger); | ||
| setTimeout(function () { | ||
| browser.click(inputTrigger); | ||
| }, 3000); |
There was a problem hiding this comment.
three seconds seems like a long time to wait here. Why was it increased so much?
There was a problem hiding this comment.
This and the previous waitForVisible could also be combined into a waitAndClick
There was a problem hiding this comment.
I've actually ended up removing the setTimeout calls and everything still seems to be passing both w/ and w/ out XDebug.
plessbd
reviewed
Sep 10, 2020
ryanrath
force-pushed
the
zrcode_bkup
branch
5 times, most recently
from
180516d to
02428b3
Compare
plessbd
reviewed
Sep 22, 2020
plessbd
previously approved these changes
Nov 2, 2020
ryanrath
force-pushed
the
zrcode_bkup
branch
from
6f42a7b to
db77bb7
Compare
ryanrath
force-pushed
the
zrcode_bkup
branch
from
c5ba815 to
2ff4108
Compare
- Added four new scripts responsible for installing & configuring XDebug, generating the code coverage data from php file execution, and generating the final reports. - Updated `shippable.yml` so that XDebug / Code Coverage is only generated during the nightly test build. - Updated numerous UI tests to work more reliably when XDebug is installed ( i.e. their timeouts were increased ) - Updated a couple of UI test selectors to minimize the effects of latency on particular tests. - Added a new webdriverhelper function called `clickSelectorAndWaitForMask` as this was a commonly feature during developing this PR. - Updated the timeout for a few other webdriverhelper functions.
- It was found while running these tests on the CentOS8 image, that the `setupBeforeClass` function was not being called as expected, this small addition will catch if the required `self::$hashFilePath` variable has been set, and if not then run the `setupBeforeClass` function so that it is populated.
- Since we're now running this script on CentOS7|8 and Rocky8.5 containers we need to take into account the different software versions / configuration options that must be set to successfully collect the code coverage data generated by our tests.
By installing Xdebug and setting up code coverage before running bootstrap we'll generate code coverage information for our install / upgrade code including the ETL code.
ryanrath
force-pushed
the
zrcode_bkup
branch
from
120462f to
dcaac39
Compare
|
I force triggered the code coverage generation in the last commit to make sure that it works. Before I set it back I just wanted to document the code coverage artifacts from each build:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
NOTE: this PR takes the place of #1350
All of the functional changes are the work of Zi-3 all that I've (@ryanrath) done is clean things up a little.
Description
generating the code coverage data from php file execution, and generating the final
reports.
shippable.ymlso that XDebug is installed / Code Coverage is only generated during the nightlytest build.
clickSelectorAndWaitForMaskas this was needed for a test or two.Motivation and Context
Adding code coverage will allow us to get a general sense of where we are lacking test code & possibly where we have unused code thus giving us a few more tools to help streamline / increase the reliability of XDMoD.
Tests performed
All the automated tests for an install / upgrade with & without XDebug installed.
Checklist: