You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem/Opportunity
Today, most of the tests in the Etaoin test suite are positive tests that validate that Etaoin can do something. They test the "happy path." There are little to no test cases that validate Etaoin's behavior in error conditions.
Proposed Solution
The behavior of each API call under error conditions needs to be understood (see issue #639) and then tests written to validate the documented behavior.
Alternative Solutions
None. This is just grunt work.
Additional context
It always sucks writing negative test cases, but they're pretty important, particularly for something like Etaoin that purports to work across multiple browsers. Very few users have the time or ability to debug error cases across multiple browsers. The Etaoin project infrastructure does this routinely, however, so all that needs to be done (he says, blithely, waving his hands) is to add negative test cases to the existing Etaoin test suite.
The text was updated successfully, but these errors were encountered:
dgr
changed the title
Improve coverage of negative test cases in test suite
Improve coverage of test cases in test suite
Aug 23, 2024
Ok, but please be aware that some % of test coverage is not a goal.
Once you run a coverage tool, it can be tempting to strive for 100% coverage.
That is not a goal for Etaoin and never will be.
Problem/Opportunity
Today, most of the tests in the Etaoin test suite are positive tests that validate that Etaoin can do something. They test the "happy path." There are little to no test cases that validate Etaoin's behavior in error conditions.
Proposed Solution
The behavior of each API call under error conditions needs to be understood (see issue #639) and then tests written to validate the documented behavior.
Alternative Solutions
None. This is just grunt work.
Additional context
It always sucks writing negative test cases, but they're pretty important, particularly for something like Etaoin that purports to work across multiple browsers. Very few users have the time or ability to debug error cases across multiple browsers. The Etaoin project infrastructure does this routinely, however, so all that needs to be done (he says, blithely, waving his hands) is to add negative test cases to the existing Etaoin test suite.
The text was updated successfully, but these errors were encountered: