You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While working on #5096, I found it difficult to determine if a change to the invalid_test_parameters() logic in the integrationv2 framework would result in changes to the tests that are run. It would be nice to have assurance that changes don't result in unintentionally reducing test coverage.
Solution:
We should consider snapshot testing the integrationv2 tests that are invoked for each libcrypto by committing the list of invoked tests into the repo and comparing this list to a new list generated in each PR. If these lists differ, the PR should fail until the committed list is updated. This will ensure that any changes to test coverage are acknowledged by the author and observable to reviewers.
The text was updated successfully, but these errors were encountered:
goatgoose
changed the title
Snapshot test for invoked integrationv2 tests
Add snapshot test for invoked integrationv2 tests
Feb 6, 2025
Problem:
While working on #5096, I found it difficult to determine if a change to the invalid_test_parameters() logic in the integrationv2 framework would result in changes to the tests that are run. It would be nice to have assurance that changes don't result in unintentionally reducing test coverage.
Solution:
We should consider snapshot testing the integrationv2 tests that are invoked for each libcrypto by committing the list of invoked tests into the repo and comparing this list to a new list generated in each PR. If these lists differ, the PR should fail until the committed list is updated. This will ensure that any changes to test coverage are acknowledged by the author and observable to reviewers.
The text was updated successfully, but these errors were encountered: