You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
By default, UT assert runs in verbose (not debug) mode, where all calls to any assert macro are added to a log, except for DEBUG messages. This means there is a record of tests that PASS, which is generally desired for coverage testing.
However, if doing fuzz testing, the point is to run a test against a large, random data set and as such there may be thousands (or even millions) of assertions. This will result in a very large log file full of "PASS" messages.
UT Assert does allow the verbosity to be set, but currently only via command line on a run-by-run basis.
Describe the solution you'd like
Allow a test to specify what its default verbosity is supposed to be. For fuzz testing this can be a lower value than the normal coverage testing default.
Describe alternatives you've considered
Currently test cases that run in a loop or otherwise would generate a large number of PASS results use a pattern where the result is checked directly in the test code, e.g.:
if (status != expected)
{
UtAssert_Failed("message");
}
Such that an assertion is only logged if it fails. However, a side effect of this means that the "pass" counter and total test counter does not increment to reflect that something was tested/checked - it only increments if it fails. In contrast, by using the normal assert macro but reducing the verbosity means that the counters will still increment, it just doesn't write a log entry for PASS cases. Furthermore, a user could manually increase the log level if they actually do want to see them all.
Requester Info
Joseph Hickey, Vantage Systems, Inc.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
By default, UT assert runs in verbose (not debug) mode, where all calls to any assert macro are added to a log, except for DEBUG messages. This means there is a record of tests that PASS, which is generally desired for coverage testing.
However, if doing fuzz testing, the point is to run a test against a large, random data set and as such there may be thousands (or even millions) of assertions. This will result in a very large log file full of "PASS" messages.
UT Assert does allow the verbosity to be set, but currently only via command line on a run-by-run basis.
Describe the solution you'd like
Allow a test to specify what its default verbosity is supposed to be. For fuzz testing this can be a lower value than the normal coverage testing default.
Describe alternatives you've considered
Currently test cases that run in a loop or otherwise would generate a large number of PASS results use a pattern where the result is checked directly in the test code, e.g.:
Such that an assertion is only logged if it fails. However, a side effect of this means that the "pass" counter and total test counter does not increment to reflect that something was tested/checked - it only increments if it fails. In contrast, by using the normal assert macro but reducing the verbosity means that the counters will still increment, it just doesn't write a log entry for PASS cases. Furthermore, a user could manually increase the log level if they actually do want to see them all.
Requester Info
Joseph Hickey, Vantage Systems, Inc.
The text was updated successfully, but these errors were encountered: