Improve the nofuzz system to allow specific exclusions #2832
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Continues the chain from #2818 to #2829 (the last of which this is currently based against).
I wasn't satisfied with the approach to just comment out & ignore large swaths of the test suite needed to implement
# nofuzz
through now, especially after thecomment_injection_fuzzer
turned out to require such large# nofuzz
regions, but I was stuck on how to proceed with specific exclusions.I think the approach taken here is pretty satisfactory: as a preprocessing step, we convert the
# nofuzz
markup into new code in the suite that signals activated/deactivated fuzzers for each execution ofexpect_lint()
.The approach could be improved -- it can be a bit of a pain to debug, e.g. when
expect_lint(file = .)
was erroring due to encoding issues, the error causes the test to exit and thus fails to run theactivate_fuzzers()
step, leading to cascading confusing errors it's hard to pin down. And there's no simple workaround here, as e.g. using awithr::defer()
approach to ensure theactivate_fuzzers()
step is run is not straightforward.But I think there's enough good progress here (and it turned up a few more true positive fixes!) that it is ready for review.