Skip to content

Linter diff test helper for testing different options #19351

Open
@MichaReiser

Description

@MichaReiser

Our current approach to testing preview behavior (or any configurable rule behavior) is to create two tests: One with preview disabled and one with preview enabled. However, both tests run on the same input file.

This makes it very hard to spot differences between preview on/off. That's why it would be helpful to have a helper that: given an input file, runs the rule on that file with preview off, then with preview on and only snapshots the diagnostics that were removed or added in preview. This should help to drastically reduce the snapshot sizes for our preview tests.

This concept can be extended to testing differences between arbitrary settings where the base run uses the default settings and the second run uses whatever settings the user provided (preview, a custom option, etc)

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedContributions especially welcomelinterRelated to the lintertestingRelated to testing Ruff itself

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions