Skip to content

Refactor test suite #405

@Leonidas-from-XIV

Description

@Leonidas-from-XIV

At the moment dune-release has a fortunately rather extensive test-suite so regressions can be detected and new tests can be added to make sure the behaviour is correct and as expected by the developers. It can even be useful for users who want to see how a specific workflow is done within dune-release.

Now the current test suite has a number of problems:

  • It uses mdx instead of dune's built-in cram tests. The latter while less powerful have some advantages as in people having more familiarity with them and supporting promotion better.
  • The output is very verbose. The tests run dune-release in a way where it produces a lot of output. This is bad for people trying to read the test because the tests are so long as well as minor output changes creating tedious test failures that require a lot of churn
  • The usage of ... (elipsis) causes the tests to not allow promotion. While it sounds like a great idea to be able to mark parts of the test output as unimportant, in practice this creates problems that are easier solved with a reproducible command invocation.
  • git leaks into the tests. If unit tests don't create their git repos, they pick up the git repo from dune-release which creates annoying "repository is dirty" test failures.

For some of these points there is some work underway, #400 ports mdx over to cram but mostly in a "mechanical" way, with very little improvement on the verbosity of the tests.

For the work in 2.0.0 it would be good to start with a good test suite which is catches only actual issues and allows adding new tests easily.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions