Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add special Github Actions console output #448

Open
Garados007 opened this issue May 22, 2023 · 0 comments
Open

Add special Github Actions console output #448

Garados007 opened this issue May 22, 2023 · 0 comments

Comments

@Garados007
Copy link
Member

Garados007 commented May 22, 2023

Github Actions allows special markup in the console output and has additional features which helps the developer to find errors and reports fast without looking in the respective log files. I suggest using the following features:

  1. Report of all failed WARN_* as warnings using ::warning::
  2. Report of all failed CHECK_* and REQUIRE_* as errors using ::error::
  3. Group the log lines of each test case, test suite and test run using ::group::, ::endgroup::
  4. Create a nicely human readable Markdown summary in the file referenced by the environment variable GITHUB_STEP_SUMMARY
  5. Add output variables with some metrics like number of test cases, errors, ... by writing to the file referenced by the environment variable GITHUB_OUTPUT. Actions after that can use these variables without reading the JUnit file.

Implementing 1, 2 and 4 can greatly improve the testing experience. 3 is only useful if there are lots of lines between the start and end token (maybe we should use them only for test suites and test runs).

I think this additional output should be always active if the environment variable GITHUB_ACTIONS is set to true which is always the case for Github Actions. There is no need to modify RunTest() as the environment variable can easily be set for debugging purpose and there is no other use case.

Remarks:

  • A failed expected error shouldn't produce any warning or error in the Github Actions output. There is no need to highlight them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants