Upload JUnit Test Reports to Codecov for Enhanced Test Visibility #13363
Conversation
psf-chronographer
bot
added
the
bot:chronographer:provided
.github/workflows/test.yml
Outdated
| if: "matrix.use_coverage" | ||
| uses: codecov/codecov-action@v5 | ||
| with: | ||
| fail_ci_if_error: false | ||
| files: ./coverage.xml | ||
| files: ./coverage.xml,./report.xml |
There was a problem hiding this comment.
Thanks @BahramF73!
According to https://github.com/codecov/codecov-action, there is also report_type:
The type of file to upload, coverage by default. Possible values are "test_results", "coverage".
Do we need to set that as well, or does code-cov detect the types of file automatically? I could not find any information regarding that on a quick search over the docs (but likely I missed something).
There was a problem hiding this comment.
I found this and I added report_type.
Awesome, yeah, let’s see how it goes! |
.github/workflows/test.yml
Outdated
| @@ -268,6 +268,15 @@ jobs: | |||
| files: ./coverage.xml | |||
| verbose: true | |||
|
|
|||
| - name: Upload test results to Codecov | |||
| if: "matrix.use_coverage" | |||
| uses: codecov/codecov-action@v5 | |||
There was a problem hiding this comment.
Are you sure this works? The docs you've linked mention codecovcli do-upload --report-type test_results under Set up by directly using the CLI, but under Set up using the action they suggest a dedicated codecov/test-results-action@v1 action, including a suggestion to use if: ${{ !cancelled() }} with it. Why not follow that?
There was a problem hiding this comment.
You're right. That action is what must be used. I just run one after the other in my projects: https://github.com/tox-dev/workflow/blob/89de3c6/.github/workflows/reusable-tox.yml#L422-L469.
tox.ini
Outdated
| @@ -62,6 +62,7 @@ setenv = | |||
| coverage: _PYTEST_TOX_EXTRA_DEP=coverage-enable-subprocess | |||
| coverage: COVERAGE_FILE={toxinidir}/.coverage | |||
| coverage: COVERAGE_PROCESS_START={toxinidir}/.coveragerc | |||
| coverage: PYTEST_ADDOPTS=--junitxml=report.xml | |||
There was a problem hiding this comment.
The docs say:
Include the optional
junit_familyflag to have pretty-printed test names in the Codecov UI.
There was a problem hiding this comment.
I'd recommend trying to integrate this in CI only since local runs don't normally need XML files.
changelog/12689.improvement.rst
Outdated
| @@ -0,0 +1 @@ | |||
| Upload JUnit test reports to Codecov to enable test result visibility in the Codecov UI. | |||
There was a problem hiding this comment.
I think it's confusing to have this as an improvement changelog entry, as that's user-facing improvements. I think contrib would fit much better.
…uration in CI workflow Rename improvement documentation and update test result upload configuration in CI workflow
tox.ini
Outdated
| @@ -62,6 +62,7 @@ setenv = | |||
| coverage: _PYTEST_TOX_EXTRA_DEP=coverage-enable-subprocess | |||
| coverage: COVERAGE_FILE={toxinidir}/.coverage | |||
| coverage: COVERAGE_PROCESS_START={toxinidir}/.coveragerc | |||
| coverage: PYTEST_ADDOPTS=--junitxml=junit.xml -o junit_family=legacy | |||
There was a problem hiding this comment.
JUnit isn't coverage, though... Also, is it possible to use long option names?
P.S. It'd be good not to generate this in local runs, but only CI.
There was a problem hiding this comment.
It'd be good not to generate this in local runs, but only CI.
TBH I wouldn't care too much... we need however to add junit.xml to .gitignore. However, if it is simple to not generate this for local runs, we can go ahead, but I wouldn't complicate things too much if not.
There was a problem hiding this comment.
Fair. In my CIs, I have built-in ways of having this separation. Perhaps, one day I'll get to refactoring this one as well...
Description
Upload JUnit test reports to Codecov to enable test result visibility in the Codecov UI.
Checklist
changelogfolder, with a name12689.improvement.rst.Changelog
Added a new changelog entry: changelog/12689.improvement.rst.
Additional Notes
This PR does not fix any specific issue but improves the integration with Codecov by uploading JUnit test reports.
Closes #12689