Skip to content

Add python lit test coverage #1936

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Add python lit test coverage #1936

wants to merge 10 commits into from

Conversation

mehrdad2m
Copy link
Contributor

@mehrdad2m mehrdad2m commented Jul 24, 2025

Context:
Currently, Catalyst's code coverage reporting only includes Python pytest tests, missing coverage from lit tests. This could result in an incomplete picture of the actual coverage, especially as the Python compiler would rely on lit tests for testing its core compilation functionality.

Description of the Change:
This PR combines lit test coverage with the existing pytest coverage. In particular, it adds a new lit-coverage target for running lit tests with coverage enabled and updates coverage-frontend to combine pytest and lit coverage data using coverage combine.

Benefits:
More accurate overall coverage metrics

Possible Drawbacks:

Related GitHub Issues:
[sc-95508]

@dime10
Copy link
Contributor

dime10 commented Jul 24, 2025

🤯

Copy link

codecov bot commented Jul 24, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.15%. Comparing base (ca53255) to head (3a40904).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1936      +/-   ##
==========================================
+ Coverage   96.61%   97.15%   +0.54%     
==========================================
  Files          66       86      +20     
  Lines        7645     9643    +1998     
  Branches      898      897       -1     
==========================================
+ Hits         7386     9369    +1983     
- Misses        195      219      +24     
+ Partials       64       55       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mehrdad2m mehrdad2m changed the title [WIP] Add python lit test coverage Add python lit test coverage Jul 25, 2025
@mehrdad2m mehrdad2m requested a review from a team July 25, 2025 19:32
Copy link
Contributor

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md on your branch with:

  • A one-to-two sentence description of the change. You may include a small working example for new features.
  • A link back to this PR.
  • Your name (or GitHub username) in the contributors section.

@mehrdad2m
Copy link
Contributor Author

Hello. You may have forgotten to update the changelog! Please edit doc/releases/changelog-dev.md on your branch with:

  • A one-to-two sentence description of the change. You may include a small working example for new features.
  • A link back to this PR.
  • Your name (or GitHub username) in the contributors section.

I don't think this requires a changelog entry since it includes only build system changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants