Skip to content

Conversation

@sAchin-680
Copy link
Contributor

Description

This PR implements automated code coverage validation for OpenTelemetry Collector components as outlined in issue #14098. The new checkcover tool enforces coverage requirements based on component stability levels and integrates seamlessly into the existing CI pipeline.

Key Features:

  • Enforces 80% minimum coverage for stable components
  • Supports repository-wide minimum via CLI flag (--repo-minimum)
  • Allows components to set higher requirements via coverage_minimum in metadata.yaml
  • Integrated into CI workflow (runs after test-coverage job)
  • Includes comprehensive unit tests and documentation

Link to tracking issue

Addresses #14066 (Automate validation and fulfillment of criteria outlined in the "Component Stability" document)

Fixes #14098 (Validate and enforce test coverage requirements)

Testing

  • Added unit tests for checkcover tool (validator_test.go)
    • Coverage file parsing and calculation
    • Minimum coverage determination logic
    • Module path resolution
  • All tests passing: 8 test cases across 4 test functions
  • Verified mdatagen tests still pass with schema changes
  • Tool builds successfully and CLI interface works correctly

Documentation

  • Tool Documentation: cmd/checkcover/README.md - Usage guide and examples
  • User Guide: docs/coverage-requirements.md - Component author guidelines
  • Implementation Details: COVERAGE_VALIDATION_IMPLEMENTATION.md - Technical documentation
  • Schema Documentation: Updated cmd/mdatagen/metadata-schema.yaml with coverage_minimum field description
  • All documentation includes examples and troubleshooting guidance

@sAchin-680 sAchin-680 requested review from a team and dmitryax as code owners November 6, 2025 08:45
@codecov
Copy link

codecov bot commented Nov 6, 2025

Codecov Report

❌ Patch coverage is 27.69953% with 154 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.91%. Comparing base (09a2719) to head (cf8161f).

Files with missing lines Patch % Lines
cmd/checkcover/validator.go 32.57% 109 Missing and 9 partials ⚠️
cmd/checkcover/main.go 0.00% 32 Missing ⚠️
cmd/mdatagen/internal/status.go 33.33% 2 Missing and 2 partials ⚠️

❌ Your patch check has failed because the patch coverage (27.69%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #14134      +/-   ##
==========================================
- Coverage   92.24%   91.91%   -0.34%     
==========================================
  Files         657      659       +2     
  Lines       41068    41281     +213     
==========================================
+ Hits        37884    37944      +60     
- Misses       2179     2321     +142     
- Partials     1005     1016      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dmathieu
Copy link
Member

dmathieu commented Nov 6, 2025

From a quick look:

IMHO, the command would be better off in internal/cmd. This is not needed as a public tool.
Also, your PR currently has the compiled binary committed. That should be gitignored.

@sAchin-680
Copy link
Contributor Author

Thanks @dmathieu! Both fixed in 8a7394e:

  • Moved to checkcover (not a public tool)
  • Removed binary and added to .gitignore

All tests passing. Let me know if you'd like any further changes or modifications.

@sAchin-680 sAchin-680 force-pushed the feat/coverage-validation-14098 branch from 8a7394e to a3a4353 Compare November 11, 2025 10:20
…lemetry#14098)

Key features:
- Validates stable components meet 80% coverage minimum
- Supports repository-wide minimum via CLI flag
- Allows components to set higher requirements in metadata.yaml
- Integrated into existing CI workflow
- Includes unit tests and comprehensive documentation

Fixes open-telemetry#14098

Signed-off-by: SACHIN KUMAR <[email protected]>
- Added comprehensive unit tests for validator.go
- Created main_test.go for CLI testing
- Added coverage_minimum validation tests in status_test.go
- Improved test coverage for error paths, edge cases, and verbose mode
- Coverage breakdown:
  * LoadCoverage: 95.6%
  * ValidateComponents: 91.7%
  * findMetadataFiles: 92.3%
  * validateComponent: 92.0%
  * getModulePath: 90.0%
  * run: 93.3%

Total improvement: +59.6 percentage points
Addresses PR feedback from @dmathieu:
- Move checkcover from cmd/ to internal/cmd/ (not needed as public tool)
- Remove committed binary from git tracking
- Add binary to .gitignore to prevent future commits

Changes:
- Moved cmd/checkcover -> internal/cmd/checkcover
- Updated module path in go.mod
- Updated CI workflow to reference new path
- Updated documentation references
- Added checkcover binaries to .gitignore
- All tests passing
@sAchin-680 sAchin-680 force-pushed the feat/coverage-validation-14098 branch from a99551c to b7b86cd Compare November 13, 2025 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validate and enforce test coverage requirements

3 participants