Coverlet MTP extension feature #1788
Open
+10,958
−258
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bertk
added
feature
Collaborator
Author
|
The code coverage dropped and shall be fixed before merge. main branch: line 90% branch 87% |
Bertk
added
the
* NO MERGE *
Bertk
force-pushed
the
coverlet-MTP
branch
2 times, most recently
from
592ff6a to
010a409
Compare
- Updated target frameworks to net472 in coverlet.core and coverlet.msbuild.tasks projects - Adjusted CoverletToolsPath for multi-targeting support in buildMultiTargeting props and targets - Created unit tests for Coverlet.MTP command line options validation - Added documentation for Coverlet.MTP integration
- Added CoverletExtensionCollector to handle test session lifecycle for coverage collection. - Introduced CoverletExtensionCommandLineProvider for command line options. - Created CoverletExtensionConfiguration to manage configuration settings. - Developed CoverletLoggerAdapter for logging integration with Microsoft Testing Platform. - Implemented CoverletExtensionEnvironmentVariableProvider for environment variable management. - Added CoverletExtensionProvider to register the Coverlet extension with the testing platform. - Created TestingPlatformBuilderHook to facilitate extension registration. - Updated project files to include necessary dependencies and configurations for Coverlet. - Added support for multiple target frameworks (net8.0 and net9.0). - Included build and packaging configurations for Coverlet.MTP. - Implemented command line options for coverage report formats and exclusions. - Established logging mechanisms for better traceability during coverage collection.
…ng and add coverage cleanup messages
…ationString for consistency and lowercase output
… consistency across environments
There was a problem hiding this comment.
Pull request overview
This pull request introduces Microsoft Testing Platform (MTP) support to coverlet through a new coverlet.MTP package, enabling code coverage collection for projects using the modern Microsoft Testing Platform instead of VSTest.
Key Changes:
- New
coverlet.MTPpackage with full MTP extension implementation - Core infrastructure updates to support managed instrumentation restore control
- Comprehensive test projects for validation and unit testing
- CI/CD workflow additions including GitHub Actions
- Dependency upgrades to support .NET 8/9 and MTP v2.x
- Documentation for MTP integration
Reviewed changes
Copilot reviewed 100 out of 105 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| src/coverlet.MTP/* | New MTP extension package with command-line providers, collectors, and configuration |
| src/coverlet.core/Helpers/InstrumentationHelper.cs | Added DisableManagedInstrumentationRestore parameter to control backup/restore behavior |
| src/coverlet.core/Coverage.cs | Added parameter support for instrumentation restore control |
| src/coverlet.collector/* | Extended collector settings with DisableManagedInstrumentationRestore flag |
| test/coverlet.MTP.unit.tests/* | Unit tests for MTP extension components |
| test/coverlet.MTP.validation.tests/* | Integration validation tests with sample projects |
| test/coverlet.tests.utils/TestUtils.cs | Added GetBuildConfigurationString() helper for cross-platform path handling |
| Directory.Packages.props | Upgraded dependencies including Microsoft.Testing.Platform v2.0.2, xunit v3.2.1 |
| .github/workflows/dotnet.yml | New GitHub Actions workflow for cross-platform CI/CD |
| Documentation/Coverlet.MTP.Integration.md | Comprehensive MTP integration guide with examples |
| global.json | Updated SDK version to 9.0.308 |
| eng/.yml, eng/.sh | Updated build and test scripts for MTP projects |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Updated target frameworks to net472 in coverlet.core and coverlet.msbuild.tasks projects - Adjusted CoverletToolsPath for multi-targeting support in buildMultiTargeting props and targets - Created unit tests for Coverlet.MTP command line options validation - Added documentation for Coverlet.MTP integration
- Added CoverletExtensionCollector to handle test session lifecycle for coverage collection. - Introduced CoverletExtensionCommandLineProvider for command line options. - Created CoverletExtensionConfiguration to manage configuration settings. - Developed CoverletLoggerAdapter for logging integration with Microsoft Testing Platform. - Implemented CoverletExtensionEnvironmentVariableProvider for environment variable management. - Added CoverletExtensionProvider to register the Coverlet extension with the testing platform. - Created TestingPlatformBuilderHook to facilitate extension registration. - Updated project files to include necessary dependencies and configurations for Coverlet. - Added support for multiple target frameworks (net8.0 and net9.0). - Included build and packaging configurations for Coverlet.MTP. - Implemented command line options for coverage report formats and exclusions. - Established logging mechanisms for better traceability during coverage collection.
…ng and add coverage cleanup messages
…ment for consistency across environments" This reverts commit 06a684c.
- Expanded MTP integration docs and added usage ToDo - Enabled package generation in coverlet.MTP.csproj - Added strong name signing for validation tests - Updated validation test project: new dependencies, utility refs, and test project copying - Updated InternalsVisibleTo for new test assemblies
Introduce a comprehensive validation test suite for the Coverlet Microsoft Testing Platform (MTP) extension, including new integration and CLI option tests. Add isolated test infrastructure with sample projects and custom MSBuild props/targets to ensure tests run in a pure MTP environment. Update packaging logic in coverlet.MTP to improve dependency handling and NuGet layout, and adjust build system to better support MTP scenarios. Also includes minor bug fixes and ensures correct local package versioning in tests.
- Enhance test output with detailed diagnostics and error context
- Use robust assertions with informative failure messages
- Create temp test projects under artifacts/tmp for isolation
- Add retries and file attribute handling to test dir cleanup
- Check for test executable and coverlet.MTP.dll before running
- Refactor HelpCommandTests for consistent path handling
- Fix sample class naming mismatch in test project
- Add condition to MSBuild import for props file robustness
- Update NuGet config and project file handling for clarity
Renamed the constant sutName to SutName in HelpCommandTests to follow .NET PascalCase naming conventions for constants. Updated all references to use the new name for consistency.
Switch all test projects to xunit.v3.mtp-v2 for Microsoft Testing Platform (MTP) integration. Bump Microsoft.Testing.Platform to 2.0.2 and manage Moq version via property. Make Coverlet MTP extension opt-in by default and register as TestingPlatformExtension. Minor formatting and encoding adjustments included.
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 24, 2025
- Add comprehensive CoverageConfigurationTests.cs for all config options using Moq - Fix devcontainer.json syntax and ensure GitHub Actions extension is included - Enable Dependabot for GitHub Actions and NuGet with custom schedule - Improve Coverlet.MTP.Integration.md: typo fix, clarify usage, update sample project and coverage command - Simplify build.yml by unifying test execution to cross-platform dotnet exec - Make CoverletDataCollector version dynamic from assembly - Remove default CoverletMTPEnabled property and make props import conditional - Set IsTestingPlatformApplication to false in csproj and tidy formatting
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 25, 2025
Youssef1313
reviewed
Dec 25, 2025
Youssef1313
reviewed
Dec 25, 2025
- Eliminate --coverlet-output and --coverlet-source-mapping-file options from Coverlet.MTP integration, code, and docs. - Coverage output directory now determined by Microsoft Testing Platform's test result directory. - Update documentation and help output to reflect new behavior and remove obsolete references. - Remove CoverletDataCollector class; rely on CoverletExtensionCollector and MTP extension model. - Update build script and test projects for new output handling and configuration. - Improve documentation examples for merging and reporting coverage. - Refine table formatting and minor test project settings.
Introduce CoverletExtensionCollectorTests to comprehensively verify construction, option parsing, environment variable handling, and integration with Microsoft Testing Platform extension interfaces. Tests cover normal and edge cases, ensuring correct behavior without modifying production code.
- Log all generated coverage report paths after report generation for better user visibility. - Add comprehensive unit tests for CoverageConfiguration.LogConfigurationSummary, verifying output for all config options and edge cases. - Add new tests for CoverletExtensionCollector, covering environment variable handling, validation, and process exit scenarios. - Enhances reliability, error handling, and user experience through improved logging and test coverage.
Bertk
changed the title
- Document new coverlet.MTP driver for Microsoft Testing Platform - Update README with installation, usage, and options for MTP - Add coverlet.MTP to driver/version table and quick start - Clarify MTP requirements and usage notes - Update changelog: add MTP feature, fix typos, standardize contributor links - Minor doc improvements: link formatting, section clarity, credits, and author status
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 102 out of 107 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/coverlet.collector/DataCollection/CoverletSettingsParser.cs
Outdated
Show resolved
Hide resolved
.../coverlet.MTP.validation.tests/TestProjects/CalculateTestProject/CalculateTestProject.csproj
Outdated
Show resolved
Hide resolved
Removed the sample calculator test project and related files. Improved documentation comments in extension and settings parser code. Updated build pipeline: clarified permissions in dotnet.yml and cleaned up test/coverage steps in build.yml.
Bertk
removed
the
* NO MERGE *
- Introduce in-process handler for test host coverage flushing and diagnostics - Centralize environment variable and debug constants - Add DebugHelper for debugger attach/launch and diagnostics - Register both controller and test host extensions in builder - Propagate tracker/exception log settings to test host - Update docs: architecture diagrams, env vars, troubleshooting - Improve test log verification and code clarity - Clean up project files and test helpers
Introduce ICoverage/ICoverageFactory abstractions and a CoverageFactory for dependency injection and improved testability. Refactor CoverletExtensionCollector to use these abstractions and support factory injection. Update Version properties to use assembly version. Add comprehensive unit tests for environment variable provider, instrumentation diagnostics, in-process handler, and collector logic. Grant InternalsVisibleTo for test project and add direct reference to coverlet.core. Includes minor code cleanups and copyright headers.
Refactored CoverletExtensionCollectorTests to reduce redundancy and improve maintainability. Removed several granular tests for environment variable behaviors and replaced them with a comprehensive end-to-end test that verifies main integration points, environment variable settings, and logging. Added a helper for mock setup and improved process exit tests to cover both success and cancellation scenarios. Edge case tests were removed as they are now covered by broader tests.
- Greatly expand and refactor tests for configuration, collector, and session handler classes - Standardize namespaces and using directives to coverlet.MTP.* - Add thorough tests for all configuration options, logging, and integration scenarios - Add full lifecycle and edge case tests for the collector, including environment variable handling and error conditions - Expand session handler tests to cover all constructors, properties, and interface methods - No production logic changes; all updates are to test code and organization - Improves coverage, robustness, and maintainability of the extension
Comprehensive tests added for CoverletExtensionCollector's report generation (multiple formats, error handling, directory creation, logging) and InstrumentationHelper's backup/restore logic (DLL/PDB restore, missing files, duplicate backup, skipping running assembly). Also includes minor formatting and brace fixes. These changes improve coverage and robustness for critical scenarios.
tebeco
reviewed
Jan 2, 2026
| <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" /> | ||
| <PackageReference Include="xunit.v3" Version="2.0.0" /> | ||
| <PackageReference Include="xunit.runner.visualstudio" Version="3.1.0"> | ||
| <PackageReference Include="xunit.v3" Version="3.2.1" /> |
There was a problem hiding this comment.
@Youssef1313 since xunit.v3 is targetting mtp v1
shouldn't tools directly use xunit.v3.mtp-v2 ?
I feel like this will just re-create chaos again when xunit.v3 will be targeting MTP v2
AArnott/Library.Template#452 (comment)
There was a problem hiding this comment.
Yes. Generally speaking, people should be encouraged to use MTP v2.
This is an example project for documentation though, and I'm not sure what the intent of it is. e.g, is it intended to be used or explain usage with VSTest anyways?
Collaborator
Author
There was a problem hiding this comment.
Yes. Generally speaking, people should be encouraged to use MTP v2.
This is an example project for documentation though, and I'm not sure what the intent of it is. e.g, is it intended to be used or explain usage with VSTest anyways?
I updated the coverlet examples to use MPTv2 package. Unfortunately xUnit.net v3 'dotnet new' Templates is using package "xunit.v3".
Current package generate this project file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<OutputType>Exe</OutputType>
<RootNamespace>xunit_test</RootNamespace>
<TargetFramework>net8.0</TargetFramework>
<!--
This template uses native xUnit.net command line options when using 'dotnet run' and
VSTest by default when using 'dotnet test'. For more information on how to enable support
for Microsoft Testing Platform, please visit:
https://xunit.net/docs/getting-started/v3/microsoft-testing-platform
-->
</PropertyGroup>
<ItemGroup>
<Content Include="xunit.runner.json" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageReference Include="xunit.v3" Version="3.2.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5" />
</ItemGroup>Switched test projects to xunit.v3.mtp-v2 for MTP support. Enhanced build pipeline with consistent results directory, improved diagnostics, and merged test results. Added copilot-instructions.md with coding and testing standards for contributors.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces several infrastructure and configuration improvements to the project, focusing on modernizing dependencies, enhancing CI/CD workflows, and enforcing code quality standards.
Key update: New coverlet.MTP package for Microsoft Testing Platform support
Continuous Integration and Automation:
.github/workflows/dotnet.ymlGitHub Actions workflow to automate building, testing (with coverage), and reporting for .NET projects across Ubuntu, Windows, and macOS. This workflow includes steps for restoring dependencies, building, running tests with coverage, generating reports, and publishing test results and coverage summaries to PRs..github/dependabot.ymlto enable automated dependency updates with Dependabot, scheduled weekly.Dependency and Package Management:
Directory.Packages.props, includingMicrosoft.CodeAnalysis(to 4.13.0),Microsoft.NET.Test.Sdk(to 18.0.1),xunit.v3(to 3.2.1), and others. Added new packages for Microsoft Testing Platform, banned API analyzers, and updated several system and test libraries to their latest versions.CoverletVersionto 8.0.0 inDirectory.Build.propsand improved test logger and output configuration for better compatibility with new test platforms and CI environments.Solution Structure:
coverlet.MTP,coverlet.MTP.validation.tests,coverlet.MTP.unit.tests) to the solution file, including their build and folder organization.Most important changes:
1. Solution and API Governance
2. Documentation
Documentation/Coverlet.MTP.Integration.md).