Add a tool to find uncovered files

[liamappelbe]

If a file is not imported by the tests, it will not be compiled, so won't be mentioned in the source report. The compiler/VM won't be aware that the file even exists. This means that entire files can be uncovered without affecting the coverage report.

We could add a tool (or an option in an existing tool) that searches for all the .dart files in the package directory and reports any that are not mentioned in the coverage data.

The tool would need some filtering options, because it's common to have scripts that are not intended to be tested or used outside of the package (eg scripts in the tool directory). Maybe by default it should only look in the src subdirectory.

The tool will also need to support workspaces when they are added.

Context: flutter/flutter#145609

[khushi-hura]

hey @liamappelbe! As far as I can understand, this issue focusses on developing a tool which could search for some uncovered files during the test of the project. Could you please correct me if I am wrong or could you please ellaborate the issue?
I think I can work on it, will be sharing my approach in a few days!
Your response will be thankful :)

[liamappelbe]

hey @liamappelbe! As far as I can understand, this issue focusses on developing a tool which could search for some uncovered files during the test of the project. Could you please correct me if I am wrong or could you please ellaborate the issue? I think I can work on it, will be sharing my approach in a few days! Your response will be thankful :)

Yep, that's the basic idea. Thinking a bit more about the design, it probably shouldn't be a separate tool. Instead it should be baked into the formatter. Add an optional parameter to FileHitMapsFormatter.formatLcov (and probably also FileHitMapsFormatter.prettyPrint) that controls the behavior of this feature.

The API could look something like this:

extension FileHitMapsFormatter on Map<String, HitMap> {
  String formatLcov(
    Resolver resolver, {
    String? basePath,
    List<String>? reportOn,
    Set<Glob>? ignoreGlobs,
    Function<bool(String)>? includeUncovered,  // <- new option
  }) {

If includeUncovered is null the behavior should stay the same as it is now. Otherwise it should be a function that returns whether the given path should be included in the repot if it's not covered. We'll also need a way for users to easily specify a sensible default behavior (eg only searching in the lib/src directory), so maybe add static function that does this and users can pass that for the typical behavior. That's just an idea though, so feel free to propose other APIs.

Once that's working you'll need to add a flag for it to bin/format_coverage.dart, and bin/test_with_coverage.dart. Once it's landed and published I'll add the option to Flutter. That way we'll get the new behavior across all the Dart and Flutter coverage use cases.

[khushi-hura]

Okay got that, will try working and let u know if I need any further help!

[Victowolf]

@liamappelbe @khushi-hura
Hello everyone, I'm Yash!

To tackle this issue, we can start by scanning all the .dart files in the package and comparing them with the coverage report. This would help identify files that aren’t covered at all (the ones never tested).

I had a few questions:

1)Should this be integrated into package: coverage, or would it be better as a standalone tool?
2)What kind of filtering options should be considered? Would a simple ignore list (like .gitignore) be enough, or should it be more flexible?
3)Should the default behavior apply only to lib/src/, or should we allow users to configure directories?

[liamappelbe]

1)Should this be integrated into package: coverage, or would it be better as a standalone tool?

It should be integrated into the formatter functions at first, then we'll add options for it to bin/format_coverage.dart and bin/test_with_coverage.dart. The command line option will probably be a list of globs.

2)What kind of filtering options should be considered? Would a simple ignore list (like .gitignore) be enough, or should it be more flexible?

Make it a Function<bool(String)>. That's the most flexible approach.

3)Should the default behavior apply only to lib/src/, or should we allow users to configure directories?

Default should be lib/src/. If the user wants to configure it more, they can pass a function or set command line flags.

[Victowolf]

@liamappelbe
Thanks for the detailed feedback! I will update the PR, that I submitted earlier to reflect these suggestions.

• I will integrate the changes into the formatter functions first and then add the necessary options to bin/format_coverage.dart and bin/test_with_coverage.dart.
• I'll implement filtering using a Function<bool(String)> to provide maximum flexibility.
• The default behavior will apply to lib/src/, but I'll ensure users can configure directories via function parameters or command-line flags.

I'll push the updates soon.

[Victowolf]

@liamappelbe
I have made the necessary updates as discussed:

• Integrated the filtering functionality into the collect function using a Function<bool(String)>? filter parameter. This allows for customizable file exclusion while collecting coverage data.
• Ensured that the default behavior applies to lib/src/, while allowing users to define their own filtering logic through a function.
• Made sure that the implementation remains flexible by accepting a filtering function rather than a static list.

Let me know if any further changes.

[Victowolf]

Hi @liamappelbe,
I’ve submitted a new PR that successfully implements the --include-uncovered flag for LCOV formatting, as discussed. This includes detection of all Dart files in the specified --report-on directory and appending uncovered ones with zero hits to the final LCOV output.

Looking forward to your review!
Thanks!
PR !!