Skip to content

fix: correct area handling in MeanAveragePrecision for size-specific mAP #1894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

balthazur
Copy link

@balthazur balthazur commented Jul 16, 2025

Description

This PR fixes an error in supervision.metrics.mean_average_precision.MeanAveragePrecision where the area used for size-specific evaluation (small / medium / large) was always zero unless explicitly provided in Detections.data.

The fix now:

  1. Checks data["area"] if present (COCO import path).
  2. Otherwise falls back to Detections.area[target_idx], leveraging Supervision’s property which already handles both bbox and mask cases.

In addition, a lightweight test suite (test/metrics/test_mean_average_precision_area.py) has been added to guard against regressions.

To reproduce:

Use a non-coco dataset as ground-truth (in my case yolov5). I am using from_yolo followed by using the Detection objects as GT's. Than, create the predictions using from_inference with results from our API. Pass both to the mAP calculator.

Before:
image

The areas will be 0 (the default fallback), leading to wrong/non-existing metrics for the specific sizes.

After:
image

Dependencies: none.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

Running tests and testing with a yolov5 dataset.

The new tests cover:
Normal detections (empty data) – area is auto-calculated.
COCO detections (area in data) – provided area is preserved.
Mixed batch – both code paths together.
Size-specific mAP no longer returns -1.0 for medium / large.
Verification that the evaluator truly uses Detections.area.
All existing tests continue to pass.

Any specific deployment considerations

None – pure Python change, no API or dependency impact.

Docs

  • Docs updated? What were the changes:

@balthazur balthazur self-assigned this Jul 16, 2025
@balthazur balthazur added the bug Something isn't working label Jul 16, 2025
@balthazur balthazur marked this pull request as ready for review July 16, 2025 18:14
@balthazur balthazur mentioned this pull request Jul 17, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant