Skip to content

fix: ID=0 bug in Mean Average Precision calculation #1895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

balthazur
Copy link

@balthazur balthazur commented Jul 17, 2025

Description

Objects were getting 0.0 mAP despite perfect IoU matches due to a bug in annotation ID assignment. The COCO evaluator treats ID=0 as "no match", but supervision was assigning the first annotation ID=0, causing it to be incorrectly classified as a false positive.

Root Cause

# BUGGY: Started IDs from 0
"id": len(annotations),  # First object gets ID=0 → treated as "no match"

Solution

# FIXED: Start IDs from 1
"id": len(annotations) + 1,  # Start IDs from 1 (0 means no match)

Impact

  • Before: Single objects → 0.000 mAP, Multi-objects → partial failure
  • After: All perfect predictions → 1.000 mAP

Testing

Added 3 regression tests covering single objects, multiple objects, and batch updates to prevent future regressions.

Reproduction Notebooks:

supervision 0.26.0 showing the bug:
https://colab.research.google.com/drive/1naVTb7qo7rbgT_AYaD8V9CvRdhK_wwPH?usp=sharing

With fix:
https://colab.research.google.com/drive/1Kz34ec1zuDbqE7lovvTWz1REKLNsyPuM?usp=sharing

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

Tests, and in a reproduction notebook

Any specific deployment considerations

For example, documentation changes, usability, usage/costs, secrets, etc.

Docs

  • Docs updated? What were the changes:

@balthazur balthazur changed the title fix: start id with 1 because 0 means no match fix: ID=0 bug in Mean Average Precision calculation Jul 17, 2025
@balthazur balthazur marked this pull request as ready for review July 17, 2025 08:03
@balthazur balthazur mentioned this pull request Jul 17, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant