Skip to content

fix: Add bounding box to nutrition extraction #1545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

abderhman487
Copy link

@abderhman487 abderhman487 commented Apr 12, 2025

What

  • Added bounding box computation for nutrition extraction using OCR word positions
  • Added bounding_box column to both prediction and product_insight tables
  • Included bounding box in nutrition extraction insights
  • Created an endpoint to allow users to propose a new bounding box during validation
  • Modified the image cropping function to use the stored bounding box information

Implementation Details

  • Enhanced NutrientPrediction and NutritionExtractionPrediction classes to include bounding box information
  • Added compute_bounding_box function to calculate the overall bounding box from individual word positions
  • Updated the NutrientExtractionImporter to include bounding box data in generated insights
  • Added support for updating bounding box through annotation in NutrientExtractionAnnotator

How This Improves User Experience

This enhancement significantly improves the visualization of nutrition extraction results during validation. Previously, the bounding box was only used internally for cropping, but was not exposed in the insight data. Now, users validating nutrition insights can see exactly which part of the image was analyzed, and even propose corrections to the bounding box if needed.

Fixes

Add bounding box to nutrition extraction #1539

@abderhman487
Copy link
Author

@alexgarel

@alexgarel
Copy link
Member

@abderhman487 so sorry, I only see this PR now, I will try to review it soon !

@abderhman487
Copy link
Author

@alexgarel @raphael0202 Any updates?

@raphael0202
Copy link
Collaborator

Hello @abderhman487,
I'm sorry for the review delay. I will review your PR this week!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

3 participants