Help: Parsing Product Nutrition Info from NFP Images

[gautam-ergo]

I'm working on a school project that involves extracting product nutrition information from images of Nutrition Facts Panels (NFPs) using Optical Character Recognition (OCR) and Natural Language Processing (NLP) techniques. I've encountered some challenges in parsing the extracted text data and would greatly appreciate any guidance.

I've just stepped in to OCR parsing world and been using the Hugging Face Transformers library to perform OCR on NFP images, and came across https://huggingface.co/openfoodfacts/nutrition-extractor which lead me to this repo 🚀 . Went through the api docs as well which pointed to https://openfoodfacts.github.io/robotoff/references/api/#tag/Predict/paths/~1predict~1nutrition/get but was not entirely sure how all this ties together.

a sample nfp image:

I'd love to hear from this community. Thanks.

[prasanna7codes]

hey @gautam-ergo this seems like a really cool project you are working on ! I would love to know how much progress you have made and can learn from you ! ( PS : I am a newbie trying to contribute in open source projects )

[raphael0202]

Hello @gautam-ergo, I've moved this to the discussion section as it is a more appropriate section for this kind of requests.
First, sorry for not replying sooner. You can find a demo standalone script to run the model here: https://github.com/openfoodfacts/openfoodfacts-ai/blob/develop/nutrisight/demo.py.
It requires the original image and the Google Cloud Vision result in a JSON format. The API was designed with Open Food Facts images in mind, for which the OCR result is always available on the same image server.