-
-
Notifications
You must be signed in to change notification settings - Fork 469
feat: init data quality dimensions #11953
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
benbenben2
wants to merge
14
commits into
main
Choose a base branch
from
feat/data-quality-dimensions
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
❌ 26 Tests Failed:
View the top 3 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
/update_tests_results |
…penfoodfacts/openfoodfacts-server into feat/data-quality-dimensions
/update_tests_results |
…nfoodfacts/openfoodfacts-server into feat/data-quality-dimensions
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
🧪 additives
API READ
All READ APIs include Product, Search…
config
🧽 Data quality
https://wiki.openfoodfacts.org/Quality
Display
📚 Documentation
Documentation issues improve the project for everyone.
exports
🕹️ Gamification
GitHub Actions
Pull requests that update Github_actions code
🍬 How much sugar
https://howmuchsugar.in/ - Mini game powered by Open Food Facts data
💥 Merge Conflicts
💥 Merge Conflicts
👮 Moderation
🧴 Open Beauty Facts
Our cosmetic analysis project https://world.openbeautyfacts.org
🐾 Open Pet Food Facts
Our pet food analysis project https://world.openpetfoodfacts.org
📸 Open Products Facts
Our project to increase the lifespan of objects. https://world.openproductsfacts.org
Products
status system
Products have a number of completion states, eg: https://world.openfoodfacts.org/states
Tags
🧬 Taxonomies
https://wiki.openfoodfacts.org/Global_taxonomies
🧪 tests
🧪 unit tests
update all products
Web
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
TL;DR:
introduced data quality dimensions (lib/ProductOpener/DataQualityDimensions.pm, tests/unit/dataqualitydimensions.t),
based on states and completeness (lib/ProductOpener/Products.pm),
all other changes are consequences of these changes.
discussion
redundancy between states and data quality info-> changes: moved all states into quality info, except pro platform related tests (en:to-be-exported, en:to-be-automatically-exported). "# On the producers platform, keep track of which products have changes to be exported" not sure what to do with that
strange with steps in the actual implementation and half steps divided by 4 -> changes: increment everything we check and divided passed tests by total count. Note that with previous implementation if all tests pass the score is 1.1
typo packaging -> packagings
packaging-code (emb) should be only for EU product and for products from animal origin
ingredients should be extended for all selected languages. Picture for ingredients as well.
en:checked is for accuracy dimension and not for completeness
origins: refers to "Origin of ingredients" not "Origin of the product and/or its ingredients (English)" and not "countries where sold" this is very rarely input. changes: removed this check.
$notempty can be replaced by the completeness score itself, if the score is 0 notempty is also 0. changes: removed this state.
completed_t, why to have this field? 1) poor chance to have all complete 2) what does it bring? 3) there should be another dashboard separately to see the score overtime, this dashboard would replace that field. changes: removed this field.
removed "en:complete" from gen_sugar/gen_sucres as it seems not working, the following 2 products appeared in the game but do not contain complete
https://world.openfoodfacts.org/api/v2/product/6001052001018
https://world.openfoodfacts.org/api/v2/product/5038862130530
Screenshot
Related issue(s) and discussion