Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Round values like 1.239999 #8501

Open
CharlesNepote opened this issue Jun 2, 2023 · 6 comments · May be fixed by #11147
Open

Round values like 1.239999 #8501

CharlesNepote opened this issue Jun 2, 2023 · 6 comments · May be fixed by #11147
Assignees
Labels
🧽 Data quality https://wiki.openfoodfacts.org/Quality Nutrition facts ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. ✅ Task

Comments

@CharlesNepote
Copy link
Member

Dozens of thousands of products have weird values like 0.40000000596046, while it's printed 0.4 on the product:
image

And the corresponding values in the JSON file:
image

This does not lead to data quality errors' detection, but it's having annoying drawbacks for data quality:

  • these values are wrong: it gives a bad opinion of Open Food Facts
  • there are a bit harder to read (and they make values review a little bit harder)
  • people waste time to fix it

More than 97,000 products are in this case (click on the [count] button).

As we never see values like this on the products, I would be in favor to automatically round these values.

The rule could be the following: any number containing more than 0000 or 9999 after the decimal separator should be rounded.

@aleene
Copy link
Contributor

aleene commented Jun 2, 2023

I think one of the apps sends these kind of numbers.

I would be in favour of some rounding routine, say to one decimal.

@alexgarel
Copy link
Member

alexgarel commented Jun 2, 2023

I would be in favour of some rounding routine, say to one decimal

I think it should be relative… so 1 decimal or 1% of precision.

@Naruyoko
Copy link
Contributor

Naruyoko commented Jun 2, 2023

It looks exactly like the result of converting from single precision floating point values, which have 24-bit fractional parts. We could cut off below 0.00000011920928955078125 (though maybe make it just a bit larger in case the ratio calculation itself introduces an error) times the value since that is the ratio between the values of the least and the most significant bits. I doubt it, but do any products have more precision?

Alternatively, we might be able to detect special patterns in the binary representation in double-precision, which should show a long string of either 0s or 1s from a certain point (ignoring a few last significant bits- rounding errors might emerge there too). For example, double represents 0.40000000596046 as 3FD999999FFFFFAF, compared to the 0.4 which becomes 3FD999999999999A. This method should prevent genuine precise values form being rounded, but it might be overengineering.

Wikipedia article on the single-precision floating-point format

@DanieliusAsm
Copy link
Contributor

DanieliusAsm commented Jun 3, 2023

I would suggest rounding to 2 decimal. Example Pictures of nutrition (salt):
image
image
image

But 2.7 then shouldnt become 2.70

@stephanegigandet
Copy link
Contributor

There's some code and some dicussion about this issue here: #3489

@github-actions
Copy link
Contributor

This issue has been open 90 days with no activity. Can you give it a little love by linking it to a parent issue, adding relevant labels and projets, creating a mockup if applicable, adding code pointers from https://github.com/openfoodfacts/openfoodfacts-server/blob/main/.github/labeler.yml, giving it a priority, editing the original issue to have a more comprehensive description… Thank you very much for your contribution to 🍊 Open Food Facts

@github-actions github-actions bot added the ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. label Sep 15, 2023
@teolemon teolemon moved this to To discuss and validate in 🍊 Open Food Facts Server issues Apr 23, 2024
@benbenben2 benbenben2 self-assigned this Dec 21, 2024
@benbenben2 benbenben2 moved this from To do to Needs review in 🧽 Ensuring Data Quality Dec 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧽 Data quality https://wiki.openfoodfacts.org/Quality Nutrition facts ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. ✅ Task
Projects
Status: To discuss and validate
Status: Needs review
8 participants