Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rounding issue in unit conversion #22

Open
WillemRomijn-werk opened this issue Jan 13, 2025 · 1 comment
Open

Rounding issue in unit conversion #22

WillemRomijn-werk opened this issue Jan 13, 2025 · 1 comment

Comments

@WillemRomijn-werk
Copy link

Describe the bug
The scale is not set correctly when converting between units. This causes rounding errors in some cases.

To Reproduce

val energyInKwh = 3.75 * kiloWattHour
val energyInJoule1 = energyInKwh `as` joule
val energyInJoule2 = 13500000 * joule

energyInJoule1 shouldBe energyInJoule2 // green
energyInKwh shouldBe energyInJoule1 // green
energyInKwh shouldBe energyInJoule2 // red

Expected behavior
The 3rd assertion should be green as well

@geertmulders
Copy link
Contributor

geertmulders commented Feb 2, 2025

Nice find! There seems to be a rounding error indeed.

Some notes:

  1. The behaviour in your example is not defined.
    Since the 3.75 is a double, there is no way to tell what the count of significant figures is. See the example below for an example where this becomes very clear. The fun Double.times(units: U) and fun Float.times(units: U) were added later, initially the idea was to only use BigDecimal and integer / long, exactly because of this.
  2. Measures are meant for internal representation, some light computation, but not for heavy computation. We should add something about this in the readme.
  3. There is something ugly about rounding, it should only be done at the last step in a longer calculation.
val energy1 = (0.1 + 0.2) * kiloWattHour
val energy2 = 0.3 * kiloWattHour
energy1 shouldBe energy2 

What do we expect here?

For now we are only concerned with multiplication and division.

Normally rounding is done in the following way:

  • Count the number of significant figures
  • The result of the multiplication or division should have the same amount of significant figures as the quantity with the fewest in the calculation.

Here we are doing 13500000 (J) * 1 (J/J) / 3600000 (J/kWh).

  • Q: What are the number of significant figures here?
    A: 13500000 has 8 significant figures, but 1 and 3600000 have an infinite number, because they are defined this way.
  • Q: What should the result be?
    A: 3.7500000 (kWh)

How can we fix this?

I think that the BigDecimal#precision() method is just what we need here, with the proper MathContext.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants