You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
The scale is not set correctly when converting between units. This causes rounding errors in some cases.
To Reproduce
val energyInKwh =3.75* kiloWattHour
val energyInJoule1 = energyInKwh `as` joule
val energyInJoule2 =13500000* joule
energyInJoule1 shouldBe energyInJoule2 // green
energyInKwh shouldBe energyInJoule1 // green
energyInKwh shouldBe energyInJoule2 // red
Expected behavior
The 3rd assertion should be green as well
The text was updated successfully, but these errors were encountered:
Nice find! There seems to be a rounding error indeed.
Some notes:
The behaviour in your example is not defined.
Since the 3.75 is a double, there is no way to tell what the count of significant figures is. See the example below for an example where this becomes very clear. The fun Double.times(units: U) and fun Float.times(units: U) were added later, initially the idea was to only use BigDecimal and integer / long, exactly because of this.
Measures are meant for internal representation, some light computation, but not for heavy computation. We should add something about this in the readme.
There is something ugly about rounding, it should only be done at the last step in a longer calculation.
val energy1 = (0.1+0.2) * kiloWattHour
val energy2 =0.3* kiloWattHour
energy1 shouldBe energy2
What do we expect here?
For now we are only concerned with multiplication and division.
Normally rounding is done in the following way:
Count the number of significant figures
The result of the multiplication or division should have the same amount of significant figures as the quantity with the fewest in the calculation.
Here we are doing 13500000 (J) * 1 (J/J) / 3600000 (J/kWh).
Q: What are the number of significant figures here?
A: 13500000 has 8 significant figures, but 1 and 3600000 have an infinite number, because they are defined this way.
Q: What should the result be?
A: 3.7500000 (kWh)
How can we fix this?
I think that the BigDecimal#precision() method is just what we need here, with the proper MathContext.
Describe the bug
The scale is not set correctly when converting between units. This causes rounding errors in some cases.
To Reproduce
Expected behavior
The 3rd assertion should be green as well
The text was updated successfully, but these errors were encountered: