-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unexpected result converting to canonical type #21
Comments
I think this is an error in UCUM itself. The UCUM value for inch is given as 2.54cm, which is a precision of 3, so 1 mile is 1.60e3m - which becomes 1609 in decimal form. The correct conversion value is 2.539999962 - so raise an issue against UCUM |
grahamegrieve
pushed a commit
that referenced
this issue
Jul 17, 2023
…cial handlers failing correctly
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Internally convert() first correctly converts 1[mi_i] to canonical unit [m] as 1609.344 but the
conversion from canonical [m] to destination type [m] converts 1609.344 to 1609 because
the identity conversion m->m uses Decimal("1",precision=1)
I would expect the unity conversion to be a noop.
The text was updated successfully, but these errors were encountered: