You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The produced output 1.8011300000000001 is at least not what I would expect from ToShortest.
This code path doesn't generate a fallback to bignum calculations. I can't fully deduce from the comments whether this is intentionally.
But the net outcome is that this makes ToShortest a no go for our usage.
We have implemented an overlay of ToShortest that takes an additional max_digits input that we can use to solve the problem, but I'm wondering whether there is a better / more general solution.
The text was updated successfully, but these errors were encountered:
The produced output 1.8011300000000001 is at least not what I would expect from ToShortest.
This code path doesn't generate a fallback to bignum calculations. I can't fully deduce from the comments whether this is intentionally.
But the net outcome is that this makes ToShortest a no go for our usage.
We have implemented an overlay of ToShortest that takes an additional max_digits input that we can use to solve the problem, but I'm wondering whether there is a better / more general solution.
The text was updated successfully, but these errors were encountered: