Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warn when comparing against float.Epsilon/double.Epsilon #7528

Open
lcsondes opened this issue Jan 13, 2025 · 0 comments
Open

Warn when comparing against float.Epsilon/double.Epsilon #7528

lcsondes opened this issue Jan 13, 2025 · 0 comments

Comments

@lcsondes
Copy link

lcsondes commented Jan 13, 2025

Describe the problem you are trying to solve

Comparing floating-point values often requires a certain amount of tolerance instead of using the == operator to check for equality.

A widely-used anti-pattern is replacing a == b with MathF.Abs(a - b) < float.Epsilon or Math.Abs(a - b) < double.Epsilon.
At a glance, these look like comparisons with a small amount of tolerance, but due to the definition of Epsilon, the only number that can satisfy these checks is 0.0, which makes these == in disguise.

On platforms where denormals are flushed to zero, Epsilon == 0.0, which is even worse.

Describe suggestions on how to achieve the rule

Issue a warning that lives in Microsoft.CodeAnalysis.NetAnalyzers, enabled by default, when a floating-point value is found in a less-than relationship with the corresponding Epsilon constant of its type, such as f < float.Epsilon or double.Epsilon > d, where f and d are arbitrary expressions of type float and double, respectively.

In case the comparand is NOT an Abs(a - b)-like expression, I think that this is still warning-worthy, because x < Epsilon is just an obfuscated way of expressing x <= 0. It is extremely unlikely that this is the intended outcome of such an expression, and not a comparison with actual tolerance.

If this is considered too broad, only warn for Abs(a - b)-like expressions on the other side of the comparison. I have no good definition for this. If the compiler somehow knows that an expression cannot be negative at runtime, this information could be used.

Do NOT issue warnings if a double is compared against float.Epsilon (note the mismatched types), or a <= / >= comparison is used, but consider issuing a different warning, since this is still probably wrong (but not guaranteed wrong).

Additional context

Consider an additional, more generic warning for all comparisons against denormal constants. This additional warning could be limited to projects that actually target affected platforms (or Any CPU), and omitted for, e.g., x64-only projects.

dotnet/runtime could have benefited from such a warning. Every single one of its uses of Epsilon fell into this trap, except the dynamic test, because that one doesn't even call Abs():

float https://github.com/search?q=repo%3Adotnet%2Fruntime+%22%3C+float.epsilon%22
double https://github.com/search?q=repo%3Adotnet%2Fruntime+%22%3C+double.epsilon%22

Related issue: dotnet/runtime#62714

Related, but different suggestions: #4480, dotnet/runtime#28054

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant