Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output geometric mean to 0.1% precision #123

Open
markshannon opened this issue Feb 2, 2022 · 3 comments
Open

Output geometric mean to 0.1% precision #123

markshannon opened this issue Feb 2, 2022 · 3 comments

Comments

@markshannon
Copy link

Outputs such as "0% slower" or "1% faster" are unnecessarily vague.
Is "1%" 0.5% or 1.5%?

@vstinner
Copy link
Member

vstinner commented Feb 2, 2022

Do you suggest to add one more digit in the output? Do you want to propose a PR for that?

@sweeneyde
Copy link

While 0.4% on individual benchmarks is probably not noteworthy, it's nice to see on the geometric mean of an entire suite. Very few changes will ever move the pyperformance geometric mean more than a couple of percentage points, so most of the time we see one of (1.01 slower, 1.00 slower, 1.00 faster, 1.01 faster, 1.02x faster), which isn't very much information IMO.

Another option would be to add three decimal places to pyperformance only (which doesn't display geometric mean at all yet).

@ericsnowcurrently
Copy link

While 0.4% on individual benchmarks is probably not noteworthy, it's nice to see on the geometric mean of an entire suite.

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants