Output geometric mean to 0.1% precision #126
Conversation
|
CC @vstinner |
|
Do you really care of a difference smaller than 1%? Could it be an option? I would prefer to display least digits as possible for the most common use cases. |
| @@ -39,9 +39,9 @@ def format_normalized_mean(norm_mean): | |||
| if norm_mean == 1.0: | |||
| return "no change" | |||
| elif norm_mean < 1.0: | |||
| return "%.2fx faster" % (1.0 / norm_mean) | |||
There was a problem hiding this comment.
Maybe leave this as it is, but take a precision argument (which can default to 2).
Then in format_geometric_mean() call format_normalized_mean(geo_mean, 3)
|
If possible, I would prefer to keep 2 digits after the dot ("1.02x faster") for benchmarks. Otherwise, people will fight to death for a digit which is not significant and means nothing. People often misunderstand and misread benchmark results. Maybe pyperf default output should be simplified to not digits :-D Only say: faster, slower or not significant ;-) By the way, IMO pyperf should say "not significant" in more cases ;-) But I'm open to the idea of adding a 3rd digit after the dot for the geometric mean which cumulates results of multiple benchmarks. pyperf stores all data in JSON. pyperf can have many options to change the formatting of the output. Maybe it can be a command line option to show "more digits". How many? I don't know. I am fine with adding options, people are free to use data as they want. I mostly care about keeping the default formatting unchanged to not open discussions about the least significant digit which is usually not significant. It's hard to draw a line between "significant digit" and "not significant digit". It depends on the duration of a single benchmark run, the kind of benchmark, etc. pyperf knows little about the benchmark. |
Closes #123.