Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark in Doku is quite old. #208

Open
stefano2734 opened this issue May 21, 2021 · 11 comments
Open

Benchmark in Doku is quite old. #208

stefano2734 opened this issue May 21, 2021 · 11 comments

Comments

@stefano2734
Copy link

And with the new features and changes in zstd 1.5, there is some interest by results and possible regressions.

The documented benchmark is excellent, but perhaps quite long for all tests.
An automatic benchmark script with comparison to older versions is also a part of a good quality system.

Here is an example of a newer test.
https://bestofcpp.com/repo/facebook-zstd-cpp-compression

@mcmilk
Copy link
Owner

mcmilk commented May 22, 2021

Yes, but currently I am short of time ... I wanted to update the benchmark in 2020 already :/

@HarvestSnowpaw
Copy link

Here's a start. I made an updated ratio vs speed chart, running benchmarks against the "generate-mcorpus" script that was used on the original, with version 1.5.0R2 x64 for a selection of the included compressors. I'd be glad to help with the rest of the benchmarks, just need a little time to set up the scripts & graphs.

Compression Ratio vs  Speed

@stefano2734
Copy link
Author

stefano2734 commented Jul 10, 2022

Thanks for this result. The scale is here not linear at x and tricky.
so I see here compression like bzip2 with speed of lz4 for zstd. Only brotli is here near.
Old Standard Deflate in most business and government is slow and with more space.

In 1.5.1 there are some performance improvements with 1.5.2 only bugfix.
Also 1.5.5 have some improvements.
So a new run with 1.5.5 is on the road for better results of zstd by new release.

@dreamflasher
Copy link

@mcmilk Thank you for your great work on this project. I'd also be very happy if you'd update the benchmarks :)

@mcmilk
Copy link
Owner

mcmilk commented Apr 2, 2023

I do not have a windows pc in the moment.

@stefano2734
Copy link
Author

stefano2734 commented Aug 15, 2023

More than 2 years after begin of this issue the benchmark should run.
Open Test by users can solve the problems of the author with time and equipment.
How about a hardware contest with different new and older hardware by amd, Intel and others?
Harvestsnowpaw made a test with 1.5.0 two years ago.
So it is not impossible.
issue #320 shows here weakness in code of 7zip.
Benchmark is here a necessary detection tool.

@HarvestSnowpaw
Copy link

HarvestSnowpaw commented Sep 7, 2023

When I get a chance, I'll update the benchmarks I did with the newest version of 7z-zstd. Thank you @stefano2734 for the note about the scale not being linear. The problem is that there is such a range of data, that otherwise the chart would be massive to account for all the speeds. I will still take a look and see if there's any way I can clean it up.

@stefano2734
Copy link
Author

When I get a chance, I'll update the benchmarks I did with the newest version of 7z-zstd. Thank you @stefano2734 for the note about the scale not being linear. The problem is that there is such a range of data, that otherwise the chart would be massive to account for all the speeds. I will still take a look and see if there's any way I can clean it up.

With non-linear scale, the user need an ideal line or milestones.
By multiprocessor there is an ideal curve of perfect scaling.
And by nonlinear scale also value lines of double or factor 4 can make better understanding.
/

@stefano2734
Copy link
Author

In Zstandard there is a actual benchmark with zstandard 1.5.6 and other compression codecs.

@dreamflasher
Copy link

The mentioned benchmarks can be found here: https://github.com/facebook/zstd?tab=readme-ov-file#benchmarks – nice! Unfortunately, they are missing fastlzma2, lzlib, lzma).

They in return reference lzbench: https://github.com/inikep/lzbench – which has a bunch of additional libs, but is unfortunately outdated for zstd.

@stefano2734
Copy link
Author

stefano2734 commented Jul 13, 2024

In GitHub of zstandard actual values to version 1.5.6 made with lzbench

https://github.com/facebook/zstd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants