Testing various new OSF encoders #652
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Including rough benchmarks here - QOI (5x faster 57% bigger file) - LZ4 (5x faster 60% bigger file) - QOI + LZ4 (5.1x faster 53% bigger file) - ZLib (20% slower than libpng with slightly worse compression) - FPNGE (AVX/SSE4.1 only) (5x faster 10% bigger file)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note this is a draft. Was just messing around with some options.
Including rough benchmarks here. Speedups are estimated since disk caching was involved and the benchmark involved other tasks.
Benchmark was converting 5 GB pcap of a roadway drive to OSF. Standard implementation took about 38 seconds wall time. All encoders besides Zlib took about 8. So about 5x faster and many things had about 7x less CPU time overall.
Note running stats on the PCAP takes about 4 seconds by itself.
Seems like FPNGE is a good option since it is backwards compatible, though only benefits X86. Perhaps a similar ARM implementation can be found.
If we are okay with a new encoding, ZPNG so far seems like the best option. It should be much faster on all platforms and can have a better compression ratio. It also seems to decode 2x faster than libpng.
Related Issues & PRs
Summary of Changes
Validation