Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pr aims to improve dream eml performance after this issue. During testing it was determined that the main cause of slow performance is excessive buffers allocations. The fix is a simple pool of buffers, created once per file.
This pr also introduces 2 new options for dream eml executable: buffer-size and pool-size. They are self-explanatory: buffer-size is the size of each of the buffers created and pool-size is the initial amount of buffers created.
I have considered doing a project-wide pool, but that seems to be impossible as dream_eml is launched per file and is not imported for .eml files.
Hopefully due to ocaml multithreading model, not placing a mutex on the buffer was a correct solution (please correct me if I'm wrong)
Test is done via simple benchmark, an eml file generating lots of repeating divs is created. After eml preprocessing, time is measured using ocaml-benchmark for different volumes of html. I have then plotted relative time of executions. Here is a plot of relative speed up, comparing previous and newer iterations. On average, 2.6x speed up is observed.
It is mostly related to major heap allocations, and reducing those, especially for bigger values, resulted in the speedup