Stop using a parallel sort in disjoint utils (backport #15410) #15411
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This commit switches the use of
par_sort_by_unstable()to the serialsort_by_unstable()implementation in the disjoint layout module. While the parallel sort is faster and has some heuristics to limit the multithreaded usage when the overhead isn't worth the tradeoff, the use of multithreading from Python is potentially fraught when mixing with multiprocessing in Python. For this reason all of our multithreaded calls are supposed to guard against running in parallel based on env vars that indicate the rust code is being executed in a multiprocessing context. However the parallel sorts were missing this check and in this case the overhead of checking we can run in multiple threads will likely outweigh any speedup from a parallel sort. So this commit just opts to drop the parallel sort and avoid the potential complications.Details and comments
This is an automatic backport of pull request #15410 done by [Mergify](https://mergify.com).