Add NUMA support and optimizations #11871
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Enable NUMA optimizations via CMake options - Introduce NUMA-aware thread assignment and memory allocation - Add NUMA debugging utilities - Enhance cache management for NUMA systems
|
Nice! The PR uses the old Debug() statements, which needs to be changed to the new Dbg / Ctl features: |
cmcfarlen
requested changes
Nov 15, 2024
src/cripts/Lulu.cc
Outdated
| { | ||
| return details::splitter<Cript::string_view>(input, delim); | ||
| } | ||
|
|
There was a problem hiding this comment.
You can remove this chunk, perhaps a merge conflict issue.
src/iocore/net/Server.cc
Outdated
| int my_thread_id = this_ethread()->id; | ||
| int my_numa_node = this_ethread()->get_numa_node(); | ||
|
|
||
| Debug("numa_sequencer", "[NUMASequencer] Thread %d (NUMA node %d) entered run_sequential.", my_thread_id, my_numa_node); |
There was a problem hiding this comment.
Debug has been removed, please use the Dbg macro instead.
src/iocore/net/Server.cc
Outdated
| if (opt.f_mptcp) { | ||
| Dbg(dbg_ctl_connection, "Define socket with MPTCP"); | ||
| prot = IPPROTO_MPTCP; | ||
| } | ||
|
|
||
| // Create the socket | ||
| Debug("numa", "[Server::listen] Attempting to create socket with family: %d, type: %d, protocol: %d", addr.sa.sa_family, |
There was a problem hiding this comment.
These Debug lines were maybe for your debug? Could they be removed?
cmcfarlen
requested changes
Nov 15, 2024
| } | ||
|
|
||
| RamCache * | ||
| RamCacheContainer::get_cache(unsigned int my_node, unsigned int node) |
There was a problem hiding this comment.
Suggested change
| RamCacheContainer::get_cache(unsigned int my_node, unsigned int node) | |
| RamCacheContainer::get_cache(unsigned int /* my_node ATS_UNUSED */, unsigned int node) |
|
|
||
| // returns true if consistent | ||
| static bool | ||
| check_pages_consistency(void *data, size_t size, const char *name = "") |
There was a problem hiding this comment.
This function is unused unless NUMA_CONSISTENCY_CHECK is defined. Please add precompile check here.
| } | ||
|
|
||
| static void | ||
| move_pages_to_current_numa_zone(void *data, size_t size) |
src/iocore/net/Server.cc
Outdated
|
|
||
| // If all threads have been added (assuming their number is equal to eventProcessor.net_threads), sort the thread IDs and set | ||
| // ready_to_run to true | ||
| if (thread_ids.size() == eventProcessor.net_threads) { |
There was a problem hiding this comment.
I get a signed/unsigned compare error for this line.
cmcfarlen
requested changes
Nov 20, 2024
| getcpu(&cpu, &node); | ||
| this->numa_node = node; | ||
| } | ||
| return this->numa_node; |
There was a problem hiding this comment.
This should handle when numa.h is not available
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This pull request adds support for Non-Uniform Memory Access (NUMA) to Apache Traffic Server, enhancing performance on NUMA systems.
Key Changes:
Performance testing has shown increased throughput, reduced CPU usage, improved latency, decreased UPI bus load
These changes aim to optimize memory access patterns and reduce latency on NUMA systems.
To build the application with NUMA support, use the following CMake command:
cmake -B build -DCMAKE_BUILD_TYPE=Release -DENABLE_MIMALLOC=ON -DENABLE_NUMA=ON -DENABLE_HWLOC=ON -Dmimalloc_DIR=""