[Performance]: Process affinity to CPU cores with multiple sockets support

[HaiShaw]

Motivation

Setting process affinity to designated cpu ids, to avoid unwanted process migrations (cost high in case of multiple sockets)

Modifications

Assign specific process to designated cores, handle multiple sockets, and HT (hyper threading: 1 Physical core runs as 2 logical cores)

Online benchmarking shows notable improvement (request rate: 4/8/16) on MI300X system with dual socket CPUs.

Checklist

• [+] Format your code according to the Contributor Guide.
• [+] Add unit tests as outlined in the Contributor Guide.
• [+] Update documentation as needed, including docstrings or example tutorials.

[merrymercy]

We have multiple threads, not sure whether we need to call them all.
also call it here

sglang/python/sglang/srt/managers/tp_worker_overlap_thread.py

Line 90 in 55842eb

with torch.cuda.stream(self.forward_stream):

[HaiShaw]

We have multiple threads, not sure whether we need to call them all. also call it here

sglang/python/sglang/srt/managers/tp_worker_overlap_thread.py

Line 90 in 55842eb

with torch.cuda.stream(self.forward_stream):

We don't need to call this per thread, currently we look forward to binding at process level. :)