-XX:ActiveProcessorCount doesn't limit the number of available cores as we might think

[franz1981]

• On quarkus 3 (without RunOnVirtualThread): -XX:ActiveProcessorCount wont't constrain the size of our (blocking) worker pool using platform threads, allowing it to consume more than the suggested number of processors
• On quarkus 3 with RunOnVirtualThread: -XX:ActiveProcessorCount correctly enforce the Loom ForkJoin pool handling VirtualThreads to be sized correctly (in term of platform threads). This won't still honor the suggested number of processors, because we internally size the Netty event loop count equal to the number of cores (i.e -XX:ActiveProcessorCount) meaning that the total number of threads is -XX:ActiveProcessorCount * 2 (i.e. event loop count + loom fork join pool + GC threads + compiler threads) > -XX:ActiveProcessorCount
• On Spring assuming they uses virtual threads, tomcat fully run it, and they handle blocking calls there too, meaning that the total number of platform threads honor -XX:ActiveProcessorCount

This is not correct and should be fixed.

2 suggestions here:

1. constrain the core count for real e,g, using taskset on linux
2. practice active benchmarking to detect CPU usage beyong what's expected (e.g. by proving pidstat output vs the server)

I know that is a bit against the purpose of keep the stress test simple, but without both this can lead to very wrong and broken results.

[edeandrea]

constrain the core count for real e,g, using taskset on linux

We are doing that in the perf lab

[holly-cummins]

I know that is a bit against the purpose of keep the stress test simple, but without both this can lead to very wrong and broken results.
constrain the core count for real e,g, using taskset on linux

We are doing that in the perf lab

I think the issue here is for the naive scripts, not the perf lab ones. For the naive scripts, doing anything other than -XX:ActiveProcessorCount in a platform-independent way without increasing the complexity of the scripts is going to be tough. And as you say, for the perf lab scripts, there's not an issue, because taskset is already being used.

(Although, since taskset is being used, we might want to drop -XX:ActiveProcessorCount from the arguments in the perf lab scripts? It may be we've already done that, since I can see ActiveProcessorCount in recent output json but I can't see it in the repo source, so I'm not sure if it's currently being used. The most recent output json is from the 19th, so it might not match the current repo contents.)

[franz1981]

we might want to drop -XX:ActiveProcessorCount from the arguments in the perf lab scripts?

+100 @holly-cummins

[edeandrea]

With the new isolation scripts we aren't using -XX:ActiveProcessorCount anymore (in the perf lab that is). The user can still specify it on their own if they wish.

[edeandrea]

Closing this because we aren't doing this anymore.

[franz1981]

@edeandrea this one wasn't for the lab scripts but for the Holly ones eh 🙏