Don't suggest to start build with infinite number of threads

[Delgan]

Hi.

Just a small enhancement proposal, as I don't think calling make with unbounded number of threads is desirable.

Using make -j will start "infinite" number of jobs according to make docs, which slows down the build and consumes too many resources.

Suggesting make -j4 seems to be a sane default, advanced users will know how to adjust it.

---

📚 Documentation preview 📚: https://cpython-devguide--1087.org.readthedocs.build/

[hugovk]

Thanks for the suggestion.

What machine are you using? Does the build take longer with make -j than make -j4, or otherwise slow the machine down? "consumes too many resources" - in what way?

On macOS with 8 CPUs, make -j uses them all and I don't notice any other slowdown. Having said that, it takes about the same time to run as make -j4, around 32s.

[Delgan]

Hi @hugovk.

It's not so much related to my machine in particular. It's just that once all the CPU cores are used for compiling, adding more jobs generally do not improve the performance and rather tend to make it worst because of the overhead of context switches ¹. It's not very much noticeable in practice, though ²³.

However, what is growing continuously is the RAM consumption. Each started job will consume memory, which causes a noticeable increase in overall RAM usage (finally "wasted" since an infinite number of jobs cannot run in parallel). This is not really desirable, and this is frequently related to out-of-memory problems for a beginner who is not familiar with make building process ⁴.

Also I suggested -j4 but maybe -j2 is preferable as it matches the example given when clicking "more detailed instructions" on the page.

Footnotes

1. Disadvantages to high make job values ↩

2. How to speed up compilation time in linux ↩

3. GNU make: should the number of jobs equal the number of CPU cores in a system? ↩

4. Is make -j (with no argument) dangerous? ↩

[terryjreedy]

For the test suite, -j starts a number of parallel tests appropriate to the CPU. Since, you say, make -j does not do that, why not recommend -j# where # is appropriate for the machine?

[Delgan]

@terryjreedy On the same page, tests are suggested to be run with ./python -m test -j3, details about the -j argument are only provided on the linked page. I suppose it's best to show a simple and functional example by default, and only go into detail on the appropriate page. It's easier to copy and paste. But in the end, it's up to what you prefer.