Update pytest to dynamically assign max processes for test runs #1393
Description
pytest docs: CPU parallelization
Right now pytest uses the -n auto flag that directs pytest-xdist to use as many processes as the computer running the tests has physical CPU cores, up to the limit set by --maxprocesses. The maximum number of processes is currently set to specific values based on our experience of the CI machine's performance.
We could likely increase throughput by switching to the -n logical flag which would use the max number of logical CPU cores, however, before we do that we need to dynamically adjust the --maxprocesses cap based on the system that is running the tests. To do that, we will have to figure out how to determine the available memory on the system and properly adjust to ensure we don't run out.
We should also investigate using the --dist worksteal flag for impact on test times which could handle re-assigning jobs to workers who have completed their test queue more efficiently.