Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full cpu run in parallel raises exception #278

Open
Landau1908 opened this issue Nov 15, 2024 · 1 comment
Open

Full cpu run in parallel raises exception #278

Landau1908 opened this issue Nov 15, 2024 · 1 comment

Comments

@Landau1908
Copy link

Hi,
To get the maximum speedup, I used full cpu (multiprocessing.cpu_count()=64) to performing increasing popsize (IPOP). During which an exception is raised(see below codes). Does the number_of_processes have to be set to es.popsize + 1 like the official example provided with EvalParallel2(elli, es.popsize + 1) as eval_all ?

Exception in thread Thread-7 (_handle_workers):
Traceback (most recent call last):
  File "d:\Users\fengyc\anaconda3\Lib\threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "d:\Users\fengyc\anaconda3\Lib\threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "d:\Users\fengyc\anaconda3\Lib\multiprocessing\pool.py", line 522, in _handle_workers
    cls._wait_for_updates(current_sentinels, change_notifier)
  File "d:\Users\fengyc\anaconda3\Lib\multiprocessing\pool.py", line 502, in _wait_for_updates
    wait(sentinels, timeout=timeout)
  File "d:\Users\fengyc\anaconda3\Lib\multiprocessing\connection.py", line 1084, in wait
    ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "d:\Users\fengyc\anaconda3\Lib\multiprocessing\connection.py", line 1016, in _exhaustive_wait
    res = _winapi.WaitForMultipleObjects(L, False, timeout)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: need at most 63 handles, got a sequence of length 66

Regards.

@nikohansen
Copy link
Contributor

Does the number_of_processes have to be set to es.popsize + 1 like the official example provided with EvalParallel2(elli, es.popsize + 1) as eval_all ?

No. It seems that the error suggests that the multiprocessing.cpu_count was not set to 64.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants