Add proxy gateways #1171
Add proxy gateways #1171
Conversation
There was a problem hiding this comment.
We need to add a testvase for this feature
Given that the separation of the role is a worse idea than i initially thought,
How about introducing a --px ... option to pass the xspec of a proxy
The naming is something where a little bikeshedding may be useful to avoid grief down the line
Thanks for getting this started
Proxy gateways do not run workers, and are meant to be passed with the `via` attribute to additional gateways. They are useful for running multiple workers on remote machines. Example usage: ``` pytest -sv --dist=load --px "id=my_proxy//socket=IP:PORT" --tx "5*popen//via=my_proxy" ``` Proxy gateways do not run workers, anda re meant to be passed
Add docs for using proxy gateways to run multiple workers on remote machines
Alon-L
force-pushed
the
feature/proxy-gateways
branch
from
315f040 to
333e2d3
Compare
for more information, see https://pre-commit.ci
| @@ -57,8 +57,17 @@ def __init__( | |||
| if self.testrunuid is None: | |||
| self.testrunuid = uuid.uuid4().hex | |||
| self.group = execnet.Group(execmodel="main_thread_only") | |||
| for proxy_spec in self._getpxspecs(): | |||
There was a problem hiding this comment.
I need to crosscheck in execnet on how proxy startup is managed
I vaguely recall that main thread only may cause trouble when more than one worker is behind a proxy
There was a problem hiding this comment.
I tested locally using 5 workers behind a local socket proxy, and it worked.
There was a problem hiding this comment.
I realized my mistake
Proxy doesn't get thr default value of main thread only as its using a own spec
Thanks for validation
|
@RonnyPfannschmidt how can I write a test that uses a remote server? Is there some predefined server available during the tests so I can test the proxy gateway's functionality? EDIT: I see that the test |
|
These are commonly passed locally As it's arbitrary code execution we haven't created a ci strategy Testing against a popen gateway may suffice |
The test `test_proxy_gateway_setup_nodes` makes sure that 2 execnet gateways are created, but only one node is allocated. The test `test_proxy_gateway` tries to run a test on a proxy.
for more information, see https://pre-commit.ci
|
Is there an ETA for the merge and release of this feature? I'm not really familiar with the project's release schedule. |
|
There's no eta but i intend to merge/ release this week I was knocked out by a cold today |
|
Alright. Get well soon. |
This adds the feature requested in issue #1170 .
Add the
proxyattribute to the xspec in order to create proxy gateways.Proxy gateways do not run tests, and aren't considered as additional workers.
Proxy gateways must be named (must have an
idattribute), and need to be passed using theviakeyword to additional gateways.Example: