TPV dry-run tool

[mira-miracoli]

We had the idea to make a small CLI TPV tool, that takes your config files, a few input params like

$ tpv-dry -d /.../config/tpv tool=metaspades [email protected] size=2G

and then mocks a Galaxy instance, like in the unit tests, calculates the destination and gives a easy-to-read output, which destination was chosen and why. eg:

RESULT for .....
--------------------------------------------------------------------------------------
DESTINATION
final: condor_tpv
mached(with ranks): condor_tpv(1), singularity(2), interactive(3)
--------------------------------------------------------------------------------------
PARAMS
cores: 8 (min of tool: 10, user: 8, role: 9)
mem: ....
...

Additionally an explanation of how the rank was calculated in this specific case, would be helpful to debug errors in the configuration and check if it works as intentioned.
EU will work on this, but all help and suggestions are appreciated :)

[cat-bro]

This is a great idea. It would be very useful.

[Slugger70]

I agree with @cat-bro - this is a really good idea @mira-miracoli

[nuwang]

Great idea @mira-miracoli! Just a few random thoughts and suggestions that crossed my mind.

• This could probably be added as a shell command like tpv lint and tpv format into this file:

  total-perspective-vortex/tpv/core/shell.py

  Line 62 in 2d7321f

  lint_parser = subparsers.add_parser(

  instead of being a separate command. It could then be invoked as tpv dry-run -c /path/to/galaxy.config -f path/to/tpv/config

• To get a reasonably accurate idea of how the actual destination will be determined, it'll be necessary to use the actual job_conf.yml that the Galaxy instance uses, and possibly galaxy.yml as well. Your suggestion of trying out the mock test path first sounds very sensible - that may well be good enough to get a reasonably accurate estimate of where the job will go and should be very fast.

• However, should there be problems with that, as in, it's not accurate enough as it doesn't take the entirety of Galaxy's configuration into account, perhaps the IntegrationTest path might work better:

  total-perspective-vortex/tests/test_mapper_resubmit.py

  Line 6 in 2d7321f

  class TestMapperResubmission(IntegrationTestCase):

Only the resubmit test case uses it at the moment, but since it is using Galaxy's own IntegrationTestCase class, the main advantage would be that it'll spin up a fairly comprehensive Galaxy instance, if we provide the galaxy.yml and job_conf.xml to it. Of course, it'll be substantially slower so the mock option sounds like the better starting point.

[natefoo]

Whoops, missed this issue, see #52.

It's missing some things let arbitrary specification of tool params or input sizes, you can basically only specify a tool ID and a username/email currently, but it's a start.

[mira-miracoli]

Thank you @natefoo ☺️

[nuwang]

Closed by: #52

Have moved the suggestion for providing an explanation, as an enhancement here: #55