Feature/rp integration

[andre-merzky]

Description

Hi Daniel, thanks for your support with this RCT adaptor. It now represents a state where some of the tests pass, and where it makes sense to do the func/exec conversion for the workload.

Todos

Notable points that this PR has either accomplished or will accomplish.

• [ ] some TODOs are noted in the adaptor code
• [ ] radical.pilot needs to be added as deployment dependency - not sure where to do that. Same as fireworks or as dask (which seem to be handled differently?)

Questions

Alas, I lost the set of commands needed to run the unit tests. I do something like:

$ source ~/miniconda/etc/profile.d/conda.sh
$ conda activate radical
$ pg_ctl -D /tmp/pg_data -l logfile start
$ pytest -vsk test_adapter

That seems to collect the tests, but does not run any of them and just hangs. Any idea what step I am missing?

Status

• [ ] requires completion, unit testing, integration testing

[lgtm-com[bot]]

This pull request introduces 4 alerts and fixes 4 when merging 54299c2da76fcfea869f302e5d1e09a8544d6285 into 0fc1a4aebc1dbb9067d5eaf6b7809c7e9cbcdfba - view on LGTM.com

new alerts:

• 4 for Unused import

fixed alerts:

• 4 for Module is imported more than once

[dgasmith]

Thanks for making the PR! A few notes:

• radical.pilot will not be a deployment dependency as users will choose the workload manager to use.
• The strategy we have often used is to use a function-local import rather than a file global one. As those functions can only be reached if an object of the import is provided it is a pretty safe way to organize things.

[andre-merzky]

• The strategy we have often used is to use a function-local import rather than a file global one. As those functions can only be reached if an object of the import is provided it is a pretty safe way to organize things.

Yep, that makes sense of course. I only mentioned that because of the travis failures, but I guess you add RP module install in the travis directives then?

[bennybp]

With v0.50, this feature is not applicable anymore. We have moved to a Parsl-only compute manager, although different kinds of managers are still possible in the future.