CWL: missing documentation on conda dependency resolver

[mr-c]

--beta-conda-dependencies and related command line options need documentation (copied from https://github.com/common-workflow-language/cwltool#leveraging-softwarerequirements-beta ? )

Originally posted by @mr-c in https://github.com/DataBiosphere/toil/issues/3803#issuecomment-927579381

┆Issue is synchronized with this Jira Story ┆friendlyId: TOIL-1037

[adamnovak]

These options are added from inside cwltool's code, and cwltool is providing the help text, right?

Can we just make our docs link to the cwltool docs for the options we get from it?

[adamnovak]

@mr-c Does cwltool have docs for these options that somehow aren't making it into Toil's option parser to be printed?

Or is this to put them in the Toil ReadTheDocs docs somewhere?

[mr-c]

These options are added from inside cwltool's code, and cwltool is providing the help text, right?

Nope, none of the cwltool argparser is used for toil-cwl-runner:

https://github.com/DataBiosphere/toil/blob/04e13e99a371065af081aaad0c7f590a5c7a76b5/src/toil/cwl/cwltoil.py#LL3243C28-L3243C32

I guess https://github.com/common-workflow-language/cwltool/blob/main/cwltool/argparser.py could be refactored so that bundles of options can be re-used..

Can we just make our docs link to the cwltool docs for the options we get from it?

Hmm.. the options should be rendered at https://cwltool.readthedocs.io/en/latest/ but that isn't working right now

Either way there needs to be more documentation with more examples; and specifically for toil-cwl-runner I'd like to see a tested example using module load .. with a local mapping file between tool names and local modules names

https://docs.galaxyproject.org/en/latest/admin/dependency_resolvers.html#lmod-dependency-resolver https://docs.galaxyproject.org/en/latest/admin/dependency_resolvers.html#environment-modules-dependency-resolver

[unito-bot]

➤ Adam Novak commented:

So someone has to copy the help text for these duplicated options from cwltool into Toil, and also make sure to link from the Toil docs to where the cwltool docs are or should be.

As for the HPC module support, are we sure that actually is working in Toil with common module systems and schedulers, and the cwltool implementation? We don’t actually have a module system over here, so we aren’t in a position to write a tutorial for it or to test/maintain it, but if all the guts of it are in cwltool itself we can make sure to expose the options in Toil.