Need a mechanism to use getargs for CmdAction(list)

[scr-oath]

trafficstars

Since CmdAction can take a str or List[str] and also callable that returns the action - can the callable also return a List[str] ? This would be helpful if you wanted to call something without a shell and still wanted to get or construct arguments from other tasks' output.

dodo.py

import logging
import shlex
from typing import List

import doit
from doit.action import CmdAction

DOIT_CONFIG = {
    'verbosity': 2,
    'default_tasks': ['check_results']
}


def task_check_results():
    def check_results(use_foo_bar, use_foo_bar_shlex):
        failed = False

        logging.warning('use_foo_bar = %s', use_foo_bar)
        if use_foo_bar != 'Used "foo bar"':
            failed = True

        logging.warning('use_foo_bar_shlex = %s', use_foo_bar_shlex)
        if use_foo_bar_shlex != 'Used "foo bar"':
            failed = True

        return not failed

    return {
        'actions': [check_results],
        'getargs': {
            'use_foo_bar': ('use_foo_bar', 'stdout'),
            'use_foo_bar_shlex': ('use_foo_bar_shlex', 'stdout'),
        }
    }


def task_echo_foo_bar():
    return {
        'actions': [CmdAction(['echo', '-n', 'foo bar'], shell=False, save_out='stdout')],
    }


def task_use_foo_bar():
    def use_foo_bar(foo_bar: str) -> List[str]:
        return ['echo', '-n', f'Used "{foo_bar}"']

    return {
        'actions': [CmdAction(use_foo_bar, save_out='stdout')],
        'getargs': {
            'foo_bar': ('echo_foo_bar', 'stdout'),
        },
        'uptodate': [False],
    }


def task_use_foo_bar_shlex():
    def use_foo_bar(foo_bar: str) -> str:
        return shlex.join(['echo', '-n', f'Used "{foo_bar}"'])

    return {
        'actions': [CmdAction(use_foo_bar, executable='bash', save_out='stdout')],
        'getargs': {
            'foo_bar': ('echo_foo_bar', 'stdout'),
        },
        'uptodate': [False],
    }


if __name__ == '__main__':
    doit.run(globals())

[scr-oath]

doit 
.  echo_foo_bar
foo bar.  use_foo_bar_shlex
Used "foo bar".  use_foo_bar

.  check_results
WARNING:root:use_foo_bar = 

WARNING:root:use_foo_bar_shlex = Used "foo bar"
TaskFailed - taskid:check_results
Python Task failed: '<function task_check_results.<locals>.check_results at 0x7fa6fa918dc0>' returned False

[scr-oath]

Hmm… related to #412 - I think, if you put shell=False, then you can return a list!

[scr-oath]

If this technique can be officially supported (either by fixing #412 or manually setting shell=False) then I think it would be good to document that you can do this!

[scr-oath]

Ok - so… for this one, given the response to #412 is to document, this one can provide this explicit behavior:

https://pydoit.org/tasks.html?highlight=cmdaction#cmd-action

For complex commands it is also possible to pass a callable that returns the command string. In this case you must explicit import CmdAction.

That can say returns the command string or list (note that you must choose the shell parameter explicitly to agree as it is not inferred by the CmdAction constructor)

[scr-oath]

You might even put it as a separate use case - i.e. how to template non-shell CmdActions… (i.e. receive variables from other tasks, and modify when returning a list - something apparently only possible in the single-string variant which does the interpolation)

obviously pick a better name for the functions in your example, but this works for me as expected:

def task_use_foo_bar_shell_false():
    def use_foo_bar(foo_bar: str) -> List[str]:
        return ['echo', '-n', f'Used "{foo_bar}"']

    return {
        'actions': [CmdAction(use_foo_bar, shell=False, save_out='stdout')],
        'getargs': {
            'foo_bar': ('echo_foo_bar', 'stdout'),
        },
        'uptodate': [False],
    }

[schettino72]

Thanks. sorry but i am a bit lost.

When you give an example, please make sure:

• it is minimal, include only what does not work as expected.
• no external libs, unless it is essential
• indicate with a comment which line is the issue 9if any)
• include expected and unexpected output.

[scr-oath]

Hey… sorry about that, I'm still a little foggy from a fever this holiday season…

I think what I'm encountering, as you can probably tell, is holes in the documentation that I kinda figure out for myself as I write up the feature request or problem report.

This feature request was for a way to use the results from one task in the arguments of another task's action(s).

Apparently, the only way to do that is to use the CmdAction with a function that returns a list, but the documentation only says the function can return a string:

Please just consider this bug a documentation request to describe:

How one can return a list from a function passed to CmdAction as long as they set shell=False and that one useful use-case is when they wish to use or modify values from other tasks in their actions.

This use case was not easy to figure out without exploring how doit worked, and would be nice if there was a documentation stanza to spell it out.

[scr-oath]

Thanks. sorry but i am a bit lost.

When you give an example, please make sure:

• it is minimal, include only what does not work as expected.

I'm confused by this - this is a feature request; aren't examples of how it could work or does work (and could be documented) helpful to the cause?

[scr-oath]

Maybe (just thinking out loud) the template for Feature Request could make that more clear - I did choose Feature Request from the list of options, so wasn't reporting a "problem" / "issue" / "bug"

[scr-oath]

Maybe we could sync on slack or whatever after the holidays. We’re considering using doit for a big project that currently has lots of disjoint / untested bash code that I’d like to move to python. I would like to ensure that newcomers can come up to speed with minimal confusion and maybe at some point we can discuss test ability (beyond just the python classes)

I hope you have a wonderful holiday season!

[schettino72]

Maybe we could sync on slack or whatever

Sorry. I have very limited "volunteer time" for open source. So I dont do slack or chat.

If you are interested I do offer paid service/support for doit.