Better subprocess integration

[dwt]

pipe Will try, but it took me days just to answer to your comments :/ For now I am just attaching my experient (pipe.py.txt).

Regarding sh: It looks quite interesting, and most of my scripting is Unix anyway (or WSL if on Windows). It is however somewhat inconvenient to use with fluentpy.

_(range(10)).map(lambda it: str(it)+"\n").call(lambda it: sh.sed("s/^/>> /", _in=it)).to(list)
                                                      ^^                         ^^

In scripting contexts, something like

_(strings).sh.sed("s/^/>> /").to(list)

would be preferable. As it is, that would make sh a thing wrapper to the sh module, that sets _in. And maybe for consistency some ish, that also sets _iter=True...

The downside would be, that it would introduce a feature that doesn't work on Windows. My popen based experiment is platform independent by contrast. Will look into it more.

Originally posted by @kbauer in https://github.com/dwt/fluent/issues/6#issuecomment-792307743

[dwt]

Thinking about sh a bit more, it seems to me that this already works surprisingly well:

import fluentpy as _
sh = _.lib.sh

sh.ls('-1').call(sh.sort.bake('-r')).print()

strings = ['foo', 'bar', 'baz']
_(strings).map(sh.sed.curry("s/^/>> /", _in=_)._).print()

_(range(10)).map(str).map(sh.sed.curry("s/^/>> /", _in=_)._).print()

A better integration is certainly possible, but it's also not too bad with existing possibilities from fluentpy. What do you think?

[dwt]

@kbauer: Did you get this issue? I'd like to your input here. :-)

[kbauer]

@dwt Testing proves illusive. Trying to get WSL 2 running has twice now made my Windows installation first incapable of connecting to networks, this time incapable of connecting to displays. If recovery works, I might yet give feedback today :/

[kbauer]

In your example, note that sh.sed.curry(...) would for normal import sh be expected to be equivalent to sh.sed("curry", ...), so that's inviting inconsistency / unexpected behavior.

Is it intended to call sed for each string separately? That would create extremely poor performance. I'd do instead:

>>> _(strings).map(each + "\n").call(lib.sh.sed.curry("s/^/>> /", _in=_)._).to(list)
['>> foo\n', '>> bar\n', '>> ,baz\n']

It also feels awkward to manually pass the _in argument. When I tried to reproduce the example, I forgot the trailing _ inside the map(), and got

>>> _(strings).map(lib.sh.sed.curry("s/^/>> /", _in=_)).to(list)
[fluentpy.wrap(>> foo), fluentpy.wrap(>> bar), fluentpy.wrap(>> ,baz)]

which is another usability issue. Compare that to some theoretical

>>> _(strings).sh("sed", "s/^/>> /").to(list)

sorry somewhat stressed

[dwt]

Hm, I've been thinking about this, and there is definitely a way to get an easy integration going.

A simple way would be to wrap sh so it plays nicely with input from chains:

import typing

import fluentpy as _

def pipe(command, *args, **kwargs):
    import sh
    
    def _prepare_stdin(stdin):
        if isinstance(stdin, (typing.Text, sh.RunningCommand)):
            return stdin  # use as is
        elif isinstance(stdin, typing.Iterable):
            return _(stdin).map(str).join('\n')._
        else:
            return str(stdin)  # just assume the caller wants to process it as string
    
    def wrapper(stdin):
        return getattr(sh, command)(*args, **kwargs, _in=_prepare_stdin(stdin))
    
    return wrapper

_(range(10)).call(pipe('sed', 's/^/>> /')).call(pipe('sort', '-r')).print()

That is not quite the nice syntax that you get from sh, but it is pretty functional.

If the syntax / usability from sh is important, that could be achieved too:

import typing
import sh

import fluentpy as _

class SHWrapper(object):
    def __getattr__(self, command):
        
        def _prepare_stdin(stdin):
            if isinstance(stdin, (typing.Text, sh.RunningCommand)):
                return stdin  # use immediately
            elif isinstance(stdin, typing.Iterable):
                return _(stdin).map(str).join('\n')._
            else:
                return str(stdin)  # just assume the caller wants to process it as string
        
        def command_wrapper(*args, **kwargs):
            def command_with_arguments_wrapper(stdin):
                return getattr(sh, command)(*args, **kwargs, _in=_prepare_stdin(stdin))
            return command_with_arguments_wrapper
        
        return command_wrapper

pipe = SHWrapper()

_(range(10)).call(pipe.sed('s/^/>> /')).call(pipe.sort('-r')).print()

I am not sure this is worth it, but it definitely is possible.

What do you think?