[BUG] typer fails for functions with *args

[arogozhnikov]

Describe the bug

typer fails for functions with *args in signature (variable number of arguments)

To Reproduce

• Create a file main.py with:

import typer

app = typer.Typer()


@app.command()
def hello(*names: str):
    typer.echo(f"Hello {names}")


if __name__ == "__main__":
    app()

• Call it with:

python main.py alpha

• It outputs:

Traceback (most recent call last):
  File "test.py", line 12, in <module>
    app()
  File "/Users/axelr/system1/envs/pipeline/lib/python3.6/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/Users/axelr/system1/envs/pipeline/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/axelr/system1/envs/pipeline/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/axelr/system1/envs/pipeline/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/axelr/system1/envs/pipeline/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/axelr/system1/envs/pipeline/lib/python3.6/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
TypeError: hello() got an unexpected keyword argument 'names'

• But I expected it to output:

Hello ['alpha']

Expected behavior

Variable number of arguments should be parsed as e.g done by ls, rm, cp and many other programs

Environment

• OS: macOS
• Typer Version: 0.3.2
• Python version 3.6

[aaditkamat]

@tiangolo, should this be a bug or a feature request? I would like to work on it if it's something that's being considered as an enhancement.

[alextremblay]

what happens if you change the type annotation from str to list?

your type annotation is telling python / typer that names will be a string, but in reality, names is a list

I don't know if correcting the type annotation will fix the issue or not, but it's at least worth a shot

[daddycocoaman]

Not sure this is a bug since this isn't really supported by Typer docs as a way to get multiple values. If you wanted multiple values in a list, you would have to follow these guidelines this way.

import typer
from typing import List

app = typer.Typer()


@app.command()
def hello(names: List[str]):
    typer.echo(f"Hello {names}")


if __name__ == "__main__":
    app()

I think using unpacking for arguments would break the fundamental features of Typer, which is clearly stating the expecting types.

[arogozhnikov]

@daddycocoaman seems to work.

However passing List[x] not just x as type hint contradicts the way it is defined in python PEPs. Mypy/pycharm/others should raise warnings on this code, see details here: https://stackoverflow.com/questions/37031928/type-annotations-for-args-and-kwargs

[daddycocoaman]

@arogozhnikov That's not the case at all. Generics like List are explicitly mentioned in accordance with PEP-484

[arogozhnikov]

@daddycocoaman

Correct example is given in your link, see section "Arbitrary argument lists and default argument values".

*args can't be anything but list - it is senseless to specify that is it a list.

[daddycocoaman]

FYI, in a normal non-typed function, *args gets seen as a tuple of unknown length until defined, not a list. With typing like in your example, it gets recognized as typing.Tuple[T], where T here is str.

Typer supports pretty clear Tuple and List classes, but in order for Typer to support an *args syntax, it would probably have to change the way Typer fundamentally works, which is by grabbing the type hints as seen by the typing module.

If you run get_type_hints on your example, you get {'names': <class 'str'>} because typing has no idea that your argument is packed, therefore Typer has no idea. It just gets the names and type. But if you check the signature of the function, you'd get (*names: str), which is a string representation of the parameters.

So the change that would have to be made is that Typer would have to compare the parameter signatures to the get_type_hints for some VERY loose comparisons, parse the signature to determine if the Parameter starts with "*" (as names does here), and then somehow consider that to be Tuple[T, ...] (sorta like with typing.cast()), but that ultimately would be a misrepresentation of what the user explicitly defined in code. 🤷🏾‍♂️

[arogozhnikov]

Thanks for a good summary.

*args gets seen as a tuple

You're correct, that's a tuple, thanks for correction.

I am surprised it is not represented the right way by typing module (I would expect it to provide a function to auto-resolve this, but didn't find one).

After your comment I understand source of this behavior in typer, but I consider it to be wrong, as it contradicts PEP examples. Mypy and pycharm seem to take care of *args typing the right way (i.e. they are PEP-compliant).

Annotations like

def f(*args: List[int])

are treated as Tuple[List[int]] in this convention and raise complaints about the code in function body.

[alicederyn]

I believe you can fairly easily determine whether a given argument is of the form *args:

>>> inspect.signature(my_func).parameters['args'].kind
<_ParameterKind.VAR_POSITIONAL: 2>

You will unfortunately need to do more than just change the type from X to List[X] though: these arguments must be passed in as unnamed positional parameters, not via the "name" of the argument (args in this case). This is why the error got an unexpected keyword argument 'names' is in the original bug description.

---

I don't know what @arogozhnikov had in mind with this request, but I'm trying to make a subcommand that can pass arbitrary arguments to another shell command, after removing any initial options, much like e.g. docker run and its ilk. This is subtly different from how Typer treats List parameters, as anything that looks like an option but which comes after the first non-option should be passed through to the list. For example, given

def main(*args: str, myparam: bool = False):
   ...

The command python myscript --myparam foo bar would pass in ['foo', 'bar'] to args and True to myparam, but python myscript foo --myparam bar would pass in ['foo', '--myparam', 'bar'] to args and leave myparam as False.

Is that possible to support with Click (and hence to hook up in Typer for *args)?

Edit: I hope this hasn't come across as demanding this feature — wanted to know if it was even possible before talking about whether it might be a thing you'd be interested in adding.

[Bonnevie]

@alicederyn did you find a way to forward arguments/options? Trying to solve the same problem, and I was originally thinking I could just do something like

def forwarder(name: str, forwards: str):
    ...

and have the forwards sweep up everything to the right, but Typer (correctly) assumes whitespace delimits arguments and fails. Using a list sort of works, if you recombine the collected values in the list (and you need to restore quotes and similar symbols)

def forwarder(name: str, forwards: List[str]):
   forwards = recombine(forwards)
   ...

This still fails on options though since Typer (correctly) tries to parse them. In my use-case I only have a limited number of options, so I can just forward them manually. Would love to hear if there are more elegant ways to solve the task.

[alicederyn]

@Bonnevie no, I gave up trying to migrate from docopt after hitting this issue, it wasn't worth the investment for the tool I was looking at

[alexreg]

I tend to agree with @daddycocoaman that this is no bug, and quite reasonable behaviour. It looks like support could be added for *args, but I'm not sure that it is either desirable or others anything over an argument of type List[str].