typed-argument-parser
typed-argument-parser copied to clipboard
feature request: function decorator
Hi!
tap is a great argument parser, thank you for making it available!
I'm writing a small program that would be useful both as a function and a CLI tool. For this use case, I think a great addition would be a function decorator that does something like this:
''' toy example for the suggested @tap_function decorator'''
@tap_function
def greeter(
greetee: str = 'earthling' # How to address greetee
):
'''greeting someone'''
print(f'Greetings, {greetee}!')
- check
__name__ == '__main__'
in the decorated function's scope if it's not, returngreeter
(this could be just the default condition) - constructs a
Function_Tap(Tap)
so that it hasgreeter
's args and kwargs as - run
kwargs = FunctionTap().parse_args()
- run greeter(**kwargs)
Of course it's possible to do this by hand, but it would repetitive.
It would also be possible to subclass Tap
and have the code that does the actual work inside that, but that seems weird.
In addition this feels more intuitive to me - though I have to say here that I write quite some python and use type annotations in my day to day work but have not done much with argparse or the like.
a CLI program also behaves like a function: you call it with some arguments, it does it's thing and returns. Classes you instantiate, manipulate, subclass... Not really meaningful for CLI programs.
Would this be in tap's scope?
I think you can give the example code with Tap
subclass which is equal to your above code, so we can talk about details more easily. How do you think about it ? @ari-s
Hi @ari-s,
Yes, this is in Tap's scope! There are some great existing projects working on this (e.g., python-fire). This would involve a significant effort and we're currently focusing on getting the core Tap implementation to be more robust. We'd appreciate any PRs and hope to get to it in the future otherwise.
Thanks, Kyle and Jesse
Hi. I have implemented a toy version of FunctionTap
with minimum modifications.
import functools
import inspect
import re
import sys
from typing import Any, Callable, Dict
from tap import Tap
from tap.utils import type_to_str
# Sphinx docstring parser adopt from https://github.com/openstack/rally/blob/master/rally/common/plugin/info.py
# Licensed under the Apache License
PARAM_OR_RETURNS_REGEX = re.compile(":(?:param|returns)")
RETURNS_REGEX = re.compile(":returns: (?P<doc>.*)", re.S)
PARAM_REGEX = re.compile(":param (?P<name>[\*\w]+): (?P<doc>.*?)"
"(?:(?=:param)|(?=:return)|(?=:raises)|\Z)", re.S)
def trim(docstring):
"""trim function from PEP-257"""
if not docstring:
return ""
# Convert tabs to spaces (following the normal Python rules)
# and split into a list of lines:
lines = docstring.expandtabs().splitlines()
# Determine minimum indentation (first line doesn't count):
indent = sys.maxsize
for line in lines[1:]:
stripped = line.lstrip()
if stripped:
indent = min(indent, len(line) - len(stripped))
# Remove indentation (first line is special):
trimmed = [lines[0].strip()]
if indent < sys.maxsize:
for line in lines[1:]:
trimmed.append(line[indent:].rstrip())
# Strip off trailing and leading blank lines:
while trimmed and not trimmed[-1]:
trimmed.pop()
while trimmed and not trimmed[0]:
trimmed.pop(0)
# Current code/unittests expects a line return at
# end of multiline docstrings
# workaround expected behavior from unittests
if "\n" in docstring:
trimmed.append("")
# Return a single string:
return "\n".join(trimmed)
def reindent(string):
return "\n".join(l.strip() for l in string.strip().split("\n"))
def parse_docstring(docstring):
"""Parse the docstring into its components.
:returns: a dictionary of form
{
"short_description": ...,
"long_description": ...,
"params": [{"name": ..., "doc": ...}, ...],
"returns": ...
}
"""
short_description = long_description = returns = ""
params = []
if docstring:
docstring = trim(docstring)
lines = docstring.split("\n", 1)
short_description = lines[0]
if len(lines) > 1:
long_description = lines[1].strip()
params_returns_desc = None
match = PARAM_OR_RETURNS_REGEX.search(long_description)
if match:
long_desc_end = match.start()
params_returns_desc = long_description[long_desc_end:].strip()
long_description = long_description[:long_desc_end].rstrip()
if params_returns_desc:
params = [
{"name": name, "doc": trim(doc)}
for name, doc in PARAM_REGEX.findall(params_returns_desc)
]
match = RETURNS_REGEX.search(params_returns_desc)
if match:
returns = reindent(match.group("doc"))
return {
"short_description": short_description,
"long_description": long_description,
"params": params,
"returns": returns
}
class FunctionTap(Tap):
def __init__(self, func: Callable, *args, **kwargs):
self.func = func
super().__init__(*args, **kwargs)
def prepare_default(self, kwargs, parameter: inspect.Parameter):
# Get default if not specified
if parameter.default != parameter.empty:
kwargs['default'] = parameter.default
def prepare_help(self, kwargs, parameter: inspect.Parameter, comment: str = ''):
# Set help if necessary
if 'help' not in kwargs:
kwargs['help'] = '('
# Type
if parameter.annotation != inspect.Parameter.empty:
kwargs['help'] += type_to_str(parameter.annotation) + ', '
# Required/default
if kwargs.get('required', False):
kwargs['help'] += 'required'
else:
kwargs['help'] += f'default={kwargs.get("default", None)}'
kwargs['help'] += ')'
# Description
if comment:
kwargs['help'] += ' ' + comment
def as_dict(self) -> Dict[str, Any]:
"""Excluding the __dict__ keys introduced by FunctionTap"""
stored_dict = {key: value for key, value in super().as_dict().items() if key in self._annotations}
return stored_dict
def configure(self) -> None:
func = self.func
signature = inspect.signature(func)
docstring_components = parse_docstring(inspect.getdoc(func))
for idx, parameter in enumerate(signature.parameters.values()):
variable_name = parameter.name
kwargs = {}
name_or_flags = (f'--{variable_name}',)
# The preparation of some kwargs is highly coupled to the origin Tap context,
# so we handel it manually.
self.prepare_default(kwargs, parameter)
self.prepare_help(kwargs, parameter,
docstring_components['params'][idx]['doc'])
assert variable_name not in self.argument_buffer
self.argument_buffer[variable_name] = (name_or_flags, kwargs)
self._annotations[variable_name] = parameter.annotation
def tap_function(func):
if __name__ != '__main__':
return func
@functools.wraps(func)
def wrapper(*args, **kwargs):
tap = FunctionTap(func)
kwargs.update(tap.parse_args().as_dict())
return func(*args, **kwargs)
return wrapper
Use tap_function
decroator to rewrite this demo may like:
@tap_function
def build_model(
rnn: str,
hidden_size: int = 300,
dropout: float = 0.2,
):
"""summary
:param rnn: RNN type
:param hidden_size: Hidden size
:param dropout: Dropout probability
"""
print(rnn, hidden_size, dropout)
if __name__ == '__main__':
build_model()
It seem that some refactoring work should be done for FunctionTap
. Any ideas on it?
Hi @ari-s,
We have implemented a similar idea to your desired feature as of Tap
version 1.8.0: https://github.com/swansonk14/typed-argument-parser/releases/tag/v_1.8.0. We call it tapify
and while it's not a decorator, it does enable you to easily expose functions to the command line. Below is an example.
# square_function.py
from tap import tapify
def square(num: float) -> float:
"""Square a number.
:param num: The number to square.
"""
return num ** 2
if __name__ == '__main__':
squared = tapify(square)
print(f'The square of your number is {squared}.')
Running python square_function.py --num 5
prints The square of your number is 25.0
.
If you specifically want a decorator, you could easily create one using tapify
, as shown below.
# square_function.py
from tap import tapify
from typing import Callable
def tap_function(func: Callable) -> Callable:
if __name__ == '__main__':
return tapify(func)
@tap_function
def square(num: float) -> None:
"""Square a number.
:param num: The number to square.
"""
print(f'The square of your number is {num ** 2}.')
Please let us know if this solves your issue.
Best, Kyle