feature request: function decorator

[ari-s]

Hi!

tap is a great argument parser, thank you for making it available!

I'm writing a small program that would be useful both as a function and a CLI tool. For this use case, I think a great addition would be a function decorator that does something like this:

''' toy example for the suggested @tap_function decorator'''

@tap_function
def greeter(
    greetee: str = 'earthling'  # How to address greetee
    ):
    '''greeting someone'''
    print(f'Greetings, {greetee}!')

0. check __name__ == '__main__' in the decorated function's scope if it's not, return greeter (this could be just the default condition)
1. constructs a Function_Tap(Tap) so that it has greeter's args and kwargs as
2. run kwargs = FunctionTap().parse_args()
3. run greeter(**kwargs)

Of course it's possible to do this by hand, but it would repetitive. It would also be possible to subclass Tap and have the code that does the actual work inside that, but that seems weird. In addition this feels more intuitive to me - though I have to say here that I write quite some python and use type annotations in my day to day work but have not done much with argparse or the like. a CLI program also behaves like a function: you call it with some arguments, it does it's thing and returns. Classes you instantiate, manipulate, subclass... Not really meaningful for CLI programs.

Would this be in tap's scope?

[wj-Mcat]

I think you can give the example code with Tap subclass which is equal to your above code, so we can talk about details more easily. How do you think about it ? @ari-s

[martinjm97]

Hi @ari-s,

Yes, this is in Tap's scope! There are some great existing projects working on this (e.g., python-fire). This would involve a significant effort and we're currently focusing on getting the core Tap implementation to be more robust. We'd appreciate any PRs and hope to get to it in the future otherwise.

Thanks, Kyle and Jesse

[wlkz]

Hi. I have implemented a toy version of FunctionTap with minimum modifications.

import functools
import inspect
import re
import sys
from typing import Any, Callable, Dict

from tap import Tap
from tap.utils import type_to_str

# Sphinx docstring parser adopt from https://github.com/openstack/rally/blob/master/rally/common/plugin/info.py
# Licensed under the Apache License

PARAM_OR_RETURNS_REGEX = re.compile(":(?:param|returns)")
RETURNS_REGEX = re.compile(":returns: (?P<doc>.*)", re.S)
PARAM_REGEX = re.compile(":param (?P<name>[\*\w]+): (?P<doc>.*?)"
                         "(?:(?=:param)|(?=:return)|(?=:raises)|\Z)", re.S)


def trim(docstring):
    """trim function from PEP-257"""
    if not docstring:
        return ""
    # Convert tabs to spaces (following the normal Python rules)
    # and split into a list of lines:
    lines = docstring.expandtabs().splitlines()
    # Determine minimum indentation (first line doesn't count):
    indent = sys.maxsize
    for line in lines[1:]:
        stripped = line.lstrip()
        if stripped:
            indent = min(indent, len(line) - len(stripped))
    # Remove indentation (first line is special):
    trimmed = [lines[0].strip()]
    if indent < sys.maxsize:
        for line in lines[1:]:
            trimmed.append(line[indent:].rstrip())
    # Strip off trailing and leading blank lines:
    while trimmed and not trimmed[-1]:
        trimmed.pop()
    while trimmed and not trimmed[0]:
        trimmed.pop(0)

    # Current code/unittests expects a line return at
    # end of multiline docstrings
    # workaround expected behavior from unittests
    if "\n" in docstring:
        trimmed.append("")

    # Return a single string:
    return "\n".join(trimmed)


def reindent(string):
    return "\n".join(l.strip() for l in string.strip().split("\n"))


def parse_docstring(docstring):
    """Parse the docstring into its components.
    :returns: a dictionary of form
              {
                  "short_description": ...,
                  "long_description": ...,
                  "params": [{"name": ..., "doc": ...}, ...],
                  "returns": ...
              }
    """

    short_description = long_description = returns = ""
    params = []

    if docstring:
        docstring = trim(docstring)

        lines = docstring.split("\n", 1)
        short_description = lines[0]

        if len(lines) > 1:
            long_description = lines[1].strip()

            params_returns_desc = None

            match = PARAM_OR_RETURNS_REGEX.search(long_description)
            if match:
                long_desc_end = match.start()
                params_returns_desc = long_description[long_desc_end:].strip()
                long_description = long_description[:long_desc_end].rstrip()

            if params_returns_desc:
                params = [
                    {"name": name, "doc": trim(doc)}
                    for name, doc in PARAM_REGEX.findall(params_returns_desc)
                ]

                match = RETURNS_REGEX.search(params_returns_desc)
                if match:
                    returns = reindent(match.group("doc"))

    return {
        "short_description": short_description,
        "long_description": long_description,
        "params": params,
        "returns": returns
    }


class FunctionTap(Tap):
    def __init__(self, func: Callable, *args, **kwargs):
        self.func = func
        super().__init__(*args, **kwargs)

    def prepare_default(self, kwargs, parameter: inspect.Parameter):
        # Get default if not specified
        if parameter.default != parameter.empty:
            kwargs['default'] = parameter.default

    def prepare_help(self, kwargs, parameter: inspect.Parameter, comment: str = ''):
        # Set help if necessary
        if 'help' not in kwargs:
            kwargs['help'] = '('

            # Type
            if parameter.annotation != inspect.Parameter.empty:
                kwargs['help'] += type_to_str(parameter.annotation) + ', '

            # Required/default
            if kwargs.get('required', False):
                kwargs['help'] += 'required'
            else:
                kwargs['help'] += f'default={kwargs.get("default", None)}'

            kwargs['help'] += ')'

            # Description
            if comment:
                kwargs['help'] += ' ' + comment
    
    def as_dict(self) -> Dict[str, Any]:
        """Excluding the __dict__ keys introduced by FunctionTap"""
        stored_dict = {key: value for key, value in super().as_dict().items() if key in self._annotations}

        return stored_dict

    def configure(self) -> None:
        func = self.func

        signature = inspect.signature(func)
        docstring_components = parse_docstring(inspect.getdoc(func))

        for idx, parameter in enumerate(signature.parameters.values()):
            variable_name = parameter.name
            kwargs = {}
            name_or_flags = (f'--{variable_name}',)

            # The preparation of some kwargs is highly coupled to the origin Tap context,
            # so we handel it manually.
            self.prepare_default(kwargs, parameter)
            self.prepare_help(kwargs, parameter,
                              docstring_components['params'][idx]['doc'])

            assert variable_name not in self.argument_buffer
            self.argument_buffer[variable_name] = (name_or_flags, kwargs)
            self._annotations[variable_name] = parameter.annotation

def tap_function(func):
    if __name__ != '__main__':
        return func
    
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        tap = FunctionTap(func)
        kwargs.update(tap.parse_args().as_dict())
        return func(*args, **kwargs)
    return wrapper

Use tap_function decroator to rewrite this demo may like:

@tap_function
def build_model(
    rnn: str,
    hidden_size: int = 300,
    dropout: float = 0.2,
):
    """summary

    :param rnn: RNN type
    :param hidden_size: Hidden size
    :param dropout: Dropout probability
    """
    print(rnn, hidden_size, dropout)


if __name__ == '__main__':
    build_model()

It seem that some refactoring work should be done for FunctionTap. Any ideas on it?

[swansonk14]

Hi @ari-s,

We have implemented a similar idea to your desired feature as of Tap version 1.8.0: https://github.com/swansonk14/typed-argument-parser/releases/tag/v_1.8.0. We call it tapify and while it's not a decorator, it does enable you to easily expose functions to the command line. Below is an example.

# square_function.py
from tap import tapify

def square(num: float) -> float:
    """Square a number.

    :param num: The number to square.
    """
    return num ** 2

if __name__ == '__main__':
    squared = tapify(square)
    print(f'The square of your number is {squared}.')

Running python square_function.py --num 5 prints The square of your number is 25.0.

If you specifically want a decorator, you could easily create one using tapify, as shown below.

# square_function.py
from tap import tapify
from typing import Callable

def tap_function(func: Callable) -> Callable:
    if __name__ == '__main__':
        return tapify(func)

@tap_function
def square(num: float) -> None:
    """Square a number.

    :param num: The number to square.
    """
    print(f'The square of your number is {num ** 2}.')

Please let us know if this solves your issue.

Best, Kyle