Additional features for @param.output

[jlstevens]

One of the major goals of param is to apply type validation and it does this well for Python literals. When it comes to callable objects, param is very lenient by assuming that the return value of the callable is of the right type. This is why you can use a imagen.PatternGenerator as the value of a param.Number for instance, even though the output of the PatternGenerator is incompatible.

For param.Number the line responsible for this hole in validation is here as self._check_value is skipped for callables. This situation could be improved by introducing support for a type annotation declaring the expected type of a method.

The API is still being decided, but perhaps we could have something like:

class Foo(param.Parameterized):

   @param.output(np.ndarray)
   def bar(...):
        return baz # Expected to be an ndarray

Then you could query for the decorated types:

>>> Foo.param_output_types()
{'bar':np.ndarray}

API questions

1. What arguments does @param.output have? Setting the type as a mandatory first argument seems to make sense (every method has a single return value).
2. Should the types be specified as classes/constructors (e.g str,np.ndarray) or parameters param.String, param.Integer or both?
3. Could this specify composite types? e.g a return value of (int, str) or maybe multiple possible types? (int, str) | None?
4. Should param have a flag to disallow unannotated callables from matching? This flag would be disabled by default for backwards compatibility.
5. I think it might be worth being able to express the 'self type' somehow. This type matches that of the parameterized class itself...

These issues are worth thinking about even if the initial approach starts simple. It might also be worth looking at the type annotation syntax standardised in Python 3....

[jbednar]

1. I would expect a docstring as another argument, explaining what the output represents, just as Parameters have docstrings.

2. Not sure. Try it out in some cases to get a feel for that.

3. A composite type is just a type, right? I.e. the type in that case is "tuple", and then there are further restrictions on the types of contained object. This argues for types specified like Parameters, where one can have "param.List(class_=...)", though we don't currently have any such composite parameter type with heterogenous contained types. I'd argue against supporting multiple possible types, as that's bad practice anyway, apart from the special case of supporting None (which would be the usual allow_None keyword, I guess).

4. Let's decide that later, after we see how well this works. Sure, potentially, at some point.

5. Why would you need to express the self type explicitly? When would you want anything other than what is already inspectable through the object or class?

It's interesting to think about how this relates to Python 3 type hinting, but there they seem to focus only on static declarations, not run-time checking as we propose here, so it doesn't seem like we could make use of what's currently available. Plus of course we want to support both 2 and 3...

[jlstevens]

1 - Agreed. Docstrings are a key part of what param offers ! 3 - None should definitely be supported (allow_None is sufficient, as you say). Happy to ignore more complex composite types for now. 5 - For instance, HoloViews elements return self from their __call__ method. This method is implemented in a base class so specifying the type as hs.Element is a bit too general. It would be good to know the return is hv.Image for hv.Image, hv.Curve for hv.Curve etc without specifying it each time.

Not saying we should be a slave to any other convention! It might still be good to look at to see if there are any good ideas worth adopting...

[jbednar]

3 - should probably have been split into two bullet points:, 3a how composite types should be specified and 3b multiple possible return types can be specified. For 3b, we agree that None is sufficient as the only initially supported variant return type. For 3a we have no answer yet, and we can probably postpone addressing it at first (though it strongly affects bullet point 2).

5 - It seems like there are at least two different use cases for "self" -- one is just when one wants to supply a Parameterized object to another object, in which case the specific type of self is fine (as that can always be compared against some more general type that's the actual requirement of the downstream consumer). Here it sounds like you mean supporting a method that returns self, and you want to declare what type is returned by that method? I can't see how we'd have access to that information from a decorator on a superclass's method, but maybe that's possible.

BTW, if you're adding a decorator for specifying the output type and docstring for a method call, you might also want to add one specifying the argument types and docstrings for the method call (potentially even the constructor?).

[jlstevens]

I agree with what you said for 3 i.e postponing how composite types are specified and only supporting allow_None for now.

Here it sounds like you mean supporting a method that returns self, and you want to declare what type is returned by that method? I can't see how we'd have access to that information from a decorator on a superclass's method, but maybe that's possible.

This was the case I had in mind and I wasn't considering implementation details. It is a feature I would like and I don't know if it is possible!

BTW, if you're adding a decorator for specifying the output type and docstring for a method call, you might also want to add one specifying the argument types and docstrings for the method call (potentially even the constructor?).

That could also be a useful feature but I think an additional such decorator would be classified as a new issue/separate feature request.

[jbednar]

I agree that's a separate feature, but one that would probably become trivial once this one is implemented.

[jbednar]

Param now has an @output decorator, which covers some of these issues but probably not all of them. Should review the above and compare to the current implementation to see if some remaining tasks can be broken off into separate open issues.