Custom validator registration

[SylvainCorlay]

I think that it would be interesting to push upstream the abiility that was added in jupyter-incubator's traittypes: registration of custom validators (non-cross) with a chaining valid method.

from traitlets import HasTraits, TraitError
from traittypes import Array

def shape(*dimensions):
    def validator(trait, value):
        if value.shape != dimensions:
            raise TraitError('Expected an of shape %s and got and array with shape %s' % (dimensions, value.shape))
        else:
            return value
    return validator

class Foo(HasTraits):
    bar = Array(np.identity(2)).valid(shape(2, 2))
foo = Foo()

foo.bar = [1, 2]  # Should raise a TraitError

In the case of the numpy array trait type, there are many possibilities that would be arguably natural but would be very likely bloating the TraitType specialization if implemented in the class.

Examples:

• only accepting attributes with a certain shape
• only accepting values that have less than a certain number of elements
• bounds on the dimensionality of the array
• requirements on dtypes (be a subdtype of a certain type, or exactly a certain dtype)
• squeezing dimensions en length 1, as we do in bqplot

[SylvainCorlay]

PS @minrk I started working on traittypes again and I am migrating bqplot to it. It was my understanding that you were using numpy trait types for parallel computing things so you could be interested in the recent changes there.

[minrk]

Looks nice to me, though I would use .validate instead of .valid, since .valid is not a verb.

If this works, doesn't that mean that we should be able to do:

@bar.validate # or @bar.default
def _foo():

?

I use numpy a bunch in ipyparallel, but I've never used it with traits.

[SylvainCorlay]

• The problem is that validate is already a public function in TraitType .
• Indeed I can use the custom cross validation with @validate but I was in a situation in bqplot where I needed to have

# pseudo code
x = Array().valid(one-dimensional)
y = Array().valid(one or two dimensional, squeeze dimensions of length 1)
morphism_matrix = Array().valid(shape(2, 2))

these custom validators are not cross-validators in that they don't require the other attributes of the HasTraits class and tend to be declaratively specified for each attribute. I reuse the same sort of validators in many situations.

Example here:

https://github.com/bloomberg/bqplot/pull/305/files

[SylvainCorlay]

@minrk regarding the serialization, you are probably still using custom serialization routines to json + binary buffers, which could potentially be shared with ipywidgets.

I am still unsure if this will work with numpy datetime objects.

[SylvainCorlay]

If we introduce it in 5.0 then traittypes will need to remove their overload of validate calling the custom validator since the logic will likely be in TraitType._validate... This is an argument for including the logic now in traitlets, unless we don't pick the name valid.

What do you think of this @minrk?

[minrk]

No, we can't break usage of the current @validate name, we would need to use something else. Using a noun doesn't feel right to me, but I guess it's okay.

[SylvainCorlay]

Ah that is not what I meant.

[SylvainCorlay]

I mean't that traittype.Array.validate (the overloaded public method) uses the custom validators registered with valid.

If we eventually have a similar API in base TraitType, registering things with valid, the validators list would be called in the private _validate, and if this list has the same name as in the one of Array, if will be called twice.

[minrk]

I see. We'd need to figure out a way to adopt the feature upstream safely, or have traittypes detect whether its additions are redundant, and make it conditional.

[SylvainCorlay]

Yeah, actually, if traittypes.Array.validators become traitlets.TraitType._validators if should be sufficient.

[rmorshea]

@SylvainCorlay why store the validators on TraitType instances, and not on each HasTraits class?

Storing on TraitTypes seems a bit dangerous since anything that calls TraitType.valid will effect all subclasses of the one class that happens to own the trait.

[SylvainCorlay]

@SylvainCorlay why store the validators on TraitType instances, and not on each HasTraits class?

Storing on TraitTypes seems a bit dangerous since anything that calls TraitType.valid will effect all subclasses of the one class that happens to own the trait.

The custom validators don't have a reference on the TraitType instance. I don't think that it would make sense to make it a traittype instance thing.

[rmorshea]

@SylvainCorlay, I guess you could apply my argument to TraitType.tag, so my criticism isn't really valid. However, the issue I wanted to address with that comment was the extendability of these chained validators in subclasses.

Without the ability to extend these validation chains people may be tempted to tamper with TraitType._validators. So my suggestion is that we implement something like this:

class Parent(HasTraits):
    a = Array().valid(dtype(int)).tag(x=1)

class Child(Parent):
    a = Array(np.identity).valid(shape(2, 2), extend=True)

This would be equally applicable to tag...

class Parent(HasTraits):
    a = TraitType().tag(x=1)

class Child(Parent):
    a = TraitType().tag(y=2, extend=True)

... if it weren't for the difficulty of tag(**metadata).

[rmorshea]

Additionally, the name valid is very confusing.

I think has_valid would make it significantly more distinct.