Option to "partially override" an attribute from a parent class?

[pymarv]

Hi,

I'm part of a team that's in the early stages of migrating a large legacy app to Python. I'm pushing pretty hard for the team to adopt attrs as our standard way of doing OOP. I know the "attr way" is to avoid inheritance as much as possible, but that's not always possible, especially when you are part of a team with a project that likes to rely on inheritance (generally nothing too crazy, just simple stuff, but still - it's used a lot in the app and it's a good fit a lot of how the app works). One thing we do a lot is have base classes that device attributes that subclasses "further refine". With these, we want to be able to "extended" the attribute definition from the parent classes vs. totally overwrite it. It doesn't seem like this is possible today in attrs. For example:

import attr
import inspect

@attr.s
class One:
    myattrib = attr.ib(
        default=1,
        validator=attr.validators.instance_of(int),
        converter=int,
    )

@attr.s
class Two(One):
    myattrib = attr.ib(
        default=2
    )

print(inspect.getsource(One().__init__))
print(inspect.getsource(Two().__init__))

Output when run:

def __init__(self, myattrib=attr_dict['myattrib'].default):
    self.myattrib = __attr_converter_myattrib(myattrib)
    if _config._run_validators is True:
        __attr_validator_myattrib(self, __attr_myattrib, self.myattrib)

def __init__(self, myattrib=attr_dict['myattrib'].default):
    self.myattrib = myattrib

So if we want to extend the definition of myattrib, we would have to copy/paste the full definition vs. just modify what is different. What would people think about an option that says "take the full definition from the superclasses and just "merge in" this one extra aspect." Having to repeat the full definition across an inheritance hierarchy gets old (and ugly) pretty quick. :-)

Just by way of context on both this ticket and the other ticket I have opened (#573 - lazy=True option), the legacy app we are using is coming from Perl. Yes, I know everyone loves to hate Perl, :-) but hear me out. :-) The app uses Moose (https://metacpan.org/pod/Moose) and it's "lightweight cousin" Moo (https://metacpan.org/pod/Moo) and for all the bad things everyone wants to cast at Perl, the Moose/Moo "OOP framework" is really nice and has some great features. It has completely changed how OOP is done in Perl and took it from horrible to really nice (nobody has done OOP in Perl for 10+ years without using Moose.) (The creator of Moose, Stevan Little, spent lots of time studying and using lots of other OOP methodologies and was able to incorporate the "best of the best" ideas.) It's very similar to attrs in many ways, but there are a few things missing that are SOOO handy, useful, powerful, etc. :-)

In terms of how Moose would write the above example (and use the "+" to signify an "override"):

package One;
use Moo;
use Types::Standard 'Int';
has 'myattrib' => (
    is      => 'ro',
    default => 1,
    isa     => Int,
    coerce  => sub { int(shift) },  # Not required for str->int but to show how others work
);

package Two;
use Moo;
extends 'One';
has '+myattrib' => (                # <== Note the '+'
    default => 2,
);

Would something like an extend=True option to attrs be a nice addition to trigger similar behavior to the "+" shown above?

Thank you

[hynek]

I do know about Moose and it has been on my agenda to look for inspirations once I run out of own tasks…which weirdly hasn't happened yet. 🤪

---

Before changing APIs, you can access the field definitions while declaring fields. Which means that the following works:

@attr.s
class Two(One):
    myattrib = attr.ib(
        default=2,
        validator=attr.fields(One).myattrib.validator,
        converter=attr.fields(One).myattrib.converter,
    )

Typing aside, writing a helper that gives you something like evolve_attrib(attr.fields(One).myattrib, default=2) would be trivial and maybe there's more way to make it more ergonomic? The data is all there for you to use.

[wsanchez]

Another detailed description in #698, which is a duplicate.

[hynek]

So I guess I could be convinced to add something like

@attr.define
class Two(One):
    myattrib = attr.field_evolve(default=2)

That would also keep myattrib in the same order which would solve #707.

However:

• I find it hard to find the motivation to implement it.
• @euresti might correct me, but it sounds like a pretty big chunk of work in mypy? And so far they haven't even merged his NG API PR. :(((

[hynek]

Over at #829 we've come up with a somewhat clunky but magic-free approach of allowing to evolve Attributes (already in) and then convert them to fields/attr.ibs.

In your case that would look like this:

@attr.define
class Two(One):
    myattrib = attr.fields(One).myattrib.evolve(default=2).to_field()

I do realize it's a tad verbose, but it's a lot clearer with less indirection than what was proposed so far and would only require us to impolement to_field for Attribute classes.

[hkclark]

Great. Many thanks for the follow up! I'll check it out!

[YAmikep]

To confirm, the above does not exist in the current version yet, correct?

[hynek]

correct, it does not.

[petered]

The above would be great to have - not only for modifying fields of sub-classes but also for "de-duplicating defaults" - when passing arguments down through multiple levels, as in https://github.com/python-attrs/attrs/issues/876. Not sure how possible this is with regard to attrs internals, but from an external perspective it would be seem more intuitive to have the syntax:

@attrs
class Two(One):
    myattrib = One.myattrib.evolve(default=2)

[reubano]

Re #829 Here's what I'm doing...

from attr import define, field, fields, validators, make_class

@define
class One:
    myattrib: int = field(
        default=1,
        validator=validators.instance_of(int),
        converter=int,
    )

def create_subclass(name, base, **kwargs):
    def gen_fields(*args):
        for field in args:
            if field.name in kwargs:
                yield field.evolve(default=kwargs[field.name])
            else:
                yield field

    def reset_defaults(cls, fields):
        return list(gen_fields(*fields))

    return make_class(name, {}, bases=(base,), field_transformer=reset_defaults)


Two = create_subclass('Two', One, myattrib=2)

In [31]: fields(One)
Out[31]: (Attribute(name='myattrib', default=1, validator=<instance_of validator for type <class 'int'>>, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=<class 'int'>, converter=<class 'int'>, kw_only=False, inherited=False, on_setattr=None))

In [32]: fields(Two)
Out[32]: (Attribute(name='myattrib', default=2, validator=<instance_of validator for type <class 'int'>>, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=<class 'int'>, converter=<class 'int'>, kw_only=False, inherited=True, on_setattr=None))

[hynek]

a quick note to myself, that the implementation needs to take into account make_class (cf #1074).

[jamesmurphy-mc]

I've implemented evolve_field in one of my codebases by subclassing _CountingAttr and using a custom field transformer (probably not how you want to do it here in upstream), and one thing I'd like to point out is that it seems natural to have evolved fields stay at the same index as the original field rather than go to the end. E.g.

@define
class A:
    x: int = 0
    y: int = 1

# attrs order: x, y

@define
class B(A):
    x: int = evolve_field(default=42)

# attrs order: x, y

@define
class C(A):
    x: int = 42

# attrs order: y, x

Notice how if you simply redefine the x attribute as in C, the order of init parameters is swapped, which may be confusing when used with inheritance. However, in B, evolve_field (which can only be used with inheritance) the order is maintained, keeping the same interface as in the parent class.

Another issue that came up during my implementation was that some sanity checks like "no attrs without a default following an attr with a default" happen before the field transformer is run, so the field transformer has no chance to fix the issues. You may not run into this since it may not be implemented using a field transformer like I did.