Dart Override-like Augments

[lrhn]

(Elevator pitch. Tall buildings only.)

Override as a language feature

We introduce override as a modifier which can be put before a member declaration. It goes as the very first modifier, before external or static, just any after annotations.

class Foo implements Comparable<Foo> {
  override 
  int compareTo(Foo other) => ...;
}

It’s a compile-time error to have an override on an instance member which does not override a super-interface member

There is a lint to notify you if a member can be marked override, and isn’t.

Non-instance member override

Non-instance members can override a prior declaration of the same member in the same scope. Example:

int foo(int x);

override // Override must be marked as `override`.
int foo(int x) => x + 20;

override
foo(v) => foo.super(v) * 2; // Invoke prior implementation using `foo.super`.

void main() {
  print(foo(1)); // Invokes last version.
}

A declaration must be marked as override if it overrides something in the same scope. (Instance members overriding only superinterface members do not have to be marked override.)

Works for getters, setters and functions. The overriding method must be “valid override” of overridden member’s signature (same kind, out of getter, setter or function, and function type is a subtype of overridden signature’s function type).

Ordered by “generalized source order” (source-order in same file, preorder depth-first for part files in part-directive source order.)

An overriding function declaration can inherit type parameters, which means:

• Can omit type parameter list entirely, in which case they’re all inherited. Then cannot refer to type parameters by name.
  • If not omitted, there must be the same number of type parameters as in the overridden declaration.
• Can change names of type parameters.
• Can omit bounds of type parameters, in which case they are inherited (with any new parameter names substituted). If type parameter bounds are included, they must be the same type as in the overridden class (same rules as normally for overriding method type parameters).

An overriding declarations can omit return, parameter types and default values, they are inherited from the overridden member (only if required for default values, if an overriding declaration makes a parameter nullable, the default value is not inherited). Can rename positional parameters.

Invoking prior (overridden) function implementation: name.super (not super.name). Since super is reserved word, this is new syntax. Such a reference is allowed for any declaration, not just the same one, to invoke the prior concrete implementation in generalized source order.

Example:

int secretCounter = 0;
int get nextCounter => ++secretCounter.super;
void get secretCounter => throw "Secret!";
void set secretCounter(void _) => throw "Secret!";

This code allows nextCounter to access the prior declaration of secretCounter, both for reading and writing, and then makes later accesses hit another getter/setter pair. (Not intended as the way to make things secret!)

Non-foo.super invocations always invoke last concrete declaration in the library for that member, which must be a valid implementation of its signature.

Variables

A variable declaration introduces a getter and sometimes a setter, both of which can be overridden as any other getter or setter.

An overriding variable declaration, like override int x;, can have its getter and setter override an existing getter or setter declaration. If marked as override, it must be overriding at least a getter or a setter, but does not have to override both.

It’s a compile-time error if there are two variable declarations of the same static-ness (static or instance, if that applies to a scope) with the same name in the same scope, even if they are in different surrounding declarations. This is mainly to ensure that an initializing constructor can initialize the variable by name, and to keep things a little simpler for implementors.

An overriding multi-variable declaration, like override int x, y; must override at least one getter or setter for each declared variable.

Scope declaration override

A class-like scope-introducing declaration can be overridden in the same scope too. Example:

class Foo {
  int foo(int x);
  static int get counter;
}
override
class Foo implements Bar {
  final int y;
  Foo(this.y); // implicit `super();` invoking `Object()`.
  override
  foo(x) => x + 1;
  override
  static int get counter => ++_counter;
  static int _counter = 0;
}
override
class Foo {
  final int z;
  override
  Foo([int y = 37, this.z = 87]): Foo.super(y);
  override
  foo(x) => foo.super(x) * 2;
  static int get counter => counter.super + 1;
}
void main() {
  print(Foo().foo(1));
}

The override can also be applied to the first declaration of an instance member, in which case it still means overriding a member of a super-interface.

Applies to class, mixin, enum, extension and extension type declarations. Must override something of the same kind.

Combines the members scopes into one scope, with members applying in their source order. (Every member declaration of every class declaration for a class is in the lexical scope for its member declarations.)

Extends member-overriding to constructors as specified below.

Instance members and static members can be explicitly invoked as this.foo.super and Foo.counter.super in case of conflicts, but only from inside the same class(-like) declaration.

Super-interfaces and header-declarations

class

Each overriding class declaration can add with and implements clauses. An extends clause can only be added if there is no prior extends or with clauses on declarations of the same class. The mixins of with clauses are applied in generalized source order. Can inherit type parameters.

Each declaration of a class can add modifiers (abstract, access-modifiers, mixin). The abstract and mixin modifiers apply if any declaration has them. If multiple different access modifiers are added, the actual restriction is a modifier which is as strict as every declared modifier ( sealed + anything is sealed, final + anything other than sealed is final, base + interface is final).

operators

The foo.super syntax does not work for operators. I recommend (if possible) introducing .-access to operators:

• 2.+(3) is equivalent to 2 + 3, and 2.+ is (finally!) a tear-off of the int.operator+ method on the instance 2. And then 2.+.super(3) (is reminiscent of random-line noise, and) invokes the overridden operator+ with 3 as argument. Special syntax handling is needed to distinguish 2.-() and 2.-(1), and 2.- as a tear-off is ambiguous. Might need something special just for unary-.

(If we go for anything like “More capable Type objects”, we should prepare for static operators too.)

enum

Each overriding enum declaration can add with and implements clauses, and can add enum elements. An enum declaration with no enum elements must start with a ; (empty list of elements) if it has any other body declarations. The enum elements of the entire enum is the concatenation of the element declarations of all the individual enum declarations in source order. Can inherit type parameters.

mixin

Each overriding mixin declaration can add implements clauses and on clauses. The super-class requirements of multiple on clauses are all combined as super-class requirements of the entire mixin. Can inherit type variables.

extension

Each overriding extension declaration can inherit type parameters and can omit the on clause, which is then inherited. If the on clause is included, it must be a mutual subtype of the on type of the overridden declaration.

extension type

Each overriding extension type declaration can inherit type parameters, can omit the type of the representation variable (in which case it’s inherited), and can omit the entire “primary constructor” declaration (.name(R id) or just (R id)), in which case it’s all inherited. (Can’t just omit the representation object, but omit the constructor name.)

If the primary constructor syntax is retained (any parentheses) then it must have the same constructor name as the overridden declaration. If representation variable is retained, it must be the same identifier as in the overridden declaration. If the representation type is retained, it must be a mutual subtype of the representation type of the overridden declaration.

Each overriding extension type declaration can also add implements clauses.

Constructors

Factory constructors can override factory constructors, generative constructors can override generative constructors.

A constructor is const if it is declared so.

Factory constructors (and other members) can freely refer to overridden members other than generative constructors. Generative constructors are special, because they must be complete in order to be valid (they must initialize all required variables.) Nothing can refer to an overridden generative constructor as Foo._.super other than the next overriding generative constructor declaration of Foo._.

An overriding redirecting generative constructor can override both a redirecting or non-redirecting generative constructor, replacing their behavior entirely.

An overriding non-redirecting generative (aka. initializing) constructor can override a redirecting or initializing constructor. If it overrides an initializing constructor, it can, but doesn’t have to, invoke the overridden generative constructor in place of a super-constructor invocation (and no other overridden declaration). It can also invoke a superclass constructor like normal, in which case it replaces the overridden declaration entirely. (If no super-constructor invocation is written, it defaults to super() as usual.) If it overrides and forwards to the overridden constructor, then it must be const if an only if the overridden constructor is (and it inherits a required const if one isn’t written). The syntax for invoking an overridden constructor is ClassName.constructorName.super(…), or ClassName.super(…) for the unnamed constructor. Any super.-parameters are forwarded to this constructor the same way they would be to a superclass constructor invocation.

Example:

class Point { // Pure API.
  abstract final int x, y;
  Point(int x, int y); // Initializes no variables.
}

override class Point { // Implementation. 
  override
  final int x, y; // TODO: Primary constructor. 
  override
  const Point(this.x, this.y); // : super().
}

override
class Point { // Declaration adding and initializing extra field.
  final Color color;
  override Point(super.x, super,y, [this.color = Color.transparent]) : Point.super(); // Inherits `const`
}

This design allows replacing a constructor with a different implementation, and wrapping and/or extending an initializing constructor.

The middle constructor replaces the implementation of the overridden constructor (which happens to be trivial) and can therefore choose to be const capable.

It’s a compile-time error if an initializing constructor initializes an instance variable, and then invokes its overridden declaration which also (directly or transitively) initializes the same instance variable. Every real constructor has a set of variables that it initializes.

It’s a compile-time error if the final declaration of a constructor is an initializing constructor, and it doesn’t initialize all required variables (as usual). _If every non-redirecting generative constructor initializes all the variables of its own class declaration, and those before, then that’s easy to keep track of. Sometimes you do want to declare a variable before you introduce the code that

Examples

class Compatible<T extends Compatible<T>> {
  Compatible();
  int cmp(T other);
}
override
class Compatible<T> implements Comparable<T> {
  override
  int cmp(T other, {bool descending = false}) => (descending ? -1 : 1) * other.compareTo(this); 
}
override
class Compatible {
  final Logger _logger;
  Compatible(this.logger): Compatible.super(); // Or `Compatible.new.super()`.
  override
  cmp(other, {descending}) {
    _logger.log("Comparing $this and $other${descending ? " (descending)": ""}");
    return cmp.super(other, descending: descending)
  }
}

Augmentations - as overrides

That’s it.

Instead of trying to avoid ordering dependencies, this buys into ordering by giving all declarations in a library a total order, and allowing most declarations to refer to the prior concrete version of any declaration in the same (or parent) scope.

You can have a top-level int _counter = 0; and any declaration later in source order can write _counter.super to read it, and any later override of int get _counter will then be ignored.

That’s not something you’ll need to use often, you will usually only need to refer to the prior version of the function being overridden, but for getter/setter pairs you may want both, and just giving the full capability (other than for initializing constructors, which never exist as partially implemented) might be just what users want in a few cases. It’s also unlikely to be something you hit by accident. Most code won’t need to split a class into two declarations, even less a method into two declarations, so there is little need to use the foo.super notation unless you’re actively building around local overrides.

Variations

A lot of the “you can omit this or that, and it’s inherited” can be dropped, if it’s bad for readability.

It’s also annoying to have to write something obvious and/or unnecessary.

In the other direction, I shied away from allowing you to omit an entire parameter list. We could allow that, then override threeParameterFunction => throw UnimplementedError("Nope!")); would be valid, and you won’t have to write (_, {longName, otherLongName}) as parameter list when you don’t need to refer to the names anyway.

The overriding of constructors is very permissive. One thing I worry about is that you can completely bypass a partially declared constructor, and then that code will never be reachable. That is good, since a partial constructor may not initialize all fields, but it leaves dead code in your program. I’m imagining a use-case where class has default constructor implementation, but a “code generator” can choose to replace it entirely. If that code generator also adds more fields to the class, then the dead constructor is no longer complete, and that’s OK since it’s also not reachable. All (reachable) constructor implementations are complete.

[lrhn]

I said I'd try to do something "override-like" for augmentations. This is ... an approach. @dart-lang/language-team

[eernstg]

I think this is more well-integrated into the language than the approach where the previous declaration is invoked using augmented(). The ability to call foo.super() is also more flexible and expressive than augmented().

However, I also think we should keep in mind that operations such as "sort members" and "sort part directives" are prone to change the ordering and hence the semantics. For example, a can't invoke b.super() after sorting if both a and b are top-level function declarations in the same file. So maybe we'll have to move one of them to a part file.

In other words, depending so much on the ordering does have a cost, no matter whether it's called augmented() or foo.super().

[lrhn]

Order dependence has a cost. It's not really possible to wrap and delegate to an existing implementation with an "augmentation" without an ordering dependency, so if we have that kind of augmentation, we will have ordering dependencies.

I have tried minimizing those dependencies, but that just added other restrictions, which could get in the way even if you don't actually augment, or only do so safely.

This was an attempt to just accept the order dependency, with no extra strings attached, so that you only pay the cost for it if you use it.

Most code won't. Code that does will have to be careful (don't short by name of you invoke something with a different name). But mostly: don't have two declarations of the same thing in the same physical scope. It's not necessary, so just don't. Then sorting is safe. Unless you're generating code and combining multiple outputs in one file. Then don't sort that file.

Either works, but don't both generate and sort. And don't make the language try to stop you, because it doesn't know whether you're going to sort the source or not, so any rules added will sometimes be too strict.

[natebosch]

(only if required for default values, if an overriding declaration makes a parameter nullable, the default value is not inherited)

I think it's better to use an explicit = null to avoid inheriting the default.

override
class Foo implements Bar {
  final int y;
  Foo(this.y): y = 37; // implicit `super();` invoking `Object()`.

Is there a typo here? I don't understand having a this.y parameter and a y = 37 initializer in the same constructor.

if we have that kind of augmentation, we will have ordering dependencies.

My gut reaction is that it will be easier for authors to reason about ordering dependencies if they are only at the level of part directives. I'm not as worried about being unable to sort members as much as I am worried that adding a new override definition may inadvertently changed the meaning of some .super references below it. I think it might be OK to allow the increased complexity. As you point out, authors only pay a cost if they use the feature.

[mmcdon20]

Is this essentially #1610 with extra features like "non-instance member overrides"?

A question about this example:

int foo(int x);

override // Override must be marked as `override`.
int foo(int x) => x + 20;

override
foo(v) => foo.super(v) * 2; // Invoke prior implementation using `foo.super`.

void main() {
  print(foo(1)); // Invokes last version.
}

Does this allow users to declare foo without a body so long as it is overridden later?

[lrhn]

Yes, y= 37 is a typo. Fixed. Thanks.

This is not so much "#1610 with extra features", the point is the extra features. It just includes #1610.

And yes, you can declare an "abstract" member signature in a non-abstract class, as long as the member has an implementation. The change here is that now the implementation can also be written later in the same class, not just be inherited.

[mmcdon20]

And yes, you can declare an "abstract" member signature in a non-abstract class, as long as the member has an implementation. The change here is that now the implementation can also be written later in the same class, not just be inherited.

Well the example was an "abstract" top level function, so I assume this applies also to top level functions and static members.

[lrhn]

That too. Which means a grammar change to allow a function declaration to be both static and have no body, or top-level and have no body. Then it's a compile-time error if the collection of all declaration for a name does not provide an implementation for the function signature.

[munificent]

Am I reading right that this is basically:

• override as an optional modifier for instance overrides instead of @override.
• override to mean "augmenting" for non-instance members?

If so, how does augmenting an instance member work?

class A {
  m() => print('A.m');
}

class B extends A {
  override m() => print('B.m');
}

override class B {
  override m() => super.m();
}

main() {
  B().m();
}

Would this print "A.m" or "B.m"? Is the idea that we see if override could be augmenting and if so, it is, otherwise, it behaves like inheritance-based override and likewise with super calls?

If so, it's not a bad idea, but I'm pretty hesitant to use the same word to talk about two fairly different kinds of extension. Inheritance and overriding is a fairly different mechanism from augmentation. The former introduces an actual new member (for example, tearing off A().m gives you a different object than B().m). The overriding member can change the signature by widing parameter types, adding optional parameters, etc. The latter only produces a single monolithic declaration that is merged at compile-time. The signatures must match exactly.

That leads me to want different keywords for these mechanism, though I admit it is hard to come up with a nice syntax for "augmentation super".

Is this a proposal you're still interested in?

[lrhn]

Would this print "A.m" or "B.m"?

It would print "B.m".

There is only one kind of relation between members of the same name in the same class. They're all ordered in an override chain, and can call the previous one using super. It doesn't matter, for that, if two of them were declared in the same declaration, or in same-named declarations in the same library.

Is the idea that we see if override could be augmenting and if so, it is, otherwise, it behaves like inheritance-based override and likewise with super calls?

The idea is that every name can refer to (the head of) a chain of declarations, like instance members do today. The declarations themselves can use super to refer to the prior declaration.

Inheritance and overriding is a fairly different mechanism from augmentation

The point is that here it isn't. Or rather, there is no "augmentation".

An overriding class declaration with the same name, in the same library, works mostly like it was a subclass, and the prior class is just shadowed and un-namable. (It's actually more like an anonymous mixin application that also adds static members, but it as an extra layer.)

Is this a proposal you're still interested in?

Probably not. If we have gotten rid of augment super, and allow only one concrete declaration, that's a simple enough feature that we don't need to address one augmentating declaration calling another.

If we want augmenting implementations that stack, then I still think this could be a better model.

[munificent]

The point is that here it isn't. Or rather, there is no "augmentation".

An overriding class declaration with the same name, in the same library, works mostly like it was a subclass, and the prior class is just shadowed and un-namable.

Sure, for instance members where we already have polymorphism. But there's no corresponding mechanism for static members or top-level functions. (I can hear Gilad shouting "first-class libraries!".)

Probably not. If we have gotten rid of augment super, and allow only one concrete declaration, that's a simple enough feature that we don't need to address one augmentating declaration calling another.

SGTM. I'll go ahead and close this but it will remain in the issue tracker if we ever want to resurrect it.