Augmentations, non-redirecting generative constructors and instance variable initializer expressions.

[lrhn]

TL;DR: All instance variables with initializer expressions of a class definition are initialized first thing, before invoking any non-redirecting generative constructor code. The the non-redirecting generative constructor code is executed in stack order, with the parameter list and initializer list of augmenting constructors executed before recursing on the augmenting definition, and the body executed after coming back. When reaching the base declaration it recurses on the superclass constructor as an outside call, which means it too initializes instance variables when firt reaching a non-redirecting generative constructor, but possibly after redirecting generative constructors. Super-parameters are accumulated along the way, put into pre-computed positions.

There is entirely too much backgound and details below, and it may be slightly more general than necessary, but I think it is consistent and usable.

This defines when initializer expressions for instance variable declarations are evaluated (first) and in which order (base declaration source order, with multi-file ordering being preorder depth-first on parts).

The biggest thing here is that an augmenting constructor (effectively) prepends its initializer list to the existing list. This ensures that the scope used to evaluate the initializer list can be stack allocated.

We need to have recursed to the base definition at the end of the initialzier list, where we call the superclass constructor. If we append new initializer lists, then we need the augmenting declaration's initializer list entries between the initialzier list and the super-constructor invocation of the base declaration, at which point we don't want to unwind the stack to the augmenting declaration's parameter scope. If we prepend, it just works.

Motivation

We have agreed that an augmenting non-redirecting generative constructor declaration with a body will execute its body after the execution of the body (or transitive bodies) of the constructor it is augmenting.

We also need to specify when instance variable initializer expressions are evaluated. Those are currently evaluated at the beginning of the invocation of a non-redirecting generative constructor. (Or just prior to the invocation, depending on perspective. Since every generative constructor performs the same initialization, it may be preferable to see it as existing outside of the constructors.)

With augmentations, we can now have augmenting constructors and augmenting variable initializer expressions. We need to place the variable initialization in the invocation hierarchy, preferably in a consistent and predictable place.

Terminology

A base (non-augmenting) declaration introduces a definition which mirrors the declaration, but at a slightly higher abstraction level. It’s possible for two differently written declarations to introduce indistinguishable definitions, if the difference is entirely syntactic. (For example abstract final int x; and int get x; both introduce a single abstract getter definition named x with type int.)

The kind of definition determines which properties it has.

An augmenting declaration is applied to an existing definition and introduces a new definition of the same kind. The new definition may refer to the prior definition, so that its implementation may refer to the augmented implementation.

Proposed augmented generative constructor behavior

When we invoke a generative constructor named g (for example C.id or C) on an class definition C with type parameters X1,..,Xn, instantiated with type arguments T1,…,Tn, with an argument list L to initialize an object o (with a runtime type that is known to extend C and implement C<T1,..,Tn>):

• Let d be the constructor definition named g of C.
• If d is a redirecting generative constructor definition, where g’ is name of the constructor it redirects to (this.id denotes a target of C.id in a class named C):
  • Bind actual arguments L to the formal parameters of G, creating the parameter scope.
  • Evaluate the arguments of the redirection clause in the parameter scope to an argument list A.
  • Then invoke the generative constructor named g’ on C instantiated with type arguments T1,…,Tn, with argument list A to initilaize o.
• If d is a non-redirecting generative constructor definition:
  • For each instance variable definition of C in base declaration order (the order of the base declaration in the total traversal ordering of a library), where the variable definition has an initializer expression, and is not declared late:
    • Evaluate the initializer expression of that definition to a value v.
      • Evaluating the initializer expression of a variable definition is defined recursively on the stack of augmenting definitions. The stack of definitions is traversed until a definition is found which introduces an initializing expression, then that expression is evaluated. If the definition is an augmenting definition and its augmented definition also has an initializing expression, then the expression is evaluated in a context where augmented may be referenced, and doing so evalautes the initializer expression of the augmented definition. That is: Having an initializer expression is a property we can know directly for each variable definition, but how to evaluate it depends on the entire stack of definitions.
    • Initialize the corresponding instance variable of o to the value v.
  • Let S be an uninitialized argument list with the number of positional arguments and names of named arguments of superParameters of d (which is the shape of the argument list of the super-constructor invocation including all explicit arguments and super parameters of the entire stack of definitions).
  • Invoke the constructor definition d of C<T1,…,Tn> with argument list L and super parameters S to initialize the value o.

We then define the invocation of a constructor definition as follows, allowing an augmenting constructor’s definition to delegate to the definition it augments.

Invocation of a non-redirecting generative constructor definition d on an instantiated class definition C<T1,…,Tn> with argument list L and super parameters S to initialize an object o proceeds as follows (at least for any class other than Object, which just completes immediately):

• Bind actual arguments to formal parameters. For each parameter in the parameterList of d (in sequence order):
  • If the argument list L has a value for the corresponding positional position or named argument name, let v be that value.
  • Otherwise, the parameter must be optional.
    • If the parameter has a default value expression, let v be the value of that expression.
    • Otherwise let v be the null value.
• If the parameter is an initializing formal with name n,
  • Initialize the variable of o corresponding to the instance variable named n of C to the value v.
  • Bind the name n to v in the initializer-list scope.
• Otherwise, if the parameter is a super-parameter.
  • If the parameter is named, set the value of the argument with name n of S to v.
  • Otherwise the parameter is positional:
    • Let i be one plus the number of prior positional super parameters in the parameter list of d.
    • If d has a superDefinition d’, add the count from superParameters of d’.
    • Otherwise if d has a superConstructorInvocation, add the number of positional arguments in the argument list of that invoaction.
    • Set the positional argument value with position i in S to v.
  • Bind the name of the parameter to v in the initializer-list scope.
• Otherwise the parameter is a normal parameter. Bind the name of the parameter to v in the parameter scope.
• For each entry in the initializer list of d, in sequence order:
  • If assert entry, evaluate the first operand in the initializer list scope. If evaluate to false, evaluate the second operand to a message value m, if there is a second operand, otherwise let m be the null value. Throw an AssertionError with m as message.
  • If variable intializer entry, evaluate expression in initializer list scope to value v. Initialize the variable of o corresponding to variable with the initializer entry name in C to the value v.
• If d has an augmentedDefinition d’
  • Invoke d’ on C<T1,…,Tn> with arguments L and super-arguments S to initialize o.
  • This recurses on the augmented definition.
• Otherwise:
  • Let U be be the instantiated superclass definition of C<T1,…,Tn>.
    • This may be a synthetic definition (for an anonymous mixin application) computed from the declared superclass and declared mixins of C. _(Or we can do the short-circuiting here since mixin applications can only)
  • If d has a superConstructorInvocation:
    • Evaluate each argument of the argument list of the super-constructor invocation in the initializer list scope, in source order, and set the entry of S with the same position or name to the resulting value.
    • Let g be the name of superclass constructor of U targeted by the superConstructorInvocation (class-name of U plus .id if referenced as super.id).
  • Otherwise let g be the name of the unnamed constructor of S.
  • Invoke the constructor named g on U with arguments S to initialize o.
  • This recurses on the superclass constructor.
• When this has completed, execute the body of d, if any, in a scope which has the parameter scope of the invocation of d as parent scope.
• Then invocation of d completes normally.

The consequence of this definition is an execution order for initialization of an instance of a class of:

• Field initializers of class, from all augmentations, in “base declaration source order”.
• Augmenting declaration parameter lists and initializer lists, for augmentations in last-to-first order.
• Base declaration parameter list, initializer list
• Base declaration super-constructor invocation, recurses to this entire list on superclass.
• Body of base declaration.
• Bodies of augmenting declarations in first-to-last order.

It ensures that we only recurse in one place, keeping a stack discipline. The parameter scope of the invocation can be stack-allocated, and be on top of the stack when the scope is used. (If we execute initializer list entries after the ones of an augmented definition, we fail to maintain that stack order.)

Details of the definitions

This focuses only on the properties of definitions that are relevant here.

A variable definition has (among others) the following properties:

• hasInitializerExpression (Boolean, derivable, true if the definition itself has an initializerExpression, or if it has an augmented definition which hasInitializerExpression, false otherwise)
• initializerExpression (optional source code, post type inference, may contain augmented reserved words if there is an augmentedDefinition that hasInitializerExpression.)
• augmentedDefinition (optional, set for definitions introduced by applying augmenting declarations, to the augmented definition, not for the definition of a base declaration).

A (non-abstract, non-external) variable declaration introduces a variable definition. This is the definition that implies a backing storage. A variable declaration also introduces a getter definition, and maybe a setter definition. Those are not relevant here.

These properties are updated when applying an augmenting variable declaration A to a variable definition d as follows:

• The augmentedDefinition of the result is always d.

• If A has an initializer expression, then that is the resulting definition’s initializerExpression and hasInitializerExpression is true.

• Otherwise the resulting definition has no initializerExpression and its hasInitializerExpression has teh same value as the hasInitializerExpression of d.

• (Any other change an augmenting variable declaration can do, which is just adding metadata. Name and types are inherited as-is.)

You evaluate the initializer expression of a variable definition d which hasInitialierExpression to a value v in a scope S as follows:

• If d does not have an initializerExpression,

  • Then d has an augmentedDefinition, d’, which hasInitializerExpression, so evaluate the initializer expression of d’ to a value v in scope S. Then evaluating the initialier expression of d evaluates to v too.

• Otherwise let e be the initializerExpression of d:

  • If d does not have an augmentedDefinition (it’s the definition of a base declaration, so augmented is not reserved in e), then evaluate the expression e to a value v in scope S. Then evaluating the initializer expression of d evaluates to v too.

  • Otherwise:

    • Let d’ be the augmentedDefinition of d.

    • Then augmented is a reserved word inside e. If augmented occurs inside e, then the hasInitializerExpression of d’ must be true.

    • Evaluate the expression e in an “augmented-allowing” scope extending S to a value v, where evaluating the identifier expression augmented performs the following operation:

      • Evaluate the initializer expression of d’ to value v’.
      • The expression augmented evalautes to v’.

    • Then evaluating the initializer expression of d evaluates to v.

A non-redirecting generative constructor definition has (among others) the following properties:

• augmentedDefintion: Optional, set if introduced by an augmentation application, the non-redirecting genreative constructor defintion that was augmented.

• parameterList: Sequence of constructor parameter definitions. Each has a name and type, can be optional or required, positional or named, have a default value expression if optional, and be either an initializing formal, a super-parameter or a “normal” parameter.

  The parameters must always match the types and names of parameterList of augmentedDefinition, if any. The “position” of a positional parameter in such a list is defined as one plus the count of positional parameters earlier in the sequence.

• initializerList: Sequence of initializer entries, either variable initializer or assert.

• InitializedVariables: Set of identifiers. The instance variables initialized by this definition stack.

• superConstructorInvocation: Optional, only set on base declaration’s definition.

• superParameters: integer count and set of identifier names, the super-parameter positions and names already filled by the initializer lists and super-constructor invocation of this definition, positional and named.

• body: Optional code block.

The definition of a non-augmenting non-redirecting generative constructor declaration G has the following properties:

• No augmentedDefinition.
• parameterList directly derived from the declaration
• initializerList directly derived from the declaration
• The superConstructorInvocation of the declaration, if it has one.
• initializedVariables: Set of names of variables initialized by initializing formals in the parameter list, and names of variable initializer entries of the initializer list. Static error if any name is initialized more than once.
• superParameters: Count of positional arguments of super-constructor invocation (zero if none) plus count of positional super parameters in parameter list, and set of names of named arguments of super-constructor invocation (if any) and names of all named super parameters of parameter list. Static error if same name occurs more than once.
• body: Constructor body, if any.
• (Everything else)

Applying an augmenting non-redirecting generative constructor declaration A to such a definition, d produces a result with the following properties:

• augmentedDefinition is d
• parameterList is a sequence of parameters derived from the parameter list of A and the parameterList of d.
  • The parameter list of A must have the same number of positional, and optional positional, parameters as the parameterList of d and the same names and optionality of named parameters as the as the parameterList of d.
  • The parameterList of the resulting definition has one entry per parameter declaration in the parameter list of A, in source order. The entry is an initializing formal or super parameter if the parameter of A is. The parameter is named or positional, and required or optional, as the parameter of A, and has the same default value expression, if any.
  • If the parameter list of A omits a type variable from a parameter declaration, the resulting parameterList’s type for that parameter is the same type as that of the corresponding parameter in d’s parameterList.
  • It’s a compile-time error if A contains an initializing formal parameter with a name that is in the initializedVariables of d.
  • It’s a compile-time error if A contains a super-parameter with a name that is in the set of names of the superParameters of d.
• initializerList is a sequence pf the entries of the initializer list of A.
  • It’s a compile-time error if the initializer list of A contains an initializer for a variable with the same name as an initializing formal of A or with a name in the initializedVariables of d.
• initialziedVariables: The (disjoint) union of the initializedVariables of d and the set of names of initializing formals of A and the variable names of variable initialziers in the initializer list of A.
• superParameters: Count of positional super-parameters of d plus number of positional super-parameters in the parameter list of A, and set of super-parameter names of d (disjoint) union with the set of names of named super-parameters in the parameter list of A.
• body: The body of A, if any.

Existing object creation and generative constructor behavior

For generative constructors, the existing pre-augmentation specified behavior is:

When you evaluate a constructor-based object creation expression (expression which invokes a constructor of a class, C, with or without an explicit new, as opposed to, for example, an object creation expression which is a list literal), the following happens (which ignores cases that would have been rejected as compile-time errors):

• A new object instance of the requested class C is created. If the class declaration for C is generic, the type arguments given to C are stored on the object, and type arguments to superclasses are stored too, somehow, so that they can be accessed in instance members.
• The denoted constructor of the mentioned class is invoked with the given argument list to initialize the new object. (“Invoke … with argument list … to initialize” is what you do when to an existing object, just “Invoke” of a constructor is a short way to refer to an constructor-based object creation expression.)

Invoking a generative constructor G of a class C with an argument list L to initialize an object o (phew) is performed as follows:

• If the constructor G is a redirecting generative constructor:

  • Bind the argument list L to its formal parameters to create a parameter scope (as usual, absent arguments imply optional parameters which then get their default value, or a default default value of null). The parent scope of the parameter scope is the member scope of the surrounding class.
  • Evaluates the arguments of the redirection clause in the parameter scope to an argument list A.
  • Invoke the target generative constructor (always of the samme class) with the new argument list A to initialize the same object o.

• If G is a non-redirecting generative constructor:

  • For each instance variable declaration in the surrounding class declaration, in source order, if the instance variable declaration has an initializer expression e:

    • Evaluate e in the body scope of the class C to a value v. As usual, if anything throws, then so does the constructor invocation to initialize.
    • Initialize the corresponding instance variable of o to v.

  • Let A be an uninitialized argument list corresponding to the super parameters of the constructor and the arguments of the super-constructor invocation, if any: One positional entry per positional super parameter or positional argument in the super-constructor invocation, one named entry per named super parameter or named argument in the super-constructor invocation, with no values set yet.)

  • Bind arguments to formals. This has more cases than normal function invocations due to initializing formals and super-parameters.

    • For each parameter of the constructor declaration, in source order:

      • If the argument list has a corresponding value, let v be that value.

        Otherwise the parameter must be optional.

        • If the parameter has a default value expression, let v be the value of that expression.

        • Otherwise let v be the null value.

      • If the parameter is an initializing formal for an instance variable:

        • Initialize the cooresponding variable of o to the value v.
        • Bind the name of the parameter to the value v in the initializer list scope.

      • If the parameter is an super parameter.

        • If the parameter is positional, let i be the number of prior positional super-parameters in the constructor parameter list plus the number of positional arguments in a super-constructor invocation, if any, plus one. Set v as the (one-based) ith positional argument of A.
        • If the parameter is named with name n, set the named argument n of A to v.
        • Bind the name of the parameter to the value v in the initializer list scope.

      • Otherwise the parameter is a normal paramter.

        • Bind the name of the parameter to the value v in the parameter list scope.

    • For each non-super-constructor entry in the initializer list of the constructor declaration:

      • If an assert entry with test expression e:
        • Evaluate e to a value v of type bool in the initializer list scope.
        • If e is false:
          • If the assert has as second expression, evaluate that expression to a value m, then throw an AssertionError containing the value m as message.
          • Otherwise throw an AssertionError with no message.
      • If an initializing entry for an instance variable, with initializer expression e (id = e or this.id = e).
        • Evaluate e in the initializer list scope to a value v.
        • Initialize the corresponding variable of o to the value v.

    • For each instance variable of the class declaration, if the corresponding instance variable of o has not yet been assigned a value, initialize that instance variable to the null value.

    • Let S be the superclass of the current class. (Not class declaration, but instantiated class, which can be an anonymous mixin application class with no explicit declaration.)

    • If the initializer list ends with a super-constructor invocation (super(args) or super.name(args)).

      • evaluate the argument expressions of that invocation in source order, in the initializer list scope. If the expression evaluates to a value v.
      • If the argument is positional, set the value at position i of A to v, where i is one plus the number of positional arguments prior to this argument in the super-constructor invocation arguments.
      • If the arugment is named with name n, set the value of the named argument n of A to v.
      • Let T be the constructor of class S targeted by the super (unnamed) or super.name (named)

    • Otherwise, if there is no super-constructor invocation, let T be the unnamed constructor of class S.

    • Invoke the constructor T of class S with argument list A to initialize o.

    • When this completes, execute the body of C with this bound to o. The parent scope of the body block’s scope is the parameter scope of the invocation.

[lrhn]

@dart-lang/language-team

As TL;DR says, I suggest doing instance variable initializer expression evaluation for a class before starting invocation of the (agumentation stack of) the non-redirecting generative constructor, AND to execute augmenting initializer expressions first, before the ones of the augmented definition, because that keeps a stack-like execution order.

[jakemac53]

The only real piece of pushback I have here is that this proposal removes the ability for an augmentation to add an explicit super constructor invocation. This was allowed in the previous proposal.

It does complicate things, because now it isn't a strict pre-pend operation for initializers, but it seems like if augmentations can add an extends clause, they will also need the ability to make an explicit super constructor call.

[stereotype441]

The only real piece of pushback I have here is that this proposal removes the ability for an augmentation to add an explicit super constructor invocation. This was allowed in the previous proposal.

It does complicate things, because now it isn't a strict pre-pend operation for initializers, but it seems like if augmentations can add an extends clause, they will also need the ability to make an explicit super constructor call.

Interesting! I had not realized that we were allowing augmentations to add an extends clause. Do we also allow augmentations to change an existing extends clause? I'm assuming we don't, because the semantics of that seem... weird.

The reason I ask is because in my mental model all classes (except Object) as having an extends clause, but as a piece of convenient syntactic sugar, we allow extends Object to be elided. Similarly, I think of all constructors (except Object()) as having a super constructor invocation, but as a piece of convenient syntactic sugar, we allow : super() to be elided.

So if we're allowing an extends clause to be added, but we're not allowing an existing one to be changed, then in this mental model, what we'd actually be allowing is for the user to replace an existing extends clause (but only if the clause they're replacing is an implicit extends Object), and allowing them to replace an existing super constructor invocation (but only if the invocation they're replacing is an implicit : super()). Which seems oddly arbitrary IMHO.

Related question: if a class is declared with extends Object, do we allow an augmentation to "add" an extends clause (and thereby replace 'extends Object')? Similarly, if we want to allow an augmentation to "add" an explicit super constructor call, do we allow this to happen if there's an existing : super() (thereby replacing it)?

• If yes, then again, that seems oddly arbitrary IMHO.
• If no, then that means that class C extends Object {} no longer has precisely the same behavior as class C {}, and the constructor C() : super(); no longer has precisely the same behavior as C();, which seems unfortunate.

[bwilkerson]

The reason I ask is because in my mental model all classes (except Object) as having an extends clause ...

I agree, all classes have an extends clause, whether explicit or implicit. The way I'd work around that in my head is to distinguish between the class that's being defined, and the fragment that's contributing to that definition.

So in my head, a fragment isn't replacing an extends clause, it's contributing one to the merge operation. And the merge operation doesn't allow multiple extends clauses to be contributed (maybe unless they're identical? I haven't looked at the current proposal in a while).

[jakemac53]

What @bwilkerson said basically - the implicit extends Object, default constructor, etc are not there until all the augmentations have been merged into a complete class definition. Then, if there is no extends clause or default constructor they are added.

[lrhn]

Adding a super constructor invocation in an augmentation makes sense.

If we do that, then I think it should be an error to have super parameters in declarations prior to adding the super constructor invocation. Or rather, having a super parameter with no declared super constructor invocation implies one of the form super() as usual. Which means that declarations until that have filled in zero positional and named arguments in the super constructor argument list too, so the first augmentation to declare a super invocation is free to start from scratch.

Until the first super parameter or explicit super constructor invocation, the super invocation remains undetermined, and open for later augmentations to determine. And if they don't, it was super() all along.

The invocation chain will then carry with it the partially or fully filled out super constructor invocation (the argument list and the target constructor) until reaching the base declaration. That contains only values and a static reference, so it doesn't need access to the scope of the later augmentations when it's invoked.

Can still work.

[lrhn]

I agree, all classes have an extends clause, whether explicit or implicit.

Let me disagree then. Pedantically and semantically only, I agree with the conclusion. I'm just not-picking terminology.

Every class has a superclass (except Object). Not every class declaration has an extends clause, defining a declared superclass. Some don't, and until now that has meant that the class they define has the Object class as effective declared superclass (which is the actual superclass of the class if there are no declared mixins).

So I do agree that every class declaration introduces a class definition, and Only the fully augmented class definition is used to introduce the actual class. The class must have s superclass.

And if that still had no declared superclass, mixins or constructors, the actual class gets Object as superclass, and gets a default constructor.

(I call these "definitions" because I want a name for the stack of declarations, and have a way to talk about its cumulative properties. We could achieve that entirely with helper functions, asking "what is the declared superclass of this stack of declarations", and just change the source to say "stack of declarations" where it currently says "declaration". I like to give things names, a stack of declarations is a definition.)

There is a difference between class C {} and class C extends Object {} syntactically, and augmentations live somewhere between syntax and semantics. The two class declarations are different wrt. modifications. You can add extends Foo only to one of them. Directly or through augmentations.

The lesson to give users is that for augmentations, saying nothing actually means nothing. Defaults come after augmentations.

[jakemac53]

If we do that, then I think it should be an error to have super parameters in declarations prior to adding the super constructor invocation. Or rather, having a super parameter with no declared super constructor invocation implies one of the form super() as usual.

I think this is probably a reasonable restriction.

[lrhn]

I think we could safely allow an augmenting constructor to have an initializer list that ends in augmented(args), as a way to pass different arguments to the augmented constructor. If you don't have one, the implicit default is to call augmented with the same argument list that the augmenting constructor was called with, which is alway a valid argument list for something with the guaranteed same signature.

It could even be useful, fx, to allow an augmenting constructor to wrap an argument:

class ZonedTask {
  final Zone zone;
  final void Function() task;
  ZonedTask(this.zone, this.task);
  void run() { zone.run(task); }
}
augment class ZonedTask {
  augment ZonedTask(Zone zone, void Function() task) : augmented(
      zone.fork(zoneValues: {#_log: Logger.instance}),
      () { 
        (Zone.current[#_log] as Logger).log("task");
        task();
      },
  );
}

(OK, not a very good example.)

I think that should give all the benefits of being able to control the augment-super-invocation, without actually being able to run more or less than once.

[jakemac53]

I think we could safely allow an augmenting constructor to have an initializer list that ends in augmented(args), as a way to pass different arguments to the augmented constructor.

I would rather just not allow it. I am much more interested in allowing you to control the order in which the bodies run, than changing the arguments passed to them, which I think is an anti feature in this case. It is just too weird to see a field formal parameter, pass a value to it, and not have that be the value the field is initialized with.

[lrhn]

We can easily not allow an augmented(...) "super call".

If we allow it, it would be consistent with how we use augmented in other methods: the same way over could use super if the augmented method had been overridden instead of augment. (Does not apply to initializer expressions, can't super-call those.)

The end of the initializer list is the point where a constructor invokes another constructor. If we ever do want to control or intercept the arguments passed to the augmented constructor, that would be the place, not anywhere in the body.

[lrhn]

Also need to specify the other off instance variable initializer evaluation. I suggest source order of the base declarations, with each base declaration then evaluating its corresponding fully augmented initializer expression.

It's a little convoluted because it mixes ordering (order of first declaration for each variable, then last-to-first order of the expressions (at least if they refer to augmented, otherwise just last).

[jakemac53]

@dart-lang/language-team Reading through this again to try and merge it into the spec, but wanted to discuss a few parts of the proposal.

@lrhn Am I correct in interpreting this such that initializing formal parameters and super parameters may appear in any augmentation (but only once for any given parameter throughout the entire chain?).

Essentially, we use that one occurrence of the parameter to inform when exactly to initialize the field (for initializing formals at least), as well as to inform the ordering of positional parameters in the super constructor call?

I think this is pretty problematic for super parameters, and also just confusing for users when looking at a constructor, you cannot tell what is an initializing formal or super parameter without visiting all augmentations. You could argue this should be an implementation detail, but I think for positional super parameters explicitly this is also highly error prone.

Regarding my error prone assertion, consider the following:

class A {
  final int b;
  final int c;
  final int d;
  
  A(this.b, this.c, this.d);
}

class B  extends A {
  B(int b, super.c, int d) : super(d);
}

augment class B {
  B(super.b, int c, int d);
}

Which position are the parameters b and c passed to the unnamed constructor of A? I honestly couldn't tell you with confidence what the behavior is reading the current proposal, but it is confusing regardless. The majority of users should not have to understand the specific ordering of how these parameters are encountered and added to the super constructor invocation, we should instead design something where it doesn't matter, if at all possible.

Proposed change

I would prefer to instead keep the existing behavior, which makes the kind of the parameter (regular, initializing formal, super) something which must be consistent across all augmentations and the base declaration.

In terms of the ordering for initializing formals, the fields should be initialized prior to visiting the initializer list of either the base constructor declaration or the last augmentation, either is probably fine.

Alternative option

We could also say that only the base declaration is allowed to declare parameters as initializing formal or super parameters. All augmentations only use normal parameter syntax regardless of the type of parameter. This would allow us to change our mind and do something more along the lines of what you propose here, in a way that isn't breaking. And it would mean less churn if converting a parameter to a super parameter or initializing formal (or back to a normal parameter).

[lrhn]

@lrhn Am I correct in interpreting this such that initializing formal parameters and super parameters may appear in any augmentation (but only once for any given parameter throughout the entire chain?).

As I remember it, you can even use the same position more than once.

class C {
  final int y;
  final int z;
  C(int x, int y) : super();
}
augment class C {
  augment C(super.x, this.y);
}
augment class C {
  augment C(super.x, this.z);
}

will combine to, effectively:

class C {
  C(int x, int y) : this.y = y, this.z = y, super(x, x);
}

(If we allow renaming positional parameters.)

Each augmenting constructor can append to the the already constructor super-argument-list by using super-parameters, and they do so independently of each other. Since they cannot double-declare, at most one version of a named parameter can be a super-parameter or initializing formal, otherwise there'd be two named super-parameters with the same name, or two initializations of the same variable. And if we don't allow renaming parameters is augmentations, then there can also be at most one positional initializing formal at each position.

you cannot tell what is an initializing formal or super parameter without visiting all augmentations.

You also cannot tell what Foo(int x) uses x for unless you find the initializer list that does augment Foo(int x) : this.banana = x. I don't see a difference between that and augment Foo(this.x);. Or if you have Foo(int x) : this._(x);, it doesn't say what Foo._ does with the variable. It's always possible to write code that is hard to read.

A caller shouldn't know what a constructor does with its arguments. That's an implementation detail. The person writing the class should know, but they also wrote the augmentation, so presumably they know why they wrote it the way they did.

I would prefer to instead keep the existing behavior, which makes the kind of the parameter (regular, initializing formal, super) something which must be consistent across all augmentations and the base declaration.

That precludes adding a field and initializing it with an initializing formal.

Given

class C {
  @primary
  C(final int x, int y);
}

we could generate

augment class C {
  final int x;
  int y;
  augment C(this.x, this.y);
}

That won't work if the original declaration had to final int this.x. (Or it would, but it would be awkward, and the analyzer would tell you that the final is unnecessary).

If we make augmenting declarations modular, we give them flexibility to take responsibility, without every other constructor needing to know what they do. Each constructor just does what it wants to do ("I need to initialize this field, so I add an initializer list entry, or an initializing formal", and nobody else needs to know what it does, or coordinate about it.)

An augmentation cannot change the signature of a constructor, but what kind of parameter it has is an implementation detail, not part of the signature.

[jakemac53]

Ultimately, I just don't really see what this complexity buys us. And I think it significantly increases the cognitive load for users without enabling anything that is actually necessary?

Yes, people can write bad code no matter what, of course. But this seems like a feature that is incredibly difficult to use in a way that isn't confusing (for super positional parameters at least). Positional super parameters are already bad enough (I have personally written bugs as a result of the names not having any meaning), and I really don't want to make them even more confusing/error prone.

The fact that the ordering explicitly matters here also makes macro application ordering matter in ways it might not otherwise, and it also could make macros less composable as a result (if they try and convert positional parameters to super parameters, now that gets very weird if several macros are involved).

A caller shouldn't know what a constructor does with its arguments. That's an implementation detail. The person writing the class should know, but they also wrote the augmentation, so presumably they know why they wrote it the way they did.

I think the design is problematic for the owner of the code. As an owner of the code, I think it is useful to be able to easily determine what kind of parameter each parameter is, when looking at a class that I wrote N years ago, or some other person wrote N years ago. There should be no assumption that I actually have any context left over, even if I did write it.

Each augmenting constructor can append to the the already constructor super-argument-list by using super-parameters, and they do so independently of each other.

That seems incredibly confusing? It also allows something you can't even do without augmentations which is pretty weird imo.

That precludes adding a field and initializing it with an initializing formal.

The current proposal is very much intended to allow this. You can use an initializing formal without a field present, as long as a field by that name is eventually added via augmentation. (but yes, the specific mechanics of how it should work and the ordering of initialization are not described).

[lrhn]

That seems incredibly confusing? It also allows something you can't even do without augmentations which is pretty weird imo.

Probably confusing, but not something you can't do otherwise using initializer list entries and explicit super constructor arguments. The initializing formal and support parameters are really just shorthands to avoid having to write the same name 2-3 times.

It's possible to desugar a stack of augmenting constructors into a single normal constructor with no initializing formals and no super parameters.

I really don't consider (this.x) as anything but a shorthand for (type x): this.x=x, and don't see any reason to treat it differently.

I do see that using positional super constructors in augmenting constructors makes them order dependent. That is a problem, and could be reason enough to not have them.

(Named ones are easier, you are either the one who passes that argument, or you are not. No two can both do it, order doesn't matter.)

So maybe no positional super parameters. That's fairly restrictive, since we have no way to add more positional arguments then, there is no explicit way to pass a single argument, like there is to initialize a single variable in the initializer list. Super parameters are not like initializing formals.

(I could see a wish to be allowed to add named parameters to constructors. That way a macro can add a field, and a parameter to initialize it, and possibly a corresponding argument in subclass super constructor invocations.)

[jakemac53]

I was chatting a bit with @munificent about another approach to this, which is essentially to always desugar initializing formals and super parameters locally into the constructor declaration (not the merged definition) where they appear.

For initializing formal parameters I think that is quite straightforward and ends up with an equivalent to what the current process above describes. You have some control over when exactly in the chain that initialization happens, and a single augmentation can both add a field and "convert" a parameter to be an initializing formal that initializes it.

For super parameters, it means you can only have any super parameters in a single declaration of all the declarations that make up a given constructor definition, and similarly if any declaration has super parameters, that is the only one which is allowed to have an explicit super call in it's initializer list. Otherwise, you end up with multiple super calls in the initializer list which is not allowed, because any constructor containing any super parameters has an implicit super call. This alleviates the ordering concerns because the entire super invocation can be derived from a single declaration.

[lrhn]

As usual, I'll argue strongly against using desugaring as a specification tool. The "merged declaration" doesn't exist, it's just a name for the meaning of the combination of the declaration and all its augmenting declarations.

Give semantics to the original syntax, that the program actually contained. (If those semantics happen to be equivalent to something not using the same feature, then that's probably just a sign that the new feature will be easy to understand as "doing the same things as ...", making desugaring a teaching tool instead. Or an implementation strategy.)

So the semantics of invoking a constructor with declaration D0 and augmenting declarations D1...Dn in source order is defined directly in terms of D0...Dn. It can use any number of properties derived from D0...Dn, or any prefix of them, as accumulated information that makes the individual steps easier to express. (But not in terms of assigning semantics to a different Dart syntax that D0...Dn can somehow be transformed into. Syntax that does not occur in the source program is just not well-defined.)

Anyway, allowing only one declaration to specify a super-constructor invocation, including using any super-parameters, does remove the ordering problem. We could probably allow other augmentations to specify named super parameters, since the order of those are irrelevant, as long as there is at most one for each name.

I don't usually like distinguishing positional and named parameters, but we already do for super-parameters, where we do not allow both positional super parameters and explicit positional super-constructor arguments in one constructor. Here we'd extend that to positional super parameters in a different declaration. Or we can say that at most one declaration can add positional parameters, and if that is the same one that introduced the super-constructor invocation, it can also only use either explicit or super-parameter arguments.

It still leaves the possibility of a prior augmentation adding super-parameters, and a later augmentation declaring a super-constructor invocation, if ordering gets messed up. I guess it'll just be an error. Having any super-parameters without a prior or current declared super-constructor invocation, means that the super-constructor gets locked into the unnamed constructor.

So:

• At most one declaration can declare a super-constructor invocation.
• If any constructor has a super-parameter, and no explicit super-constructor declaration, and no prior constructor has declared a super-constructor invocation, that implicitly declares a super-constructor declaration of super().
• If a constructor declares an explicit super-constructor invocation, it's a compile-time error if that invocation has an explicit positional argument, and the constructor also has a positional super-parameter.
• It's a compile-time error if any declaration has a positional super-parameter, and the accumulated super-constructor invocation of the prior constructors has any positional arguments.
• It's a compile-time error if any declaration has a named super-parameter, and he accumulated super-constructor invocation of the prior constructors has a named argument with that name.
• Otherwise an augmenting declaration adds its super-parameters as arguments to the accumulated super-constructor invocation of its augmented declaration, to produce the accumulated super-constructor invocation of the declaration itself.

In short:

• First super-parameter or explicit super-constructor invocation declares the super-constructor. If no explicit constructor, then implicitly super().
• Any declaration can add named arguments to the super-constructor invocation, but at most one per name.
• At most one declaration can add positional arguments to the super-constructor invocation. (And if it's the introducing one, as usual it can't use both explicit and super-parameter positional arguments.)

If two different macros both want to add positional arguments, they're probably stepping on each other's toes.

Each declaration can also use any argument to initialize an instance variable as an initializing formal. The semantics of that is just to initialize the variable with the same name, and as usual, no variable must be initialized twice. That's entirely local to the single declaration, nobody else needs to know.

I don't remember exactly where we ended on parameter naming, but I think it was "no renaming". In that case, I'd make it:

• It's a compile-time error if a base declaration positional parameter with declared name b, and an augmenting declaration's corresponding positional parameter has declared name a, neither a nor b is _, and a ≠ b.

That is, a declaration can choose to not name a parameter it doesn't care about. If the based declaration didn't name a parameter, then any augmenting declaration can name it whatever it wants to, if it needs to refer to it.

And the same for type parameters. (Why someone would declare int foo<T, _>(...) => ... I don't know, but if they do and an augmenting declaration needs to refer to the second type argument, it can give it any name.)

[jakemac53]

While I don't see anything technically wrong or unsafe, I also really don't see this ability to morph a parameter into a super parameter as being a useful feature, outside of the augmenting declaration which actually adds the super call. What problem is this feature trying to solve?

For macros, I don't intend to expose this ability at all anyways. Currently macro authors are not in control of writing the function declaration when augmenting an existing function (or constructor), and I think that is a good thing. When editing the definition of something, you only have access to edit the body/metadata/initializer list, and that makes a lot of sense. Macros are just expected to write out the more verbose initializer list entry to initialize variables or pass them on to the super constructor.

Additionally if a macro wants to add an extends clause, then I expect that macro to be in charge of writing a fully valid super call. It seems very far fetched to me to envision a scenario where two macros are coordinating to create a fully valid super call, when all the parameters had to be defined for both already anyways.

I am still pretty opposed to this idea of allowing even named super parameters to be defined (almost) anywhere, we could always add it later if enough people ask for it. But it seems like unnecessary complexity to me, which is harder to walk back than add in later.

[lrhn]

Ack. I'm trying to give features that correspond mostly to what you can do in a single declaration, even if you spread it across multiple augmenting declarations. Mainly because restrictions that aren't necessary feel arbitrary.

That doesn't mean it's a good idea to use the features, and we may be better off with a more restricted starting point. As you say, we can always allow more.

So OK, let's say that only one declaration can introduce a super-constructor invocation, and no other declaration can modify that. The first one to have a super parameter or super constructor invocation is that. Which generally means no modification, only additions (of metadata, initializer list entries, including indirectly as initializing formals) or continuations of the body.

[munificent]

Oh boy is this issue long. 😅 @lrhn, is there anything here we still need to do given now the augmentations proposal is now much more limited around constructors?

[munificent]

I'm going to go ahead and close this because I think with the now pared-down proposal, there isn't anything in here we need to keep.