Does implicit null count as an initializing expression?

[eernstg]

Here's a corner case which is perhaps somewhat ambiguously specified. Thanks to @sgrekhov for bringing up this issue!

Consider the following program:

// --- Library augmentation 'augment.dart'.
augment library 'main.dart';

String f(String? s) => s ?? 'Default';

augment String? sq = f(augmented); // OK?

// --- Library 'main.dart'.
import augment 'augment.dart';

String? sq; // Implicit initializer `= null`.

The augmentation specification has the following rule:

It is a compile-time error to use augmented in an augmenting field's initializer if the member being augmented is not a field with an initializer.

So the question is whether or not the implicit initialization to null counts as an initializing expression? I'd recommend that it does not, and the line marked OK? is a compile-time error, for simplicity.

However, it could also be argued that some macros can be simpler if they are allowed to use augmented as shown above both in the case where there is an explicit initializing expression and in the case where the variable has an implicit initializer with the value null. The argument would be "It's useful, so why not?".

[Update: At this time I think "It's useful, so why not?" wins. I'd recommend that we use the semantic criterion "this variable has an initial value" rather than the syntactic "this variable declaration has an initializing expression". So OK? above should be OK!.]

@dart-lang/language-team, WDYT?

[lrhn]

Yes and no! (Aka, "Why not both?", or "It depends!")

There is no good reason to disallow the example program here, it works. Which means that the rule "can't call augmented on variable with no initializer" is too strict. It should probably be allowed if the variable's type is nullable, in which case augmented evaluates to null.

That is:

If augmented refers to a variable declaration (as defined by a declaration and a number of prior augmentations) with no initializer expression, and the variable's type is nullable, augmented evaluates to null. If the variable's type is not nullable, then it's a compile-time error.

EDIT: The following is handled by the spec, and is not a problem.

But what is the type of the variable, if it has no declared type, yet?

final x = 42;
augment final x = augmented.toRadixString(16) * 1.5;
augment final num x;

Is this valid at all? If not, why not?

If valid, what is the static type of augmented here? If it's int, then it's because we have inferred a preliminary type for x, because the type of augmented is the type of the declaration. If it has one? (I definitely don't expect final num x = 42; augment x = augmented.toRadixString(16).length; to work. If the variable has a declared type, then that is the type of augmented. If it doesn't, it's type is ... the static type of the most recent initializer expression. If none, it's Object??)

We need to decide what the type of augmented is, at every augmentation, and how it's derived. And whether this is valid (and if not, why not):

final x = 42;
augment final x = augmented.toRadixString(16);
augment final x = StringBuffer(augmented);
augment final x = (augmented..write("!Banana")).toString();
augment final x = augmented.length;
augment final int x;

EDIT. End of part that you can ignore.

We probably also need to check if it works with late. Because it's weird enough without augmentations.

late int x;
augment int x = augmented + 1;

This looks like it could be valid, because it's a non-local late variable, so it's never "defintely unassigned".

I'd go with the same rule as above: The agumentee variable has no initializer expression and is not nullable, so it's an error. If it had been nullable, the augmented would have a nullable type and value null. This isn't about the variable, the variable doesn't exist yet, it's a about accessing the initializer expression.

If there is an initializer expression, an augmentation can can refer to that expression, even more than once.

late final int x;
augment int x = 21;
augment int x = augmented + augmented;

will define a variable equivalent to late final int x = 21 + 21;

[lrhn]

@eernstg pointed me to this part of the spec

If the variable declaration in the original library does not have a type annotation, then the type is inferred only using the original library's initializer. (If there is no initializer in the original library, then the variable is inferred to have type dynamic like any non-augmented variable. This ensures that augmenting a variable doesn't change its type. This is necessary to ensure that macros running after signatures are known can't change the signature of a declaration.

So, phew! Every variable's type is defined by its initial declaration, augmentations can only change modifiers and values.

Also, please change dynamic to Object? if at all possible. (I know backwards compatibility, but if there is any augmentation, we can make the type of augmented be Object? in augmentations, and maybe even make it the type of the entire declaration.)

Without macros there should be no issue, the program must be complete, so we can do type inference on each expression. For macros, we're at step ... 3?... when we have done type inference, which is also where we can write bodies and refer to augmented, so that seems fine too.

[sgrekhov]

If augmented refers to a variable declaration (as defined by a declaration and a number of prior augmentations) with no initializer expression, and the variable's type is nullable, augmented evaluates to null. If the variable's type is not nullable, then it's a compile-time error.

What about late variables?

late final String v;
augment late final String v = "Augment: $augmented"; // Expect runtime LateInitializationError?

[lrhn]

late final String v;
augment late final String v = "Augment: $augmented"; // Expect runtime LateInitializationError?

That was my first thought too, but I think it's wrong. The augmented does not refer to the parent variable, it refers to the parent declaration's initializer expression. That expression itself isn't late, that's a property of the entire variable.

I would make this a compile-time error because augmented refers to an absent initializer expression and at a type that isn't nullable.

Had the declaration been

late final String? v;
augment late final String? v = "Augment: $augmented"; // Expect runtime LateInitializationError?

I'd allow it to declare a variable equivalent to:

late final String? v = "Augment: ${null}";

If the augmented had referred to the entire variable, then the augment expression would initialize the variable, and

late final String x = "banana";
augment late final String x = "($augment)";

would run into an error when it tries to set x the second time, after it was already set to "banana".

There is definitely only one variable, and if it's late final, it will only be initialized once.

[eernstg]

Interesting!

If augmented denotes the initializing expression then it should be fine to evaluate it multiple times:

int counter = 0;
late final String v = 'Counter: ${counter++}';
augment late final String v = "Augment: $augmented, $augmented"; // 'Augment: Counter: 0, Counter: 1'.

We could also say that augmented is a reference to the implicitly induced getter of the variable (late or not), and multiple usages will just call that getter multiple times. That seems less crazy to me. ;-)

There is definitely only one variable

I tend to think of augmentations of functions and variables as introducing new entities (fresh, private name) and implicitly using that new entity when evaluating augmented.

How, otherwise, do we explain the semantics of augmented(42) in an augmentation of a method or function? We can't say "copy paste the code from the augmented declaration in here".

[lrhn]

You can indeed repeat the augmented. I included augment int x = augmented + augmented; above to showcase precisely that.

Let's check the specification:

• Augmenting a variable with a variable: Augmenting a variable with a variable only alters its initializer. External and abstract variables cannot be augmented with variables, because they have no initializer to augment.

Since the initializer is the only meaningful part of the augmenting declaration, an initializer must be provided. This augmenting initializer replaces the original initializer. The augmenting initializer may use an augmented expression which executes the original initializer expression when evaluated.

plus some compile time errors including:

It is a compile-time error if:

• An augmenting initializer uses augmented and the augmented declaration is not an initializing variable declaration.

This issue started because of that compile-time error. As written it's clear. I've argue that the augmentation should be allowed to use augmented if the augmented declaration is not an initializing variable declaration, but its type is nullable, because then the augmentation (possibly inserted by a macro) doesn't have to distinguish between having an initializer or defaulting to null. (That should be possible, we don't have to give an error until doig type inference.)

About the initializer expression, the spec is equally clear:

Since the initializer is the only meaningful part of the augmenting declaration, an initializer must be provided. This augmenting initializer replaces the original initializer. The augmenting initializer may use an augmented expression which executes the original initializer expression when evaluated.

That clearly says that the new initializing expression replaces the existing initializing expression, and occurrences of augmented in the replacing expression correspond to execution (should be "evaluation") of the augmented initializer expression. Which does mean evaluating it more than once if augmented occurs more than once.

That can be a problem, if someone writes an initializer expression expecting it to only be evaluated once, and then a macro evaluates it twice, say to do switch (augmented) { >= 0 => augemented, _ => throw StateError("Must not be negative: ${augmented}")}. The answer to that would be "then don't do that".

Alternatively, we could say that initializer expressions are evaluated once and cached, if augmented occurs more than once in an augmenting initializer expression. That would be special to variable initializers. I don't want getters to cache. If someone wants that, they can use a pattern switch (augmented) { case var augval => ... augval ... augval ...}. But they can do that for initializer expressions too. So maybe that should just be our position: Don't use augmented more than once in initializer expressions. You can, but you shouldn't.

(It is wrong that the initializer is the only meaningul part, one can want to add annotations too.)

So the current spec is clear, the question is whether there is a better behavior.

I haven't been able to find one, other than possibly caching the value. Any attempt to make the augmented in the initializer expression being defined in terms of the getter of the augmentee risks needing multiple "is initialized" flags for a single late variable, which is defintely something I want to avoid.

[davidmorgan]

Is there anything left to do on this one?

[eernstg]

I believe there is consensus on the following:

With respect to augmented, an original (non-augmenting) declaration D of a non-late variable with a nullable type and no initializing expression is treated as having the initializing expression null.

This implies that an augmenting declaration that augments D can use augmented, and it has the value null. For the case where the variable is late, see https://github.com/dart-lang/language/issues/3735.

[lrhn]

With respect to augmented, an original (non-augmenting) declaration D of a non-late variable with a nullable type and no initializing expression is treated as having the initializing expression null.

(And add "non-abstract, non-external" too.).

I agree with "treated as initializing expression null" can work.

Generally, we only check whether a non-late variable has an initializer expression at the end of augmentation application. That's where it's an error if it has none and the type is non-nullable. It makes sense to also say the "and behave like null if nullable" there. I guess it also applies to any intermediate augmentation application result if the augmenting variable declaration initializer uses augmented. At that point, it's an error if the augmented declaration is non-nullable and not initialized, and it's treated as null if it's nullable and not initialized. So it is consistent - the result of application must be valid at any step where it is potentially read, so after all augmentations, and before any augmentation using augmented.

Asking whether there is an initializing expression is where it gets tricky. I'd say the answer is "no". It's just that any behavior which expects to evaluate an initializing expression will evaluate to null instead.

[eernstg]

Note that the implementation (in the analyzer, the CFE isn't there yet) does behave as if the proposal in this issue had been accepted:

int? i;
augment var i = 4 + (augmented == null ? 2 : -2);

This is accepted by the analyzer, which implies that int? i; is treated as int? i = null;, and augmented can be used in the augmenting declaration.

[eernstg]

@dart-lang/language-team, we're currently introducing co19 tests where this behavior is being tested. The analyzer behaves in the way that is proposed in this issue (that is: the implicit null does count as an initializing expression).

The spec has the following:

It is a compile-time error to use augmented in an augmenting field's initializer if the member being augmented is not a field with an initializer.

Note that the spec does not specify an exception for the case where the type of the variable is nullable.

The following example would be OK according to the proposal in this issue and according to the analyzer, but an error according to the rule above:

int? i;
augment var i = 4 + (augmented == null ? 2 : -2);

It seems wasteful to change the analyzer if the spec will soon be changed such that the current behavior is correct. Can we settle the issue?

[lrhn]

The change to that phrase, to allow using augmented with no augmented initializer expression, would be:

It's a compile-time error to use augmented in an augmenting variable declaration's initializer if the variable declaration being augmented does not have an initializer expression and its declared type is not nullable.

Evaluating augmented inside an augmeting initializer expression evaluates the augmented variable's initializer expression and evaluates to that value, if the augmened variable has an initializer expression, otherwise it evaluates to null (in that case the augmented variable's type is nullable, so evaluating to null is sound).

[jakemac53]

I would be fine with the change as described in the previous comment.

[eernstg]

Consider the relation to https://github.com/dart-lang/language/issues/3977.

In that issue I'm arguing that we should not report an error for int i; in the following context (at top-level):

int i;
augment int i = 1;

The point is that the merged declaration satisfies all requirements, and hence it isn't helpful to report a compile-time error on the introductory declaration based on the missing initializing expression.

In other words, we're only checking that the variable is initialized after merging.

However, if we do the same thing for a variable declaration whose type is non-nullable then we'd get the following:

int? j;
augment int? j = augmented ?? 1; // Error at `augmented`.

The error occurs because the augmented declaration does not have an initializing expression. After merging the declaration of j does have an initializing expression, but augmented refers to the augmented declaration, which in this case is the introductory declaration.

Do we have a compelling reason to treat those two cases differently? Otherwise I'd recommend that we treat the nullable variable declarations just like other variable declarations, which would make augmented an error above.

[jakemac53]

Do we have a compelling reason to treat those two cases differently?

Only that one is valid on its own and the other is not. But, in the "valid" case there is no use for the augmented expression, it is equivalent to just a hard coded null, so from that perspective I would be OK with making both uses of augmented an error.

I do not see any mention in the spec of an implicit null initializer. So we could instead reasonably say there is no initializer to refer to. Or, if I missed it an there is an implicit initializer specified, we could say that is only added after all augmentations have been merged, if there is no initializer still.