Weird behaviour of `for-in` type inference

[FMorschel]

Consider this repro:

void bar(List<int>? list) {
  var /*List<int>*/ newList = list ?? [];
  for (var /*dynamic*/ value in list ?? []) {}
}

I was trying to understand why these differ when the syntax is the same. I also tried doing this:

void bar(List<int>? list) {
  var newList; // dynamic
  for (var /*int*/ value in newList = list ?? []) {}
}

For some reason, it does infer it correctly. This also would work with:

void bar(List<int>? list) {
  var /*List<int>*/ newList = list ?? [];
  for (var /*int*/ value in (list ?? []).toList()) {}
}

Or:

void bar(List<int>? list) {
  var newList; // dynamic
  for (var /*int*/ value in [...?list]) {}
}

Is this explained anywhere? Can we improve this inference somehow? I know the last workaround is not that hard (or adding <int>), but this is inconvenient and confusing since the same syntax works fine in other places.

CC @eernstg @lrhn

[eernstg]

void bar(List<int>? list) {
  var newList = list ?? [];
  newList.expectStaticType<Exactly<List<int>>>;
  for (var value in list ?? []) {
    if (1 > 2) value.argleBargle;
  }
}

void main() {}

typedef Exactly<X> = X Function(X);
extension<X> on X { X expectStaticType<Y extends Exactly<X>>() => this; }

The declaration of newList has its declared type inferred by the initializing expression, which is again inferred in the empty context (_). This means that [] is inferred in the context which is the static type of list except the ?, yielding <int>[]. This means that newList has type List<int>.

However, the context type for list ?? [] in the for-in statement is Iterable<_>, which means that [] will get the context type Iterable<_>, yielding List<dynamic> (it should actually be List<Object?> but that's a fix which hasn't landed yet). Consequently, value gets the element type of the iterable (dynamic, for now).

[eernstg]

Correction, the type argument of the iterable is actually already inferred as Object?:

X whateverYouWant<X>() {
  print(X);
  throw "Can't really return an arbitrary `X`";
}

void main() {
  for (final v in whateverYouWant()) {} // Prints 'Iterable<Object?>'.
}

[FMorschel]

Oh, so this would actually be:

• https://github.com/dart-lang/language/issues/4516 ?

I'm not sure. If it is, you can probably close this. If it isn't, is there an issue for this already?

[eernstg]

It's not exactly the same situation. #4516 is concerned with the situation where the ?? expression has an empty context, the left hand operand of ?? has a type T, and the right hand operand can't be inferred to have a type which is a subtype of T. In that case we basically give up and behave as if the right hand operand had the empty context.

In this issue we do have a context type (Iterable<_>), and the types are both List<...>, which means that it would be possible for the right hand operand to be inferred with the same type argument, except that we won't use the left hand operand type as the context type for the right hand operand when the ?? expression as a whole has a context type (no matter what it is).

[eernstg]

I guess we could play around with an idea like the following:

With a for-in statement of the form for (W v in iter) where W is final or var (that is, when v does not have a type annotation), the context type of the iterable iter is _.

This would allow iter to obtain the "most natural type" (because there is no context type), and in particular, the left hand operand e1 of ?? for an iter of the form e1 ?? e2 would be allowed to influence the inference of the right hand operand e2. This would make sense for the example in this issue. I'm not sure, but it might work fine more generally.

[FMorschel]

Wouldn't it be the same as the example above:

void bar(List<int>? list) {
  var newList; // dynamic
  for (var /*int*/ value in newList = list ?? []) {}
}

If it is, it should be fine, I think.

[eernstg]

Yes, changing list ?? [] to newList = list ?? [] in the for-in statement will also change the context type for list ?? [] to _, and the type of the assignment will then be List<int> (not dynamic, even though newList has type dynamic).

So that's a workaround which can be used to experiment with the effect of adopting this change.

[eernstg]

One thing will break:

void main() {
  for (final _ in {}) {} // Error!
}

The problem is that {} infers as a map when it occurs with context type _, but it infers as a set when it occurs with context type Iterable<S> for any S (including Iterable<_>). However, that's only an issue when the iterable is a collection literal with no elements, as soon as we have something like {1} or {1: "one"} there's no ambiguity and the set will be inferred as a set.

So this is probably not a problem, even though we can find a program that breaks.

[FMorschel]

This also breaks for some reason:

void bar([List<int>? list]) {
  var newList; // dynamic
  for (var value in newList = list ?? /*Set<dynamic>*/ {}) {}
  //                ^ for_in_of_invalid_type
}

It infers Object instead of Iterable here.

It would be great if we could try both approaches. The _ context by default, and if that fails, Iterable<_>, but I suspect this is not possible today. And would not solve the case above, still, but would not throw an error when the inference wouldn't work.

[FMorschel]

Now I have an even weirder question:

void bar([List<int>? list]) {
  for (var /*Object?*/ value in list ?? {}) {}
  for (var /*dynamic*/ value in list ?? []) {}
}

How can it be that the first case resolves to Object? while the second resolves to dynmamic? 🙃

[eernstg]

This also breaks for some reason:

This one fails because the standard upper bound of List<int> and Set<dynamic> is Object (see this topic which has lots of discussions about upper bounds).

You can help the standard upper bound algorithm by using a more general type for one of the operands such that it is a supertype of the other. You're basically saying to the algorithm that it should use Iterable. The following might be OK for your use case:

void bar([Iterable<int>? list]) {
  dynamic newList;
  for (var /*int*/ value in newList = list ?? {}) {}
}

If this is inconvenient then we could of course also do this: for (var value in newList = (list as Iterable<int>) ?? {}) {}.

This is not particularly pretty, but it's a different topic (so further comments should be added to the upper-bound topic that I mentioned previously).

Now I have an even weirder question:

Sure! ;-)

Here's a variant of the example:

extension<X> on X {
  X get show {
    print(this.runtimeType);
    return this;
  }
}

void bar([List<int>? list]) {
  for (var /*Object?*/ value in list ?? ({}..show)) {}
  for (var /*dynamic*/ value in list ?? ([]..show)) {}
}

void main() => bar();

In both cases the context type for the iterable of the for-in statement is Iterable<_>, which means that both {} and [] will be inferred accordingly (with type argument dynamic, according to the invocations of show).

Next, we're taking the standard upper bounds of List<int> and Set<dynamic> (that is, Object) and we're taking the standard upper bound of List<int> and List<dynamic> (that is, List<dynamic>).

The latter explains why the type of value is dynamic as indicated in the comment.

However, the former looks like a bug. I created https://github.com/dart-lang/sdk/issues/62200 in order to clarify that.

[eernstg]

The last remaining questions about this behavior have been clarified in https://github.com/dart-lang/sdk/issues/62200. The crucial point is that the type of an ifNull-expression (e1 ?? e2) with a context type schema S will be the greatest closure S1 of S in the case where the standard upper bound of the type of e1 and the type of e2 is not a subtype of S1.

More details here.