User defined type promotion

[MarvinHannott]

The user should be able to write special methods that have the same properties like the is operator. The syntax could look like this:

extension Nullable<T> on T? {
  Promotion<T> get isNotNull => this != null;
}

The return type Promotion<To> signals to the runtime to what type typeof(this) gets promoted to, if To is a sub-type of typeof(this). If it is not, that should be a compile-time error. Promotion should be an abstract class that acts like a marker, but actually is a bool so that isNotNull could be called in any bool-context like a function call. Also, isNotNull actually returns bool. It's only a compile-time construct to enable type promotion inside if-statements. Promotion can also only ever be used as a return type, so no variable could ever be of type Promotion. We don't actually want to track the promotion throughout the program.

Let's clarify all this in an example:

int? a = 0;
if(a.isNotNull){
 // runtime knows that `a` is not `null`
}
bool p = a.isNotNull; // Fine!
void takeBool(bool b){}
takeBool(a.isNotNull); // Also fine!
Promotion<int> i = a.isNotNull; // Not allowed! 

A more complex and useful example: Let's say there is a class Result<T,E>, and there are two sub-classes of that, Ok<T,E> and Err<T,E>. I only want to expose Result to the user and not bother them with the sub-classes. But that's not possible because the user needs to check whether his Result is Ok or Err like

Result<int,String> r = Result.ok(2);
if(r is Ok<int,String) {
...
} else if(r is Err<int,String>){
...
}

That's very bothersome and taxing on the user. It would be much nicer if he could just call if(r.isOk) and if(r.isErr) and get the same result. This could be implemented like

abstract class Result<T,E>{
  Promotion<Ok<T,E>> get isOk => this is Ok<T,E>;
  Promotion<Err<T,E>> get isErr => this is Err<T,E>;
} 

What's not so clear to me at this point is whether the compiler should just assume that a promotion is valid. There are circumstances where such a behavior would be useful, but it may also introduces undefined behavior.

[lrhn]

What I see here is a request to allow the type promotion algorithm currently running internally in a function body, on local variables, to work better across function calls. That is, allow abstraction, where a sub-expression of a function body is turned into a new function and a call to that function. Since promotion information is only accumulated inside a single function body, there is not way to pass that information into the new function, or get information back out.

With something like this, it must necessarily be part of the method signature that its boolean result signifies something about its arguments (including this). I wouldn't restrict it to this. So we'd annotate the boolean return type. Let's strawman it as bool<this is Ok<T, E>> where this could also be one of the arguments.

What does that actually mean? If the result is true, then the value passed as argument (or this) is definitely of the guaranteed type. That much is clear. If the result is false, is it definitely not? Or could we just not say? (Likely the latter, will come back to that).

To ensure the promotion is linked to the return value, when analyzing the function implementation, we must know statically which return statements return true, which return false, and which perhaps returns a dynamically determined boolean that we know matches the check. (So you can write bool<x is int> check(Object x) => super.check(x); and inherit the promotion from the super-check). And we must know, for certain, at all return points whether x is an int or not, at least unless we return a boolean that already carries that information.

This already happens locally inside functions - a bool result remembers which promotions it implies. If something happens later which could change those variables, the bool result is "demoted" to no longer imply the promotion. This syntax just makes it explicit and allows it to traverse function calls.

Can such a boolean be stored in a variable? Would that mean that all booleans can carry promotion information

Let's assume we can write the annotations explicitly on any boolean. It's completely static metadata, not something reified at runtime, because all it affects is static promotions.

bool<x is int> foo(Object x) {
  bool<x is int> isInt = checkInt(x); // Promotes x if true, not if false.
  something(other, isInt); // Must not change `x`. If so, `isInt` is "demoted" to plain `bool`.
  return isInt; // Valid if `isInt` still carries promotion information.
}

When the promotion information is part of them function signature, we need to know how it affects function subtyping (including method overriding).

Dart uses normal subsumption subtyping, so a value of a subtype must safely be usable everywhere the supertype is expected.

A function signature bool<x is int> Function(..., Object x, ...) can be used anywhere a bool Function(.., Object, ...) is expected. You can ignore the promotion information. So bool<x is int> <: bool.

Now compare bool<x is int> Function(..., Object x, ...) and bool <x is num> Function(..., Object x, ...). This is where it matters whether a false boolean implies that the value is definitely not of the type. Let's assume that is not the case. A false value means nothing. (We'll get back to that!) In that case, the former function type can be used anywhere the latter can, because it accepts the same arguments, and the result of the former implies the result of the latter (if we know x is definitely an int, we also know it's definitely a num). So promotion implications are covariant. (Not surprising, it's like you actually returned the promoted value, you just don't have to because the caller already had the value since the only thing you can return promotions of are the arguments and the receiver.)

That could work.

However, why only carry promotion information on true, and not false. We do that all the time in type promotion, like x == null promotes to the non-null type on the false branch. So, maybe make the promotion information be a value-implication:

bool<true => x is int>

(Grammar really sucks, too many > in there. Solution ... add more: bool<<true => x is int>>! Or not.)

Then it'd be possible to do:

bool<true => x is Future<T>, false => x is T> isFuture<T>(FutureOr<T> x) => x is Future<T>;

And then we could also carry promotion information in other types, as long as they have primitive equality, like enums:

Distinction<Distinction.nul => x is Null, Distinction.future => x is Future<T>, Disinction.value => x is T>
  distinguish(FutureOr<T>? value) => 
     value is Future<T> ? Distinction.future : value is T ? Distinction.value : Distinction.nul;

(It's important whether the contract is that a value equal to the implication value means x is promoted, or the value has to be identical. That determines which check you must make to get the promotion. We probably need to require identical, but can loosen it to == on a few types, like bool, num and enums.)

We may also want to provide more than one promotion on the same boolean:

bool<true => s is int & y is int> checkBoth(num x, num y) => x is int && y is int;

Would we allow such annotated booleans as arguments too?

void doIf(bool<true => x is int> isInt, Object x) {
  if (isInt) print(x + 1);
  print(x);
}
...
  var isInt = x is Int;
  something();    
  doIf(isInt, x);

The following would not work, though

class Box {
  Box? value;
}

int<0 => x is Null, default => x is Box> boxDepth(Box? x) => x == null ? 0 : boxDepth(x.value) + 1;

because we don't have enough information to know that boxDepth(x.value) cannot return -1. We need range-information on the return to do that. So, if we could also add another constraint on the return value, like int<..., _ >= 0> (too many >s!), then we could actually infer that the boxDepth was correct.

But then, we'd have introduced an awful lot of complication into the type system, basically a generalized contract format, which the compiler would have to be able to understand and propagate correctly.

Trying to propagate type information around along with values risks getting into dependent typing, especially if we generalize it. Would we be able to do:

void treatJson(JsonKind<
    JsonKind.number => value is num, 
    JsonKind.string => value is String, 
    JsonKind.bool => value is Bool, 
    JsonKind.nul => value is Null, 
    JsonKind.list => value is List<dynamic>, 
    JsonKind.map => value is Map<String, dynamic>> kind,
    Object? value) {
  switch (kind) {
    case JsonKind.number: handleInt(value); break
    // ...
 }
}

where the type of the second parameter depends on the value of the first. (But, you do have to check the value of the first to get promotion, and the caller has to guarantee that it's true.)

The effect is really to allow abstraction. We can now pluck out a part of a function body, put it into a separate function, and still get the full type promotion that we get locally, because we can send the promotion information along with the arguments. That's "nice", but it also means that we expose the internal promotion information in a way that might make it hard to do better promotion in the future. (Or not, if better promotion still implies the earlier promotion, then it's subtype compatible.)

Is it worth the effort? Not convinced. It's a very complicated feature for something that has workarounds.

What I do today is to let the function return the promoted value or null, like the existing Result class, then rely on built-in null aware operations.

Then you can do

  r.asValue?.doSomething() ?? r.asError!.somethingElse();

[MarvinHannott]

@lrhn Wow, thanks for explaining my own request better than I did 🙂. I'd like to respond to some of your points, many of which I didn't think so deeply about.

I wouldn't restrict it to this.

Well, I thought that to be a good idea so that no additional syntax would be required. Also, I couldn't really think of a use case where such a function should take arguments.

Can such a boolean be stored in a variable? Would that mean that all booleans can carry promotion information

That's not necessarily what I had in mind. I thought that the return value is just a plain boolean, not carrying anything, except inside an if.

When the promotion information is part of them function signature, we need to know how it affects function subtyping (including method overriding).

That's exactly what I wanted to avoid. It shouldn't affect function sub-typing. There shouldn't be any typedef like bool<x is int> isInt(Object x) or a function argument like Promotion<int> isInt(). But my idea was that the promotion method could be passed anywhere where bool Function() is asked.

To be honest, I didn't think about method overriding, but I don't think such a method should be overridden. I can't really see why it would. So just not allowing it 😉.

Or as an even better alternative: only allow promotion methods as extension methods.

However, why only carry promotion information on true, and not false. We do that all the time in type promotion, like x == null promotes to the non-null type on the false branch.

Sure, that's something I didn't think about. That at least works for two possible states, but not more. One could use your notation, or just add another generic type parameter to Promotion<T,A>. However, to generalize on that, we'd need algebraic data types, which would make all this obsolete, I guess.

And then we could also carry promotion information in other types, as long as they have primitive equality, like enums:

Sounds complicated.

We may also want to provide more than one promotion on the same boolean:

Is that really useful? Sounds also complicated.

Would we allow such annotated booleans as arguments too?

That's not what I had in mind, but I wouldn't complain. Tracking promotion beyond function calls sounds also pretty complicated, but also pretty useful.

What I do today is to let the function return the promoted value or null, like the existing Result class, then rely on built-in null aware operations.

Sure, that works.

[mateusfccp]

This is something I've been wanting a lot. It would be really nice to be able to specify promotions.

A very common case is, for example, using the isBlank from quiver package. Although isBlank already checks if the given String? is null, it's not promoted, so I have to do if (string == null || isBlank(string)), which is redundant.

However, I am pretty sure this is not an easy issue to solve...

[MarvinHannott]

@mateusfccp That's actually a cool use case.

Alternatively, you could always use the ! operator...

[mateusfccp]

@mateusfccp That's actually a cool use case.

Alternatively, you could always use the ! operator...

I never use !, because it's unsafe and you can always promote. Also, using it sequentially will make it check if it's null in runtime, which is avoided when you promote. Thus, I have 0 reason to use bang operator...

[MarvinHannott]

extension on String {
  final bool isBlank() => this is String && this.isNotEmpty;
}

The final is there so it is not overriden by a subclass. The compiler can infer from the body of isBlank that this is String. It just seem logical here without going into implementation details of the compiler as to what is happening.

Ehm, extension methods aren't overridden by sub-classes.

This kind of inference might look doable in your little example, but what you are actually asking for is that the compiler could infer object types from every function call, no matter how complex that function is or how many recursions that entails. That's just not realistic, and it could even be confusing to the programmer.

I had a fear that this might become complicated, that there could be sort of a meta stuff going on where f(g(h(x))) promotes x to y then g promotes it to z, etc. but I don't think it's founded

What? This is crazily complicated. And the thing you think is unfounded is exactly what you are asking for.

[cedvdb]

I removed my previous post because it wasn't thought through and was misleading on my goals.

Ehm, extension methods aren't overridden by sub-classes.

The original example in the other thread was :

abstract class TaskStatus { 
  final bool isCompleted() => this is TaskCompleted;
}

I didn't think this through either, this might still be unrealistic