language
language copied to clipboard
dart-lang
Support method/function overloads
This has been discussed periodically both in the issue tracker and in person, but I don't think there's a tracking issue yet. (EDIT: Here is the original issue from 2011 - https://github.com/dart-lang/sdk/issues/49).
Now that we're moving to a sound type system, we have the ability to overload methods—that is, to choose which of a set of methods is called based on which arguments are passed and what their types are. This is particularly useful when the type signature of a method varies based on which arguments are or are not supplied.
Not in Dart 2. This is a significant change to the object model of Dart. Currently, a Dart object has at most one accessible member with any given name. Because of that, you can do a tear-off of a method. If you could overload methods, tear-offs would no longer work. You would have to say which function you tore off, or create some combined function which accepts a number of different and incompatible parameter signatures. It would make dynamic invocations harder to handle. Should they determine that method to call dynamically? That might cause a significant code overhead on ahead-of-time compiled programs.
I don't see this happening by itself. If we make a large-scale change to the object model for other reasons, then it might be possible to accommodate overloading too, but quite possibly at the cost of not allowing dynamic invocations.
We can certainly still have dynamic invocations: If you use the type dynamic explicitly and invoke an expression of that type then you will get a dynamic invocation, and it is an important part of the Dart semantics that we have enough information available at run time to actually make that happen safely (that is, we will have a dynamic error if the invocation passes the wrong number of arguments, or one or more of the arguments has a wrong type, etc).
Apart from that, even with the most complete static typing you can come up with, it would still be ambiguous which method you want to tear off if you do x.foo and foo has several implementations. So it's more about first class usage (passing functions around rather than just calling them) than it is about static typing.
@lrhn:
If you could overload methods, tear-offs would no longer work.
You already cannot tear off what users write instead of overloads, which is multiple methods:
class Foo {
void bar() {}
void barString(String s) {}
void barNumber(num n) {}
}
... so given that overloads would be sugar for that, I don't see it any worse.
@eernstg:
and it is an important part of the Dart semantics that we have enough information available at run time to actually make that happen safely
Is it being dynamically invokable a requirement? I don't think it is.
I'd heavily like to see a push for overloads in the not-so-distance future. My 2-cents:
(@yjbanov and @srawlins get credit for parts of this discussion, we chatted in person)
Proposal
Don't allow dynamic invocation of overloaded methods
... or limit how they work:
class Foo {
void bar() => print('bar()');
void bar(String name) => print('bar($name)');
void bar(int number) => print('bar($number)');
}
void main() {
dynamic foo = new Foo();
// OK
foo.bar();
// Runtime error: Ambiguous dispatch. 2 or more implementations of `bar` exist.
foo.bar('Hello');
}
If you wanted to be real fancy (@munificent's idea, I think), you could have this generate a method that does dynamic dispatch under the scenes. I'm not convinced this holds its weight (and makes overloading, which should be a great static optimization a potential de-opt), but it's an idea.
I realize this adds a feature that is mostly unusable with dynamic dispatch, but Dart2 already has this issue with stuff like reified generics.
Consider this very common bug:
var x = ['Hello'];
dynamic y = x;
// Error: Iterable<dynamic> is not an Iterable<String>
Iterable<String> z = y.map((i) => i);
Limit tear-offs if the context is unknown
Rather, if the context is ambiguous, then make it a static error.
void main() {
var foo = new Foo();
// Static error: Ambiguous overload.
var bar = foo.bar;
// OK
var bar = (String name) => foo.bar(name);
// Also OK, maybe?
void Function(String) bar = foo.bar;
}
... another option is have var bar = foo.bar basically generate a forwarding closure (a similar de-opt to the dynamic dispatch issue). Again, not my favorite, but I guess no more bad than code already being written.
Side notes
Let's consider how users are working around this today:
- Using
Objectordynamicwithischecks and optional arguments:
class Foo {
void bar([dynamic nameOrNumber]) {
if (nameOrNumber == null) {
print('bar()');
return;
}
if (nameOrNumber is String) {
// ...
return;
}
if (nameOrNumber is num) {
// ...
return;
}
}
}
- This works with dynamic dispatch
- This works with tear-offs
- This isn't very efficient, and it's very hard/impossible to create complex overloads
- You lose virtually all static typing
- Creating separate methods or constructors:
class Foo {
void bar() {}
void barString(String s) {}
void barNumber(num n) {}
}
- This doesn't work with dynamic dispatch
- This doesn't work with tear-offs
- This is the most efficient, but cumbersome and creates a heavy API surface
- Best static typing
I think the idea for overloads is no worse than 2, and you can still write 1 if you want.
EDIT: As @srawlins pointed out to be, another huge advantage of overloads over the "dynamic"-ish method with is checks is the ability to have conditional type arguments - that is, type arguments that only exist in a particular context:
class Foo {
void bar();
T bar<T>(T i) => ...
List<T> bar<T>(List<T> i) => ...
Map<K, V> bar<K, V>(Map<K, V> m) => ...
}
It's not possible to express this pattern in dynamic dispatch (or with a single bar at all).
By the way, this would have solved the Future.catchError issue:
class Future<T> {
Future<T> catchError(Object error) {}
Future<T> catchError(Object error, StackTrace trace) {}
}
... as a bonus :)
@matanlurey,
Is it being dynamically invokable a requirement? I don't think it is.
That was actually the point I was making: It is important that there is a well-defined semantics of method invocation, and if just one static overload is allowed to exist then every dynamic invocation will need to potentially handle static overloads, and that presumably amounts to multiple dispatch (like CLOS, Dylan, MultiJava, Cecil, Diesel, etc.etc.), and I'm not convinced that it is a good trade-off (in terms of the complexity of the language and its implementations) to add that to Dart.
In particular, the very notion of making the choice among several method implementations of a method based on the statically known type is a completely different mechanism than standard OO method dispatch, and there is no end to the number of students that I've seen over time who just couldn't keep those two apart. (And even for very smart people who would never have a problem with that, it's likely to take up some brain cells during ordinary daily work on Dart projects, and I'm again not convinced that it's impossible to find better things for those brain cells to work on ;-).
@eernstg:
and if just one static overload is allowed to exist then every dynamic invocation will need to potentially handle static overloads
Why? If we just don't allow dynamic invocation to invoke static overloads, nothing is needed.
In particular, the very notion of making the choice among several method implementations of a method based on the statically known type is a completely different mechanism than standard OO method dispatch, and there is no end to the number of students that I've seen over time who just couldn't keep those two apart
I just want what is already implemented in Java/Kotlin, C#, or other modern languages. Do they do something we aren't able to do, or is this just about preserving dynamic invocation? As I mentioned, the alternative is users write something like this:
class Foo {
void bar() {}
void barString(String s) {}
void barNumber(num n) {}
}
Not only do we punish users (they have to name and remember 3 names), dynamic invocation cannot help you here (mirrors could of, but that is no longer relevant).
It's worth mentioning that if we decide to support overloads without dynamic invocations, this means that adding an overload to an existing method will be a breaking change--one that probably won't be obvious to API designers.
Depending how we do it, we theoretically could support a dynamic fallback overload:
class Future<T> {
// This one is used for any dynamic invocations only.
Future<T> catchError(dynamic callback);
Future<T> catchError(void Function(Object));
Future<T> catchError(void Function(Object, StackTrace));
}
It's not clear to me this is particularly worth it, though. Other hotly requested features like extension methods would also suffer from being static only, and changing a method from invocation to extension would be a breaking change.
I expect it won't be too surprising to users that changing an existing method is a breaking change. Adding a new method being a breaking change, on the other hand, is likely to be very surprising, especially since it's safe in other languages that support overloading.
Right, because they never supported dynamic invocation (or only do, like TypeScript).
One project @srawlins was working on back in the day was a tool that could tell you if you accidentally (or on purpose) introduced breaking changes in a commit. I imagine a tool could help, or we could even add a lint "avoid_overloads" for packages that want to be dynamically-invoke-able.
Users aren't going to know to run a tool to tell them that overloads are breaking changes any more than they're going to know that overloads are breaking changes. And even if they did, the fact that adding an overload requires incrementing a package's major version would make the feature much less useful for anyone with downstream users.
I don't think a lint would do anything, because upstream API authors don't control whether their downstream users dynamically invoke their APIs. In fact, since we don't have robust and universal support for --no-implicit-dynamic, the downstream users probably also don't know when they're dynamically inovking APIs.
OK, I think we can note that this feature would be breaking for dynamic invocation and leave it at that.
The language team hasn't given any indication this particular feature is on the short-list for any upcoming release, and I'm assuming when and if they start on it we can revisit the world of Dart (and what support we have for preventing dynamic invocation entirely).
I would like to hope this issue continues to be about implementing the feature, not whether or not it will be a breaking change (for all we know this will happen in Dart 38, and dynamic invocation has been disabled since Dart 9).
EDIT: For anyone reading this, I am not saying that will happen.
I also think overloading would be a fantastically useful feature, but it's complexity is not to be under-estimated. If the language folks seem to cower in fear every time it comes up, that's not without reason. This tweet sums it up pretty well:
C# language design is
10% exploring cool ideas
75% overload resolution
15% being sad at past decisions we made
This tweet sums it up pretty well
It's nice of the author to leave a hint though: "I would do it more like F#. It is there in a very basic, simple form" :smile:
@munificent Definitely understand it shouldn't be underestimated. Do we have a requirement that all new features support dynamic invocation? If so, didn't we already break that with type inference?
I'm with @munificent on the need to recognize the complexity ('C#: 75% is overload resolution' ;-), but I'm not worried about the complexity of specifying or even implementing such a feature, I'm worried about the complexity that every Dart developer is involuntarily subjected to when reading and writing code. In particular, I'm worried about the inhomogeneous semantics where some decisions are based on the properties of entities at run time, and other decisions are based on properties of entities produced during static analysis—one is fine, the other is fine, but both at the same time is costly in terms of lost opportunities for developers to think about more useful things.
One way we could make the two meet would be based on a dynamic mechanism that compilers are allowed to compile down to a static choice whenever that's guaranteed to be correct. For instance, using the example from @matanlurey as a starting point:
abstract class Future<T> {
...
Future<T> catchError(Function)
case (void Function(Object) onError)
case (void Function(Object, StackTrace) onError)
default (Function onError);
...
// Could be a bit nicer with union types.
Future<T> catchError2(void Function(Object) | void Function(Object, StackTrace))
case (void Function(Object) onError)
case (void Function(Object, StackTrace) onError);
}
class FutureImpl<T> implements Future<T> {
Future<T> catchError
case (void Function(Object) onError) {
// Implementation for function accepting just one argument.
}
case (void Function(Object, StackTrace) onError) {
// Implementation for function accepting two arguments.
}
default (Function onError) => throw "Something";
...
// The variant with union types would just omit the default case.
}
There would be a single method catchError (such that the tear-off operation is well-defined and preserves the full semantics), and the semantics of the declared cases is simply like a chain of if-statements:
Future<T> catchError(Function onError) {
if (onError is void Function(Object)) {
// Implementation for function accepting just one argument.
} else if (onError void Function(Object, StackTrace) onError) {
// Implementation for function accepting two arguments.
} else {
default (Function onError) => throw "Something";
}
}
However, the declared cases are also part of the interface in the sense that implementations of catchError must handle at least the exact same cases, such that it is possible to generate code at call sites where it is statically known that the argument list satisfies a specific case. In that situation we would have code that directly calls one of those cases (such that there is no run-time penalty corresponding to the execution of a chain of if-statements, we have already chosen the correct branch at compile-time).
For instance, we always know everything about the type of a function literal at the call site. Special types like int and String are constrained by the language such that we can't have one instance which is both at the same time, and with sealed classes we can have more cases with that property.
This means that we will have multiple dispatch in a way where the priority is explicitly chosen by the ordering of the cases (so we avoid the infinite source of complexity which is "ambiguous message send"), and the mechanism will double as a static overloading mechanism in the cases where we have enough information statically to make the choice.
I'm not saying that this would be ridiculously simple, but I am saying that I'd prefer working hard on an approach where we avoid the static/dynamic split. And by that split I don't mean code which is full of expressions of type dynamic, I mean code which may be strictly typed (--no-implicit-cast and whatnot), because that would still allow the same operations applied to the very same objects to behave differently, just because the type checker doesn't know the same amount of things at two different call sites.
... Java/Kotlin, C#, or other modern languages. Do they do something we aren't able to do, or is this just about preserving dynamic invocation?
Neither (we can surely make a big mess of things as well ;-), but, to me, it is very much about avoiding a massive amount of subtleties for the developers, also for code which is statically typed to any level of strictness that we can express.
Most Dart developers don't want dynamic invocation (in Dart2, it is actively bad in many places with reified types), so it seems to me trying to preserve that feature for new language features isn't worth the time or effort.
@matanlurey, if that's concerned with this comment, it sounds like maybe you did not notice that I'm not talking about dynamic invocations, or certainly not only about them:
I don't mean code which is full of expressions of type
dynamic, I mean .. strictly typed .. [code that still causes a] massive amount of subtleties for the developers
Do we have a requirement that all new features support dynamic invocation?
I think dynamic invocation is red herring. C#'s dynamic supports full C# overload resolution. The complexity isn't around dynamic invocation. It is more inherent to how overload resolution interacts with type inference, overriding, generics, generic methods, optional parameters, and implicit conversions.
(We don't have implicit conversions in Dart yet, but we will once you can pass 0 to a method that expects a double.)
I just slapped this together, but here's a sketch that might give you a flavor of how it can get weird:
class Base {
bar(int i) {
print("Base.bar");
}
}
class Foo<T extends num> extends Base {
bar(T arg) {
print("Foo<$T>.bar");
}
}
test<T extends num>() {
Foo<T>(null);
}
main() {
test<int>();
test<double>();
}
I might be ignorant, but isn't there a similar set of complexity for extension methods? Meaning that if we need to eventually figure out how to dispatch extension methods, at least some of the same logic holds for dispatching overload methods?
(It looks like, casually, most OO languages that support extensions support overloading)
Potentially, yes, but I think they tend to be simpler. With extension methods, you still only have a single "parameter" you need to dispatch on. You don't have to worry about challenges around tear-offs. Things might get strange if we support generic extension classes. I don't know. But I would be surprised if extension methods weren't easier than overloading.
Thanks for this! A few more questions, but don't feel like they are important to answer immediately :)
Potentially, yes, but I think they tend to be simpler.
Are there some limitations we could add to overloads to make them easier to implement and grok? I might be incredibly naive ( /cc @srawlins ) but I imagine 95%+ of the benefit could be gained with a few simplifications:
- Either no tear-off support, or force users to write
void Function(String) bar = foo.bar - Don't support overloading on bottom or top types
- Don't support overloading on generic types
For example, today I was writing a sample program for a r/dailyprogramming question.
I wanted to be able to write:
abstract class DiceRoll {
int get amount;
int get sides;
}
abstract class DiceRoller {
/// Roll a dice defined by the expression "NdN".
List<int> roll(String expression);
/// Roll [amount] of dice with [sides].
List<int> roll(int amount, int sides);
/// ...
List<int> roll(DiceRoll roll);
}
But I'd either have to write:
abstract class DiceRoller {
/// Roll a dice defined by the expression "NdN".
List<int> rollParse(String expression);
/// Roll [amount] of dice with [sides].
List<int> roll(int amount, int sides);
/// ...
List<int> rollFor(DiceRoll roll);
}
Or do something extra silly like:
abstract class DiceRoller {
List<int> roll(dynamic expressionOrAmountOrRoll, [int sides]) {
if (expressionOrAmountOrRoll is int) {
if (sides == null) {
throw 'Expected "sides"';
}
return _rollActual(expressionOrAmountOrRoll, sides);
}
if (sides != null) {
throw 'Invalid combination';l
}
if (expressionOrAmountOrRoll is String) {
return _rollAndParse(expressionOrAmountOrRoll);
}
if (expressionOrAmountOrRoll is DiceRoll) {
return _rollActual(expressionOrAmountOrRoll.amount, expressionOrAmountOrRoll.sides);
}
throw 'Invalid type: $expressionOrAmountOrRoll';
}
}
The former is hard for the users to use (and find APIs for) and the latter sucks to write, test, and basically forgoes any sort of static type checking.
You don't have to worry about challenges around tear-offs.
Does that mean tear-offs wouldn't be supported for extension methods, or that it's easier?
I imagine folks would find it weird if you could do:
void main() {
// This will, or will not work, depending on if `map` is an extension method or not?
wantsAClosure(['hello', 'world'].map);
}
void wantsAClosure(Iterable<String> Function(String) callback) {}
Things might get strange if we support generic extension classes
Do you mean (psuedo-syntax):
/// ['hello, 'world'].joinCustom()
extension String joinCustom(this Iterable<String> parts) {
// ...
}
Or:
extension Map<K, V> groupBy<K, V>(this Iterable<V>, K Function(V) groupBy) {
// ...
}
Either no tear-off support, or force users to write
void Function(String) bar = foo.bar
I believe the latter is what C# does. It helps, though it causes some weird confusing behavior. It's always strange for users when you can't take a subexpression and hoist it out to a local variable.
Don't support overloading on bottom or top types
I don't think that's the cause of much of the pain.
Don't support overloading on generic types
That might help, but it's probably too painful of a limitation in practice. One of the key uses of overloading is being able to extend the core libraries without breaking them, and many of the classes where that would be most helpful, like Iterable and Future, are generic.
abstract class DiceRoller { /// Roll a dice defined by the expression "NdN". List<int> roll(String expression); /// Roll [amount] of dice with [sides]. List<int> roll(int amount, int sides); /// ... List<int> roll(DiceRoll roll); }
Yeah, I've run into this exact scenario.
Just allowing overloading by arity (number of parameters) would help many of these simple cases and doesn't require a lot of static typing shenanigans. Dynamically-typed Erlang supports it. Though it would interact in complex ways with optional parameters in Dart.
You don't have to worry about challenges around tear-offs.
Does that mean tear-offs wouldn't be supported for extension methods, or that it's easier?
That it's easier. Once you've don't the extension method lookup statically, you know exactly what method is being torn off, so you can just do it.
With overloading, there are interesting questions around whether the lookup should be done statically, dynamically, or some combination of both.
Things might get strange if we support generic extension classes
Do you mean (psuedo-syntax):
/// ['hello, 'world'].joinCustom() extension String joinCustom(this Iterable<String> parts) { // ... }
I mean:
extension class Iterable<int> {
int sum() => fold(0, (sum, element) => sum + element);
}
test<T>(Iterable<T> elements) {
elements.sum(); // <--???
}
I'm sure @leafpetersen and @lrhn can figure out how to handle all of this, but I'm not sure if I could. :)
Thanks! I am sure I will understand this issue eventually :)
One of the key uses of overloading is being able to extend the core libraries without breaking them, and many of the classes where that would be most helpful, like Iterable and Future, are generic.
Did you mean one of the key uses of extension methods, or overloading?
Overloading and extension methods are orthogonal. Both allow "adding a method" to a class without breaking an existing method with the same name. If you have both, there is a good chance that the extension method won't completely shadow the original method. Extension methods are not virtual, which is annoying. You can add them from the side, which is useful. We don't have a way to add a virtual method from the side, and I'm not sure it's possible.
The languages with overloading mentioned so far do not have optional parameters the same way Dart does. They do have optional positional parameters, so that might not be an issue. We still have to handle cases like:
int foo(int x, [int y, int z]) => ...
int foo(int x, {int y, int z}) => ...
...
theFoo.foo(42);
Likely it's just an unresolved overload error at compile-time. Again, a dynamic invocation might not apply here, but if it does, then it's not clear that there is a solution.
Maybe we can solve it by (theFoo.foo as int Function(int, int, int))(42). I'd like as to actually induce a preference on the expression.
As for
extension class Iterable<int> {
int sum() => fold(0, (sum, element) => sum + element);
}
test<T>(Iterable<T> elements) {
elements.sum(); // <--???
}
my way of figuring that one out would just be to say "extension method does not apply". The static type of elements is Iterable<T>, which is not the same as, or a subtype of, Iterable<int>, so the static extension method cannot be used. Since elements does not have a sum method, your program won't compile.
Now, if it had been:
test<T extends int>(Iterable<T> elements) {
elements.sum(); // <--???
}
then the extension method would likely have applied.
More controversial is:
extension List<T> {
R join<R>(R base, R Function(R, T) combine) => ...;
}
Should that function "override" the join function on List? Shadow join completely, or only act as an alternative overload? What if I named it fold instead? That kind of conflict is troublesome, but probably not a real issue (it'll just allow people to shoot themselves in the foot, and linting can tell you to stop being silly).
Anyway, this is about overloading, not extension methods.
Would it make it easier if it is available only for positioned parameters? From my experience, overloadable methods are normally very well defined and with few parameters.
Atenciosamente,
Jonathan Rezende
Em 9 de jul de 2018, à(s) 06:53, Lasse R.H. Nielsen [email protected] escreveu:
Overloading and extension methods are orthogonal. Both allow "adding a method" to a class without breaking an existing method with the same name. If you have both, there is a good chance that the extension method won't completely shadow the original method. Extension methods are not virtual, which is annoying. You can add them from the side, which is useful. We don't have a way to add a virtual method from the side, and I'm not sure it's possible.
The languages with overloading mentioned so far do not have optional parameters the same way Dart does. They do have optional positional parameters, so that might not be an issue. We still have to handle cases like:
int foo(int x, [int y, int z]) => ... int foo(int x, {int y, int z}) => ... ... theFoo.foo(42); Likely it's just an unresolved overload error at compile-time. Again, a dynamic invocation might not apply here, but if it does, then it's not clear that there is a solution. Maybe we can solve it by (theFoo.foo as int Function(int, int, int))(42). I'd like as to actually induce a preference on the expression.
As for
extension class Iterable
{ int sum() => fold(0, (sum, element) => sum + element); } test<T>(Iterable<T> elements) { elements.sum(); // <--??? } my way of figuring that one out would just be to say "extension method does not apply". The static type of elements is Iterable<T>, which is not the same as, or a subtype of, Iterable , so the static extension method cannot be used. Since elements does not have a sum method, your program won't compile. Now, if it had been: test<T extends int>(Iterable<T> elements) { elements.sum(); // <--??? } then the extension method would likely have applied.
More controversial is:
extension List<T> { R join<R>(R base, R Function(R, T) combine) => ...; } Should that function "override" the join function on List? Shadow join completely, or only act as an alternative overload? What if I named it fold instead? That kind of conflict is troublesome, but probably not a real issue (it'll just allow people to shoot themselves in the foot, and linting can tell you to stop being silly).
Anyway, this is about overloading, not extension methods.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dart-lang/sdk/issues/26488#issuecomment-403424824, or mute the thread https://github.com/notifications/unsubscribe-auth/AF6Y3jQTXj788r5T9g9c8q-24N1wE3Qaks5uEygbgaJpZM4IhzBM.
I also tend to agree that trying to combine overloads and optional parameters (either named or positional) is probably not worth its weight. A lot of the places that optional parameters are used today are to emulate overloads, and users would likely use overloads instead if available.
Not allowing optional named parameters with overloads will make it difficult to backwards-compatibly extend APIs that were originally defined as overloads. This would incentivize API designers to add lots of overloads with additional positional parameters, which is generally less readable and more fragile than named parameters.
My preference for APIs like
int foo(int x, [int y, int z]) => ...
int foo(int x, {int y, int z}) => ...
would be to disallow the definition at compile time. This ensures that overload calls are never ambiguous, and that API designers are aware when they try to design an API that would be ambiguous and can avoid it (e.g. by making int y mandatory in the first definition above).