improvement-proposals
improvement-proposals copied to clipboard
SIP-64: Improve the Syntax of Context Bounds and Givens
I think something is off in this branch. The Named Tuples commits (from fd964a9 to d649f6e) are included here as well.
I imagine this was intended to only start at 267b8ff?
@JD557 yes, indeed. I force pushed as an independent separate branch, incorporating fixes to the two typos pointed out by @johnynek.
The proposal could be split further into two or three independent areas, but there are also connections between the parts:
- Both context bounds and new givens use
asto introduce an optional name. - Implementing context bounds for type members relies on deferred givens.
So one logical progression could be (1) deferred givens replacing abstract givens (2) new given syntax (3) context bound changes. But the motivation why the new given syntax is harmonious comes in part from the fact that it is in agreement with names for context bounds.
think we should add a discussion (apologies if I missed it) of the fact that some using/givens don't have a single parameter to dispatch on. For such cases we would still employ the using variant. To me I'd like to see a case that we could completely kill the using variant.
I don't think that's possible or desirable. In my world view type classes are a kind of types for types. That means they can only refer to a single type. Multi-parameter type classes are really constraints passed as context. So "multi-parameter type class" is already a misnomer. The name was invented in Haskell because Haskell does not have a general context passing mechanism, all has to be force-fitted into the typeclass paradigm. And of course there are also bits of context that don't constrain any type parameters. So it seems natural to keep using clauses for these cases, and reserve context bounds for true (that is, single-parameter) type classes.
Some random thoughts about the parts of the proposal:
-
Named Context Bounds: The necessity to type
summon[...]never bothered me, so I don't see a clear motivation for this change apart from the possibility of achieving the same effect with fewer keystrokes. Also, I really dislike the usage of theas(soft) keyword. I know it is entirely new for Scala, but we cannot ignore that it is used in multiple languages (Rust, Kotlin, C#, and TypeScript come to my mind just now), and in all of them, it means some kind of type-coercion or unchecked cast. What's more, in all of those languages, it has the same general form:<term> as <type>. We want to use it backward as<type> as <term>. This will be highly confusing to any newcomers just discovering Scala. Last, but not least, it breaks the convention that the names of terms always come before their types. I don't like this inconsistency, but I realize it would require advanced syntactic acrobatics to avoid it. -
Aggregate Context Bounds: That is a very good change; let's approve it.
-
Expansion of Context Bounds: We must check if the change doesn't introduce any weird and non-intuitive type inference. Apart from that, it is an excellent idea.
-
Context Bounds for Type Members, Deferred GIvens Another very good proposal. I was going to side with the opinion that
deferredshould be a modifier, but the interpretation of it as a compiler-provided macro convinced me. -
New Given Syntax While I agree that the current
given ... : ... with ...syntax is far from perfect, alternatives (except for the trivial cases) are extremely confusing. If I stumbled ongiven [A : Ord] => Ord[List[A]]:before reading the proposal, I wouldn't even have an idea how to parse it. Named alternatives also suffer from problems with users' expectations about the meaning of theaskeyword, as mentioned in point 1. I wouldn't change anything as I think there is a smooth transition from:
def lexicographicOrd[T](using Ord[T]): Ord[List[T]] = ???
through
given lexicographicOrd[T](using Ord[T]): Ord[List[T]] = ???
to
given lexicographicOrd[T](using Ord[T]): Ord[List[T]] with
That makes the last syntax intuitive, even if it is awkward. The proposal doesn't mention alias givens at all, leaving a question about whether it intends to introduce a huge inconsistency between ordinary given instances definitions and alias given definitions.
-
There is no point 6
-
Abolish abstract givens Very good change, let's approve it.
@Kordyjan Given clauses are usually written without a name, and that's mostly where the old syntax is weird.
I know it is entirely new for Scala
as is not entirely new, it is already used for import renamings.
This proposal is up for the next SIP meeting on May 24th which is soon and I think we should prepare the meeting discussions here.
In general I think the ambition of this SIP is great. I find the first parts of the new syntax and expansion schemes for naming context bounds, multiple context bounds, context bounds for type members reasonable and with support from the compiler to rewrite old syntax and a long grace period before removal I think it is doable.
However, the controversial/"risky" part of this proposal is the change of syntax for givens, and the main concern is language stability versus language improvement. Or in other words: is the improvement to givens worth the costs of migration and potential risk of decreased willingness to go for Scala Next?
@odersky It would be great if the below issues can be clarified in the proposal or in this comment thread, as input to the meeting:
- If we introduce a new syntax for givens, how would the deprecation scheme relate to the LTS roadmap? Would the current 3.3-LTS given syntax still available in the next 3.x-LTS ? (Will 3.7 be then next LTS?)
- What parts of the new syntax for given can be back-ported to 3.3-LTS so that the ones stuck on that LTS still can use the new syntax in parallell with the old syntax? (Is it right to assume from the current text that all changes can be back-ported?)
- I'd like an overview of the different kinds of givens in a before-after-table or similar, as I think it is still difficult to get the big picture of all the kinds of given and the different syntax variants over time.
Assuming something like (or similar):
case class MyType(i: Int)
case class MyGenType[A](a: A)
trait MyOldTypeClass[T]:
...
trait MyNewTypeClass:
type Self
...
I think it would be great if you include a summary overview for the proposed new given syntax, perhaps like this (or similar) in the proposal:
| kind of given | current | proposal |
|---|---|---|
| named given value | given x: MyType = MyType(42) | given MyType as x = MyType(42) |
| anonymous given value | given MyType = MyType(42) | no change |
| ...etc. |
As there are many types of givens, some available here, I think it is valuable if there is a such a systematic walk-through of all combinations so we don't miss any feature interaction and also can get an overview of how big the syntax changes are. In the current proposal only a few of all possible kinds of givens are shown in the new syntax and it is not clear to me if more variants are impacted.
I think the new arrow syntax using => is the most controversial/radical change so its important to include explanations of interpretations/expansions for that in the actual proposal (some rationale is currently available in the review comment only). I find this rather difficult to read: given [A : Ord] => Ord[List[A]]: ... and I wonder if a keyword instead of => such as for or similar could make it easier to grok? Have you considered alpha keyword alternatives instead of a symbol?
Another question I have:
The only downside is that deferred givens are restricted to be used in traits, whereas abstract givens are also allowed in abstract classes.
Can deferred givens be allowed also in abstract classes? If no, why not?
Can deferred givens be allowed also in abstract classes? If no, why not?
It would complicate things a bit and we would lose flexibility. We want to have a strict separation of classes/traits where deferred givens are defined and where they are implemented. If deferred givens can be defined in abstract classes, then they would in turn not be implemented in abstract classes. But we sometimes might want to implement deferred givens or context bounds in classes that are still missing the implementations of some members.
We also have precedent for this split: trait parameters. These are defined in traits and the corresponding arguments must appear in the first implementing class, where it does not matter whether that class is abstract or not.
It would complicate things a bit
Well, I'm inclined to favor avoidance of unnecessary exceptions so that rules are simpler. But if the implementation is tricky, the relaxation of the restriction on deferred givens can be added later.
I added a new section that does a systematic comparison between current and proposed syntax.
I added a new section that does a systematic comparison between current and proposed syntax.
Many thanks! I think the added systematic comparison is very nice to have.
I found typos as reported below (or I'm confused...):
- Under "Current, Anonymous"
// Alias with using clausethere should be a constructor application, i.e. missing()should be added as in:
// Alias with using clause
given [A](using Ord[A]): Ord[List[A]] =
ListOrd[A]()
- Same missing () under "Proposal, Anonymous", should be:
// Alias with using clause
given [A](using Ord[A]) => Ord[List[A]] =
ListOrd[A]()
- Same missing () under "Current, Named", should be:
// Alias with using clause
given listOrd[A](using Ord[A]): Ord[List[A]] =
ListOrd[A]()
- Same missing () under "Proposal, Named", should be:
// Alias with using clause
given [A](using Ord[A]) => Ord[List[A]] as listOrd =
ListOrd[A]()
- There is a name missing (listOrd) in the "Current, Named", under
// Typeclass with using clauseshould be:
// Typeclass with using clause
given listOrd[A](using Ord[A]): Ord[List[A]] with
def compare(x: List[A], y: List[A]) = ...
- There is one colon too much here under "Current, Named":
// Parameterized typeclass
given listOrd: [A: Ord]: Ord[List[A]] with
def compare(x: List[A], y: List[A]) = ...
should be
// Parameterized typeclass
given listOrd[A: Ord]: Ord[List[A]] with
def compare(x: List[A], y: List[A]) = ...
@odersky I got a new idea, after your reasoning about that the important thing should come first: In the propsed arrow syntax case the important thing comes last; the most important thing is the actual given thing, not the type param precondition.
If we allow the context bound inside the type param, the actual given will be the important thing and we can do away with the => symbol sallad, like so:
// Parameterized typeclass
given Ord[List[A : Ord]]:
def compare(x: List[A], y: List[A]) = ...
What do you think about that?
@odersky as mentioned during yesterday's lab meeting, here is an extended example comparing the old and proposed syntax which could be added to the proposal.
| New syntax | Old syntax |
|---|---|
|
|
@bjornregnell Unfortunately that would be ambiguous. If there are no context bounds then
given List[Ord[A]]
could mean: for all A, List[Ord[A]] or it could mean: List[Ord[A]] for a specific class named A.
@EugeneFlesselle This is a nice example, that mostly shows savings from having context bounds as members, which translate into deferred givens, instead of having to code up everything yourself using abstract givens.
It does use more syntax than in this SIP. In particular, infix is and Self` types are not proposed in this SIP. But I think it should be easy to adapt the example to use parameterized type classes.
Unfortunately that would be ambiguous.
But if there is a context bound its not ambiguous? And it's the same with normal generic params. like in def f[A](x: A) = ??? where A could be a class name that gets shadowed. We could have semantics that makes it unambiguous in relevant cases. I just thinks this is on the perfect spot to read with clear interpretation:
given Ord[List[A : Ord]]:
def compare(x: List[A], y: List[A]) = ...
And it seems as if the proposed => is the most questioned part of the proposal, judging from this tread and last meeting SIP discussions.
But I get it, that it would mean analogously as if def f(x: A) = ??? would be interpreted as having A as a type param even without the type param list [A]. In a nested paramerized given there needs to be a way to determine that A is an unbounded type param. Hmmm.
I can write:
scala> def g[A: Ordering](x: A) = x.toString
def g[A](x: A)(implicit evidence$1: Ordering[A]): String
but not:
scala> val f: [A: Ordering] => A => String = [A: Ordering] => (x: A) => toString
-- [E040] Syntax Error: -------------------------------------------------------------------------------------------------------
1 |val f: [A: Ordering] => A => String = [A: Ordering] => (x: A) => toString
| ^
| ']' expected, but ':' found
I have to:
scala> val f: [A] => Ordering[A] ?=> A => String = [A] => (o: Ordering[A]) ?=> (x: A) => x.toString
val f: [A] => (x$1: Ordering[A]) ?=> A => String = Lambda$1764/0x00007f043456f9a8@17ee816c
Shouldn't context bounds work also with function lambda syntax?
Shouldn't context bounds work also with function lambda syntax?
Yes, this would be another useful generalization.
Here is one concrete example of my concerns with the type bounds on abstract type members:
package strawman {
package coll {
trait Sequence[A]
package immut {
trait Sequence[A]
}
}
}
package package1 {
trait Foo:
type MySeq : strawman.coll.Sequence
}
package package2 {
trait Bar:
type MySeq : strawman.coll.immut.Sequence
}
package app {
class ConcreteSeq
given strawman.coll.Sequence[ConcreteSeq] as IsColl with {}
given strawman.coll.immut.Sequence[ConcreteSeq] as IsImmutColl with {}
class Foobar extends package1.Foo with package2.Bar:
type MySeq = ConcreteSeq
}
gives
-- [E164] Declaration Error: tests/run/hello.scala:44:2 ------------------------
44 | class Foobar extends package1.Foo with package2.Bar:
| ^
|error overriding given instance given_Sequence_MySeq in trait Foo of type strawman.coll.Sequence[Foobar.this.MySeq];
| given instance given_Sequence_MySeq of type strawman.coll.immut.Sequence[Foobar.this.MySeq] has incompatible type
|
| longer explanation available when compiling with `-explain`
1 error found
The double indirection to give a name that is actually semantically meaningful from anonymous things truly makes compatibility scenarios impossible to predict for anyone but compiler writers.
To summarize my thoughts in one line per proposed item ahead of the meeting:
- Naming context bounds: good ✅
- New Syntax for Aggregate Context Bounds: good ✅
- Expansion of Context Bounds: good ✅
- Context Bounds for Type Members, Deferred Givens a. Context Bounds for type Members: strong concerns for compatibility scenarios ❌ b. Deferred Givens: syntax concerns, should be a modifier, but otherwise OK ⚠️
- Cleanup of Given Syntax: I don't want to fight it ✅
- Abolish Abstract Givens: can never remove under any circumstance ❌ (deprecation that can be silenced is fine)
Here is one concrete example of my concerns with the type bounds on abstract type members:
I notice that if you change it to
package strawman {
package coll {
trait Sequence[A]
package immut {
trait Sequence[A] extends coll.Sequence[A]
}
}
}
it compiles. So, to summarize the conditions that make it fail:
- We need two unrelated typeclass traits with the same name but in different packages (that's already a big code smell).
- We need two traits that each have a context bound with one of these typeclass traits on the same abstract type.
- We need a class that mixes in these two traits.
In that case we are indeed stuck. We simply can't mix the two traits in one class. But there are much simpler and more common scenarios where the same happens:
trait Foo:
def foo(): Int
trait Bar:
def foo(): String
class C extends Foo, Bar // unrecoverable error
So, yes, since Scala does not have a general member renaming mechanism (since the JVM does not support that), there are traits that can't be mixed in together. But I believe the case with deferred givens is so artificial that it does not materially change that situation.
For removing abstract givens: We can do it slowly and only after a survey of existing libraries. Let's say no project in open CB uses abstract givens. Can't we remove the feature then? (it would stay around under a source flag for sure).
If we are worried about Tasty compatibility: The Tasty format would still support them in the whole 3.x series. But we should be able to drop them from the source language at some point.
Shouldn't context bounds work also with function lambda syntax?
Yes, this would be another useful generalization.
I think context bounds in polymorfic function types like val f: [A: Ordinal] => A => ... should be included in this SIP as it seems like an uncontroversial and intuitive generalisation. The actual implementation can come in due course; by including it in this SIP, we have a more coherent, regular syntax update.
Here is my summary comment as input to the meeting:
-
There are typos in the nice comparison summary as reported here that I think should be fixed: https://github.com/scala/improvement-proposals/pull/81#issuecomment-2124769512
-
I'm in favor of the proposed new naming context bound using the
askeyword. It's in line with import renaming. -
I'm in favor of proposed new syntax for aggregate context bounds using braces as it's easier to read.
-
I'm in favor of generalizing abstract type members to allow context bounds as it improves regularity and it is a surprise if it's not possible to do that.
-
I'm not sure I understand all the intricacies of abstract givens and deferred givens, but it seems more intuitive to me that it is a modifier, and I don't really get how "a story of a magic method" is better than a modifier; a newcomer just needs to learn the syntax anyway and if it is not for some reason valid with the existing keyword
abstractas in aabstract given ...modifier then I think adeferredmodifier makes more sense than a magic method, with a good doc explanation of whydeferred given ...is needed and useful. In the proposal example there is even an `override given ...``for the deferred one, so I'd argue that a modifier is more regular as it is the same syntax position. But all this seems like very intricate details and I can accept any of a modifier or a magic method if others think there are prevailing arguments for any of them. For me its ok to keep experimenting with this if we accept the SIP. -
I'm mostly OK with cleaning up the given syntax and I like
asbetter than double-colon, esp. for anonymous givens. The price for the change in badwill is probably worth it, but there needs to be a really good communication effort with the community to explain in clear terms why it is needed and better on a balance. I am hesitant towards the new=>syntax but after contemplating alternatives I could not find anything better, and there is the precedent in the analogy with=>in pattern matching. But I have an itch (or hunch) that the interaction with function types and polymorfic function types and context function types need more investigation, see also comment by @johnynek above but that can come during the experimental phase if we accept this SIP. -
I think allowing context bounds in polymorfic function types and its de-sugaring into context function types should be included in the proposal to make it more complete.
For removing abstract givens: We can do it slowly and only after a survey of existing libraries. Let's say no project in open CB uses abstract givens. Can't we remove the feature then? (it would stay around under a source flag for sure).
If we are worried about Tasty compatibility: The Tasty format would still support them in the whole 3.x series. But we should be able to drop them from the source language at some point.
That is first-order TASTy compatibility. The second order I'm talking about is that we have to allow, somehow, libraries to upgrade to newer versions of the compiler without breaking their own compatibility (other than requiring users to also upgrade their compiler). If it requires source changes, fine; if it requires changing compiler flags, fine; but there needs to be a way.
(it would stay around under a source flag for sure).
This would qualify as acceptable, indeed.
IIRC, there were some arguments on the pre-sip discussion about limits with the new T: C as c and explicit using paramters still being required to express certain cases? Or maybe it was about the improved syntax for given definitions? Maybe someone remembers or has a pointer.
@lrytz I am not sure. We need using parameters for things that are not typeclasses. Or if one wants to insist on a using clause at a different place than where it would be inserted automatically. Was there something else?
@sjrd As a manager of this SIP, perhaps you can summarize here its current status and result from our last SIP-meeting. I'm not sure the status label here on github is correct in relation to previous votes. I also think it would be good if you ping the Contributors-thread and explain the current status to the community in order to invite experimentation and feedback.
I think the latest edits by @odersky is a great improvement in explaining the rationale for the new syntax, making it easier to understand how to apply the prameterization [A : B] => ... variant if you know the rational.
@odersky Perhaps the proposal should be clear about the corner case of givens for function types, even if they are rare or even unrecommended, also the variant of a given for a polymorfic function type. I guess it boils down to having parenthesis in the right places, and I think it would be good to state those rules in the proposal.
@sjrd As a manager of this SIP, perhaps you can summarize here its current status and result from our last SIP-meeting. I'm not sure the status label here on github is correct in relation to previous votes.
It's not easy to write a summary that encompasses the various concerns that were raised. Here is a summary of the summary, though:
- There seems to be a majority of members who think the changes are good in principle.
- There seems to be agreement on the first few changes.
- The syntax of the last few things is highly controversial: we have differing opinions on how to interpret the meaning of things, and that has an impact on whether we perceive the syntax to reflect that meaning well or not.