Kotlin
Collection literals
This is an issue for discussion of collection literals proposal. The full text of the proposal is here.
Please, use this issue for the discussion on the substance of the proposal. For minor corrections to the text, please open comment directly in the PR https://github.com/Kotlin/KEEP/pull/417.
Finally!! 🎊🎊🎊
Really wish that tuples and maps weren't excluded though. And also addition of a slicing syntax would have been nice, although technically it is only somewhat related here.
Is the plan to release this as preview in 2.2.0?
First, as someone who occasionally teaches Kotlin to developers who are coming from other languages (most notably Java, JS and Swift), no one ever mentioned any difficulties learning the language because of the lack of collection literals. The main syntax difficulties I've seen was with the mandatory parenthesis after keywords (optional in Swift and Rust).
Clear intent. A special syntax for collection literals makes it clear that a new instance consisting of the supplied elements is created. For example, val x = listOf(10) is potentially confusing, because some readers might think that a new collection with the capacity of 10 is created. Compare it to val x = [10].
I don't think this is so black-and-white. Java uses int[] x = new int[10] to declare an array of size 10, and Kotlin doesn't have new and avoids writing the types, so [10] could very well be read by a Java developer as creating an array of size 10.
That's not to say I'm against collection literals (I'm not), but they are definitely not a major issue at the moment, and I think that overestimating their need will lead to rushing the implementation. Compared to how little the lack of collection literals impacts everyday code, adding them now will introduce a major change in how Kotlin code looks, and will create a big before/after rift. Newcomers will always have to learn about emptyList and listOf: even 10 years into the future, there will still be resources from before literals, and there will still be developers who still have the habit.
// before 1 if (readlnOrNull() in listOf("y", "Y", "yes", "Yes", null)) { // ... } // after 1 if (readlnOrNull() in ["y", "Y", "yes", "Yes", null]) { // ... }
An important part of the Kotlin design is how the shortest code is (almost) always the best code. Is in [] the best code? What's [] here?
By making this code simpler we are emphasizing it to be correct to newcomers. Is this the kind of code we want Kotlin developers to write? AFAIK, this example is currently discouraged in code review, instead recommending to create a HashSet beforehand, or using a when statement, which will generate a switchtable:
when (readlnOrNull()) {
"y", "Y", "yes", "Yes", null -> …
}
AFAIK, the when statement is optimal here, because it uses less memory and is faster than all other options. I think the explicit listOf is actually useful here: it emphasizes that a temporary list will be created, which leads users to search for a more optimal way to write this code (this is a fairly common question in the Kotlin Slack's #getting-started, I think I've answered it two or three times in the past year).
If we're going to have a shorter syntax here, then it should be the optimal way of writing this code. For example, the compiler could recognize that we're using in on a collection that contains only literals, and replace the expression by a switchtable.
My point isn't really about this specific example, but more about the general approach: we've lived well without collection literals so far, so if we're adding them now, how can they help us make the language actually easier, not just a few keystrokes shorter?
I think having in ["foo", "bar"] be a compiler intrinsic, and not 'just' a collection literal, provides a lot more value to this feature. First, because it makes the optimal solution for a common pattern simpler. Second, because this is code newcomers intuitively want to write already, and there is thus no learning curve.
I think we should take advantage of the fact that newcomers will want to use collection literals everywhere, to silently make these patterns optimal without actually using collections everywhere we can. I'm sure there are many more cases of the same pattern. And, worst case, you can always opt-out by using listOf.
overestimating their need will lead to rushing the implementation
Collection Literals are being proposed by people in one form or another since over a decade ago and is one of the most requested feature. Not sure what you mean by "rushing" here. They are way overdue.
// before 1 if (readlnOrNull() in listOf("y", "Y", "yes", "Yes", null)) { // ... } // after 1 if (readlnOrNull() in ["y", "Y", "yes", "Yes", null]) { // ... }
An important part of the Kotlin design is how the shortest code is (almost) always the best code. Is
in []the best code? What's[]here?By making this code simpler we are emphasizing it to be correct to newcomers. Is this the kind of code we want Kotlin developers to write? AFAIK, this example is currently discouraged in code review...
This is common idiomatic code in several languages including Python, Rust etc. Although it is more common to use tuples instead of magic desugaring.
If we're going to have a shorter syntax here, then it should be the optimal way of writing this code. For example, the compiler could recognize that we're using in on a collection that contains only literals, and replace the expression by a switchtable.
See the desugaring link I shared above.
Hi, I'm chiming in to mention that I don't really think that Kotlin's lack of collection literals is a chronic problem that warrants a new syntax to solve. I unfortunately don't find any of the items enumerated in the Motivation section of the proposal compelling, whereas I think there's some amount of drawback to the new syntax that's added to support this.
I share some of the opinions from @CLOVIS-AI above, particularly:
[Collection literals] are definitely not a major issue at the moment, and I think that overestimating their need will lead to rushing the implementation.
I'm not sure if working on this proposal drains resources from other areas of language development, particularly things like robust pattern matching, but I'd love to see resources invested in areas that are major problems for the Kotlin language rather than collection literals 😄.
I think that some of the restrictions on the of operators are unintuitive and complicate the mental model when working with operators enough that I think it warrants not treating them as an operator. Especially:
Restriction 1. Extension operator fun of functions are forbidden. All operator fun of functions must be declared as member functions of the target type Companion object.
(which is particularly unfortunate since it prevents ever creating collection literals for external types)
Restriction 3. All of overloads must have non-nullable return type equal by ClassId to the type in which static scope the overload is declared in.
Restriction 4. All of overloads must have the same return type. If the return type is generic, the constraints on type parameters must coincide.
Restriction 7. All of overloads must have no extension/context parameters/receivers.
Restriction 8. operator fun of functions are not allowed to return nullable types.
When you put all of these together, they don't act like regular Kotlin functions. And I think the mental model of "functions (including operators, which are just functions you call differently) behave like this, except this particular function that behaves differently" is much harder to work with than "functions behave like this, and collection constructors behave like this".
Also, when you put all of those restrictions together, it ends up looking a lot like the restrictions on constructors. Was any consideration given to making them constructor-like? E.g. collection-constructor(vararg ...) in the type.
Different issue: when I want a particular type of collection that can't be inferred, it seems quite awkward to get it via a collection literal. Lets say I want to pass a TreeSet to a method that takes Set. IIUC I would need to define a TreeSet local variable for the collection literal and then pass that in. This is very awkward. If I can cast it inline ([a, b] as TreeSet) it's a little better, but still awkward. Was some syntax to specify the type you wanted considered? For example, TreeSet[a, b]. That syntax works particularly nicely if we treat the operator methods as constructors instead.
I've wanted collection literals for a long time, so I'm happy to see this. But I am disappointed that maps literals are out of scope. I would guess that the use of collection and map literals in most of my code is probably 50/50 so this definitely seems like a half solution to me.
I can see why they would want to start with just list-like collections, but I agree, I hope we get maps eventually too.
My understanding through talking with other people in the ecosystem and reading prior issues was that collection literals were not being implemented yet because it was complex to decide how they would work with custom types, and because the team was researching an alternative to varargs as an initialization mechanism. Since this design is based on varargs, have those concerns been eliminated somehow?
The Performance section mentions that varargs are faster for array-backed collections, but that's just ArrayList, isn't it? All other data structures we use daily (HashSet, LinkedHashSet, HashMap, LinkedHashMap and all custom types) are most likely not array-backed (or we would use ArrayList). We're talking about introducing new syntax here, so rules can be bent a bit. Is there really no design that can improve these use-cases?
What will be the behavior with custom types delegating to existing collections?
value class MyCustomList(private val data: List<String>) : List<String> by data {
companion object {
operator fun of(vararg items: String) = List.of(*items)
}
}
The spread operator enforces an array copy here, right? Doesn't that negate the claims in the Performance section that a single array will be created? value class so far could be used as a free (or near-free) passthrough in hot paths to make code readable without performance impacts.
I want to thank everyone involved on this KEEP for making so thorough. I can't help but think, however, that this is a lot of complexity for the single purpose of removing 6 characters in listOf(10). The provided conversion examples are not that much better and in many cases require explicit type declaration, which is overall discouraged in Kotlin. It also seems that such a complex resolution algorithm will close many doors in the future by making resolution more complex than it currently is, no?
Especially since this is all varargs all the way down anyway, has the team considered adding the get operator hack to the standard library?
interface List<T> : Collection<T> {
…
companion object {
operator fun <T> get(vararg items: T): List<T>
}
}
It seems to me that this would provide all the same benefits, with the only downside that [5] must be written List[5]. Everything else brought by this KEEP is already available that way, without any of the ambiguities and without requiring any language change.
I understand that not having to write the type is part of the Kotlin spirit, but this KEEP has a lot of limitations and downsides to achieve something we can almost do today, and this KEEP includes a mention that it will forbid this syntax on types for which collection literals are made available, which means we won't be able to use this trick even when it leads to shorter code.
I love the idea of including list literals, but the whole question of what to do about custom types, mutable collections, etc... it dramatically complicates the feature and dramatically complicates the experience of reading and understanding the call site for very little win.
Limiting literals to immutable collections simplifies things for everyone — beginner programmer, staff engineer reading PRs for footguns, spec writer, and Kotlin compiler implementer. The literal is an immutable list literal, full stop. It solves the 95% case, data literals in code. And if you want the 5% case, no problem: you can use extension functions to convert the literal into the desired type, or just use the old constructor functions.
It won't win a code golfing contest, but character count isn't THAT important. Certainly not more important than the mental overhead imposed by contextual semantics.
The implied restrictions and complicated resolution rules make me even more yearned for #348 (Type guided APIs List.of, List.empty, MutableList.of, ArrayList.of, and being able to of on external types) or just always requiring the type (List [], Set [], but no bare []) for a much simplier mental model.
Will collection literals that doesn't capture function environment being compiled to singletons?
I really like how thorough this KEEP doc is. Prior to reading it I didn’t appreciate all of the ways collection literals interact with the type system!
I dislike the asymmetry introduced here. Previously the following functions were syntactically symmetric:
-
listOf() -
setOf() -
listOfNotNull() -
mutableListOf() -
persistentListOf()
These functions fit together nicely. When I change a value from listOf to persistentListOf it doesn’t feel like I’m giving anything up. But with this syntax, that approach seems worse.
I fear developers will start to make their code perform worse once this syntax is available:
- val words = mutableListOf("foo”, "bar")
+ val words = ["foo”, "bar"].toMutableList()
Although several important concerns have been raised, I believe the arguments against the syntactic value of collection literals overlook the broader picture of Kotlin’s syntax. Consider the following characteristics of the language:
- No semicolons
- No
newkeyword - Trailing lambdas
- Single-expression functions
-
:instead ofextends/implements - Lambas created with only brackets -- no need for a
funkeyword - ... many more
Kotlin’s focus on conciseness is clear. However, using listOf, setOf, and similar functions to declare basic collections like lists feels inconsistent in this aspect. Also, almost every programming language uses braces for lists, so it's quite unintuitive for newcomers and even experienced developers working with different languages.
Regarding the migration to the new style, this would be only a matter of configuring a linter, and I'm sure the IDEA folks will also implement an inspection suggesting the new syntax whenever it makes sense, just like it's done for several other features.
@edrd-f Kotlin’s focus on conciseness is clear. However, using listOf, setOf, and similar functions to declare basic collections like lists feels inconsistent in this aspect. Also, almost every programming language uses braces for lists, so it's quite unintuitive for newcomers and even experienced developers working with different languages.
It's also very unintuitive for newcomers to have collection literals that only work in some situations.
Consider the following case, where we want to call a function foo(Set<Int>).
fun bar() {
println(foo([1, 2, 3]))
}
Let's say the list is starting to become a bit too big, and we want to put it in a local variable.
fun bar() {
val tmp = [1, 2, 3]
println(foo(tmp)) // Type mismatch, found List<Int>, expected Set<Int>
}
I bet that this will be particularly unintuitive to newcomers.
Whereas, currently,
fun bar() {
val tmp = setOf(1, 2, 3)
println(foo(tmp))
}
is perfectly readable for anyone, no matter their level of experience, and there is no surprise.
And, sure, IntelliJ's "introduce local variable" will insert the type declaration for you so the behavior doesn't change, but newcomers don't use that. Also, this means all cases of using a set in a local variable will look like:
fun bar() {
val tmp: Set<Int> = [1, 2, 3]
println(foo(tmp))
}
where the type has to be written explicitly, which I'd argue is more anti-Kotlin than calling a top-level setOf function, and is definitely "less concise". Also, newcomers can easily CTRL CLICK a setOf function to know what it does, which won't be possible (or much harder) with literals.
I give this example because I do think it will trip people up, but as mentioned in the KEEP, this won't be the first time such a difference is introduced. A typical example would be emptyList(), which doesn't need a type parameter when passed as a parameter, but does require one when used in a local variable. However, this example is much more 'in-your-face' as it will happen each time a non-List collection will be used.
@CLOVIS-AI sure, there are many examples where it can be confusing, but where do we draw the line to what is acceptable for beginners and what's not? If newcomers need to learn why it's necessary to write listOf instead of [], why wouldn't they be able to learn why it's necessary to type the collection explicitly in the example you gave?
My point is that the list syntax is inconsistent with the tradition of conciseness in Kotlin. Also, I believe people are underestimating the importance of a shorter syntax for some use cases. For example, I have a personal project similar to punkt and I chose to use Groovy for the DSL instead of Kotlin because I need to use a lot of lists and typing listOf every time is a waste of time in a context where I need to live code fast.
I can see how it enforces the "spirit" of being clean and elegant, but I still don't think saving a few characters for the type is worth this complexity during type resolution, and there are cases where you can only return to plain old functions, which is surely another style split. Please, don't rely on lint when designing a language because it will never work well.
Also, to me val l = List [0, 1, 2], val s = Set<Int> [] feels much more Kotlin-y than val l: List<_> = [0, 1, 2], val s: Set<Int> = [] because it is said that they're "constructor"s.
This is overall an amazing feature implemented in probably the best way possible for the language. The decision to use of for implementation is very clever as a way to get automatic compatibility with Java libraries, and I think the removal of List [ ... ] syntax from the original proposal is smart since it retains the existing functions like listOf() without making them look too dated.
However, I don't know why limited-size tuples are excluded from this proposal by the seemingly arbitrary requirement for a vararg argument in one of the of function overloads. Especially with the operator function being named "of", one would assume that it should work in a case like Vector3.of(x, y, z), and therefore support the syntax [x, y, z]. I think the only argument I can think of against this is to prevent the collection builder syntax from being enabled for Java classes that aren't actually collections or tuples (e.g. Optional.of(value)), but in that case, the restriction could be kept only for recognizing Java classes and not for implementing Kotlin classes where one can choose whether or not to use the operator keyword.
Also, under the current proposal it would already be possible to abuse the syntax and make a dummy vararg function that throws an error and has a @RequiredOptIn to prevent calls to it from being compiled. Even if the syntax isn't officially intended, it is probably going to be used quite often by authors of libraries and especially for DSLs, because of the benefit of shorter syntax and hiding internal types from users. Thus, it seems like explicitly supporting tuples in the collection literals syntax would result in cleaner code in these cases.
In the "Motivation" section:
- Special syntax for collection literals helps to resolve the emptyList/listOf hassle. Whenever the argument list in listOf reduces down to zero, some might prefer to clean up the code to change listOf to emptyList. And vice versa, whenever the argument list in emptyList needs to grow above zero, the programmer needs to replace emptyList with listOf. It creates a small hassle of listOf to emptyList back and forth replacement. It's by no means a big problem, but it is just a small annoyance, which is nice to see to be resolved by the introduction of collection literals.
The emptyList function was introduced to declare intent. To resolve this hassle, deprecating the emptyList function should be enough, effectively rejecting the initial design.
However, I don’t understand how introducing a new syntax (a third one) for creating an empty list helps to address this issue.
In the "Motivation" section:
- ... For example, val x = listOf(10) is potentially confusing, because some readers might think that a new collection with the capacity of 10 is created.
At the same time, the solution proposed in this KEEP defines the operator List.of.
Therefore, it is not clear to me whether the syntax "list of 10" is a good choice or a bad one.
And new users have the right to naively believe that Kotlin supports it.
I agree; novices have the right to naively believe whatever they want, but that doesn’t make it true.
The feature brings more value to newcomers rather than to experienced Kotlin users and should target the newcomers primarily.
This feature has a broad impact on language, not just on newcomers. We should consider its potential effects, not just its intended purpose.
Examples in "Motivation" section:
// before 6
for (key in listOf(argument.value, argument.shortName, argument.deprecatedName)) {
if (key.isNotEmpty()) put(key, argumentField)
}
Same of example 1.
Java. Java explicitly voted against collection literals in favor of of factory methods.
There is a limited support for array literals in Java: String[] a = {"Hello", "World"};
And when the expected type (See the definition below) is unspecified, expression must fall back to the kotlin.List type
"The feature brings more value to newcomers rather than to experienced Kotlin users", so should the default type be a mutable list? The issue of mutability is not trivial, and I believe that learning a new language is not the right time to deal with it. However, I would fully agree with that statement if this feature were also intended for experienced developers.
It's always possible to add a special syntax for maps in future versions of Kotlin. To start with, we want to concentrate on collections. That's why we limit the proposal only for collections for now.
Languages that support collection literals, support dictionary literals too, "and new users have the right to naively believe that Kotlin supports it".
This design looks not "friendliness to newcomers", because writing val list=[1] and get a compilation error on val map=["a": 1] is confusing.
This not "makes a good first impression on the language".
"Contains" optimization
This code should work better:
if(when (readlnOrNull()){"y","Y","yes","Yes",null->true else->false})
The feature brings more value to newcomers rather than to experienced Kotlin users and should target the newcomers primarily.
This feature has a broad impact on language, not just on newcomers. We should consider its potential effects, not just its intended purpose.
This feature has a large positive impact on DSLs designed to look and behave like configuration files, as well as data science and scripting use-cases.
Another area where it has a large, unambiguous benefit is when declaring default values for class and method parameters. In these cases you can replace val list: List<String> = emptyList() with val list: List<String> = []. This case appears very often.
I agree that the feature may introduce some confusion in the rest of the language but I think the impacts are rather small. The primary regression is that creating non-List collections like
val hashSet = hashSetOf(1, 2, 3)
is replaced by
val hashSet: HashSet<_> = [1, 2, 3]
which is actually more verbose than the previous syntax, but I would argue that creating a non-List collection by elements actually occurs quite infrequently in Kotlin, and sometimes even in these cases, it is already better (i.e. clearer in code) to instantiate the collection first and then add the needed element(s) manually.
Then there are the concerns regarding there being too many ways to instantiate collections as well as migration of existing code. The migration issue is valid and quite serious but I'd argue that it's temporary - sometimes languages just need to evolve regardless of the existing codebases or you end up with very outdated syntax. As an analogy, I don't think anyone today would argue against Java 9 adding List.of just because there is a large pool of existing Java code creating collections by manually adding elements to ArrayLists.
As for having too many ways to instantiate collections, I'd argue that it just comes from backwards compatibility and that it is better than the alternative of deprecating common/stable features which essentially equates to having incompatible versions of languages like Scala 2/3. Furthermore, I don't think it's actually a big issue, since in some cases in Kotlin, there are already multiple alternative syntaxes for the same operation such as the "get" or "set" operators where both the bracket syntax and the original function call are used often and the latter is needed for nullable calls. With that in mind, it would not be that inconsistent to have collections be instantiated sometimes by the existing global functions and other times by collection literals.
In the "Motivation" section:
- Special syntax for collection literals helps to resolve the emptyList/listOf hassle. Whenever the argument list in listOf reduces down to zero, some might prefer to clean up the code to change listOf to emptyList. And vice versa, whenever the argument list in emptyList needs to grow above zero, the programmer needs to replace emptyList with listOf. It creates a small hassle of listOf to emptyList back and forth replacement. It's by no means a big problem, but it is just a small annoyance, which is nice to see to be resolved by the introduction of collection literals.
The
emptyListfunction was introduced to declare intent. To resolve this hassle, deprecating theemptyListfunction should be enough, effectively rejecting the initial design.However, I don’t understand how introducing a new syntax (a third one) for creating an empty list helps to address this issue.
In my opinion, thinking of the meaning of listOf() with no parameters introduces some extra cognitive load in the way that the [] syntax doesn't have. Therefore [] replaces listOf() and also removes the need for emptyList().
Another area where it has a large, unambiguous benefit is when declaring default values for class and method parameters. In these cases you can replace val list: List<String> = emptyList() with val list: List<String> = []. This case appears very often.
I don't find this argument convincing in the slightest unfortunately, and it seems a bit like trying to play code golf.
... more verbose than the previous syntax ...
I personally want to state that verbosity and complexity should not be conflated and are in fact different things. I don't find the argument of reducing the creation of a collection by ~7 ASCII characters convincing, and I do think that there is a non-negligible complexity increase to implementing these collection literals as they're proposed.
I would argue that creating a non-List collection by elements actually occurs quite infrequently in Kotlin, and sometimes even in these cases, it is already better (i.e. clearer in code) to instantiate the collection first and then add the needed element(s) manually.
Do you have data to support this claim? I find it hard to believe that other collection functions like setOf and mapOf are infrequently used. I use them quite frequently!
I just checked for some quick numbers, seeing how many Kotlin files on GitHub use each type of function, which I think is a good point of reference in general.
Show table...
| function + empty variant | function with no elements | empty variant | |
|---|---|---|---|
listOf |
1.7M | 179k | 586k |
mutableListOf |
791k | 692k | |
mapOf |
446k | 34.2k | 84.5k |
setOf |
259k | 28.7k | 64.3k |
mutableMapOf |
246k | 227k | |
arrayListOf |
145k | 106k | |
mutableSetOf |
115k | 110k | |
hashMapOf |
65k | 43.8k | |
listOfNotNull |
45.1k | 409 | |
hashSetOf |
24.1k | 16.6k | |
linkedMapOf |
8.6k | 5.1k | |
linkedSetOf |
4.9k | 3.6k | |
sortedSetOf |
3.9k | 2.9k | |
sortedMapOf |
2.7k | 1.6k | |
setOfNotNull |
1.3k | 48 |
Do you have data to support this claim? I find it hard to believe that other collection functions like
setOfandmapOfare infrequently used. I use them quite frequently!
I was mostly referencing setOf() (and mutableListOf()/arrayListOf()) and basing it off of personal experience. Sets are usually used to deduplicate elements or as part of a specific algorithm, and in both cases you typically create empty mutable sets, sets with one element, or sets with elements from other collections. Collection literals make no difference in these cases as currently I would use syntaxes HashSet(), val set = HashSet(); set += element, and val set = HashSet(collection) instead
mapOf() is indeed frequently used but maps are not part of the scope of this KEEP and presumably a literal for maps would be added to the language in the future.
Another area where it has a large, unambiguous benefit is when declaring default values for class and method parameters. In these cases you can replace val list: List = emptyList() with val list: List = []. This case appears very often.
I don't find this argument convincing in the slightest unfortunately, and it seems a bit like trying to play code golf.
In this case the collection literal is removing the need to mention the type List twice so I think it is actually improving the syntax rather than just shaving off some characters. This improvement is analogous to when Java added the ability to infer type parameters (List<String> x = new ArrayList<>();), which similarly removed the need to mention a type twice in the same line and I think made a considerable improvement in readability of Java code.
Plus, even just shaving off a few characters alone does count towards code clarity in a function/class definition, as it often allows you to keep it in one line rather than splitting it up into multiple lines.
... more verbose than the previous syntax ...
I personally want to state that verbosity and complexity should not be conflated and are in fact different things. I don't find the argument of reducing the creation of a collection by ~7 ASCII characters convincing, and I do think that there is a non-negligible complexity increase to implementing these collection literals as they're proposed.
Isn't this true for most/all of Kotlin's operators in general? Features like delegation, get/set operator, etc all add complexity to the language too but in the end make the syntax simpler and thus easier to reason about. Yes, there is a lot of hidden complexity behind delegation but as a user, why do I need to care? I just see that my lazy and Android mutableState variables are working as expected.
In the same way, collection literals make the language more complex but nonetheless can reduce mental load for DSLs, data science, providing parameters to function calls, and when instantiating regular Lists.
When I'm using collection literals I don't have to care about all the implementation-side operator fun of functions and all their restrictions. All I have to think about is that typing [x, y, z] provides a collection of the type expected by the function I'm calling. This is less effort than right now, where I need to figure out whether the function I'm calling wants an Array, primitive array, List, MutableList/ArrayList, Set, or MutableSet/HashSet and then call the right collection builder to pass the correct type to the function.