Context receivers

[shadrina]

The goal of this proposal is to introduce the support of context-dependent declarations in Kotlin, which was initially requested under the name of "multiple receivers" (KT-10468).

See the proposal text here.

[fvasco]

In the section VM ABI and Java compatibility, the Kotlin code:

context(C1, C2)
fun R.f(p1: P1, p2: P2)

should be equal the Java code

public static final kotlin.Unit f(C1 c1, C2 c2, R r, P1 p1, P2 p2)

Kotlin does not allows void return type, it should be Unit

[altavir]

First of all, let me thank you for a superb (and long-awaited) proposal.

As for the discussion, my primary concern is still the syntax. The proposal mentions that the context(T) construct could be mixed with a function call in the code and it should be treated properly. But what Is more important, it confuses the reader. When I read the code, I can't see clearly that this declaration is attached to the following declaration. I can understand the reasoning behind not using @ (a lot of people have negative emotions about that), but some kind of visible syntax would be nice. Another concern is about using round brackets for types. In kotlin, we always use round brackets for instances, so context(MyContext) reads as a companion object of MyContext, not as the type itself. In Groovy one can use class name instead of class and it brings a lot of confusion sometimes. So I would recommend at least using triangle brackets <> for receiver type to provide clear separation. Like context<MyContext> fun A.doB().

The second point is about the proposal itself. One of the important uses of multiple receivers is the ability to flexibly define behavior intersections. For example, consider this code:

interface A

interface B

context(A,B) fun doSomething()

It could be called as with(a,b){ doSomething}, but in some cases, we also can have:

interface C: A, B

c.doSomething()

This seems to be an idiomatic example of using multiple receivers and I think, it should be covered in the proposal.

[fluidsonic]

Great and thorough proposal with quite a lot of history 🙌

context Syntax

I thought exactly the same as @altavir. Here it's especially bad: var g: context(Context) Receiver.(Param) -> Unit. Looks like a call within a function body.

context<A, B> is easier to understand and also makes more sense to me.

• You specify types. You don't pass values.
• You have a context that is "parameterized" by A and B.
• No backward compatibility issue as context<A> fun … isn't valid Kotlin code today (afaik) even with a line break.
• Generics like context<Comparator<T>> still aren't an issue.

Could also be context<A & B> instead, having future union types in mind. May cause problems with this@A syntax. And then later on allows typealias C = A & B with context<C>.

Builders

The use case Creating JSONs with JSONObject and custom DSL is basically a builder DSL. However in Contexts and coding style you write "Context receivers shall not be used for such builders". I'd say it depends on the DSL's purpose whether or not a context makes more sense.

[edrd-f]

I think the proposal should go deeper on how context receivers would work with annotations and the suspend modifier. It only says:

Also, suspend and @Composable functions can be retrofitted to work as if they are declared with context(PropertiesScope) modifier and so will pass a set of current context properties via their calls, ensuring interoperability of the corresponding mechanisms.

However, while this retrofitting doesn't happen, how would these functions be declared? There are several options:

1:

@DslMarker context(JsonBuilder) fun String.by(obj: @DslMarker context(JsonBuilder) suspend () -> JsonObject)

2:

context(JsonBuilder) @DslMarker fun String.by(obj: context(JsonBuilder) @DslMarker suspend () -> JsonObject)

3:

context(JsonBuilder) @DslMarker fun String.by(obj: context(JsonBuilder) suspend @DslMarker () -> JsonObject)

4:

context(JsonBuilder) @DslMarker fun String.by(obj: suspend context(JsonBuilder) @DslMarker () -> JsonObject)

[fluidsonic]

@mcpiroman it's already mentioned as a potential future extension.

[fluidsonic]

Another naming-related issue came to mind:

• We have context on the consuming side.
• We have with on the providing side.
• We have withContext in kotlinx-coroutines.

I already found code that accidentally used with instead of withContext. With the new context soft keyword this will probably get more confusing.

But it may also be an issue with kotlinx-coroutines because withContext is a really generic name.

[edrd-f]

@altavir @fluidsonic - about the suggested <> syntax, the problem is that in context<Comparable<T>>, Comparable is a literal type while T is a generic type, so the semantic meaning of <> gets ambiguous.

[fluidsonic]

@edrd-f

@altavir @fluidsonic - about the suggested <> syntax, the problem is that in context<Comparable<T>>, Comparable is a literal type while T is a generic type, so the semantic meaning of <> gets ambiguous.

I'm not sure I understand you correctly. It could also be context<Comparable<Int>>. Then Int is a literal type used as an argument to a generic parameter. The same way context<Foo> has Foo as an argument to a generic parameter that is unnamed and brought into scope in the function (more or less).

We're entering the ambiguous realm anyway. Consider object Foo. context(Foo) in the fun position is quite different from context(Foo) in regular code.

[altavir]

@edrd-f I don't not see the problem, because we use the receiver type here exactly the same way, we would use it in any other generic case like typeOf<> which takes the type parameter. And in any case, it is better than round brackets since there is no distinction from the function call and, as I already said, association with the companion.

I would prefer something even more distinct, But triangular brackets are better than round ones.

[fvasco]

@edrd-f please consider that context<Comparable<T>>() is already valid Kotlin syntax.

[BenWoodworth]

The main tradeoff we had to make with the proposed syntax is that in case when context is generic, then the use of the generic parameter happens before its declaration, e.g (from Use cases section):

context(Monoid<T>) // T is used
fun <T> List<T>.sum(): T = ...
//  ^^^ T is declared

I might've missed something, but why not put the contexts between <T> and the function's receiver/name? Noise/parsing issues?

fun <T> context(Monoid<T>) List<T>.sum(): T = ...

[mcpiroman]

I might've missed something, but why not put the contexts between <T> and the function's receiver/name? Noise/parsing issues?

fun <T> context(Monoid<T>) List<T>.sum(): T = ...

The problem is, generally, the name matters the most (that's why it's before type) but with such syntax it is shifted near the end of the line. When looking at the function's definition, the context it has is more of a implementation detail, yet, you have to read it first before you get to the name or parameters, which are more important. Secondly, this aligns more with where annotations are put and this feature for me is closer to annotations.

[b-camphart]

to @mcpiroman's point, maybe the syntax can put the context in the next line, similar to the existing where syntax:

fun <T> List<T>.sum(): T     
context(Monoid<T>) { 

}

It would follow an existing kotlin design pattern and reduce the noise in the main function definition line. Basically, the fact that the function requires additional context is a detail and thus should be below the main part of the definition.

I think this would also help with discoverability on the calling side. Instead of the example sum function simply not existing without all the extra context and thus having to go hunt for the definition, instead it would be treated like an extension function with missing parameters and simply show you a red underline (and tell you what context parameters are missing).

[rnett]

Big +1 for the Scope Properties and Contextual Classes proposals. Scope properties would make my compiler plugins quite a bit nicer (put IrPluginContext in context and a few other things), and contextual classes is exactly what I need for a deep learning library (so that layers can only be defined inside a graph).

[rnett]

to @mcpiroman's point, maybe the syntax can put the context in the next line, similar to the existing where syntax:

fun <T> List<T>.sum(): T     
context(Monoid<T>) { 

}

It would follow an existing kotlin design pattern and reduce the noise in the main function definition line. Basically, the fact that the function requires additional context is a detail and thus should be below the main part of the definition.

This was considered and rejected with

This placement would be consistent with Kotiln's where clause, but it is not consistent with receivers being specified before the function name. Moreover, Kotlin has a syntactic tradition of matching declaration syntax and call-site syntax and a context on a call-site is established before the function is invoked.

which I very much agree with, I don't like having parameters after the declaration.

[YarnSphere]

Great proposal, thank you!

Syntax wise, in the spirit of solving the "usage of generic before its definition" I'd like to propose something perhaps slightly more verbose but that I think reads quite well and I didn't see suggested:

context fun <T> with Monoid<T> List<T>.sum(): T = ...

or (smaller and because in is already an existing keyword):

context fun <T> in Monoid<T> List<T>.sum(): T = ...

With multiple receivers and suspend:

context suspend fun <T> with/in Monoid<T>, Scope<T> List<T>.sum(): T = ...

In my opinion, it turns context into a function modifier that reads very similarly to suspend while still allowing the definition of generics before their usage. However, it introduces an additional keyword in the function definition, making it slightly more verbose (although the removal of the parenthesis might make the syntax less "noisy").

In type form, something like this could be used:

val sum: context suspend with/in Monoid<T>, Scope<T> List<T>.() -> T

However, in type form, since (if I'm not mistaken) the "usage of generic before its definition" is not a problem, I'd be in favour of keeping the originally proposed syntax (or the, imo better variant, with triangle brackets).

The problem is, generally, the name matters the most (that's why it's before type) but with such syntax it is shifted near the end of the line.

With this in mind, the contexts could also be moved to the end:

context fun <T> List<T>.sum(): T with Monoid<T> = ...

or, because this could read as "returning a T together with a Monoid<T>", a different keyword could be used:

context fun <T> List<T>.sum(): T given Monoid<T> = ...

which would read as "returning a T given a Monoid<T>. Although, according to the proposal, given means something different in Scala which might be a problem and given could also be confused to mean an argument (return T given an argument of type Monoid<T>).

With multiple receivers and suspend:

context suspend fun <T> List<T>.sum(): T with/given Monoid<T>, Scope<T> = ...

And in type form (although, once again, the originally proposed syntax could still be used):

val sum: context suspend List<T>.() -> T with/given Monoid<T>, Scope<T> 

All in all, I enjoy the idea of having context as a modifier akin to suspend. However, maybe the "usage of generic before its definition" is not a problem worth solving.

Regardless, I just thought I'd provide my 2 cents on the syntax (as everyone enjoys doing whenever new proposals appear :stuck_out_tongue:) and I'll be happy to use whatever comes out in the end. :)

[TheBestPessimist]

Hello,

I'm looking at the example for AutoCloseScope

interface AutoCloseScope {
    fun defer(closeBlock: () -> Unit)
}

context(AutoCloseScope)
fun File.open(): InputStream

fun withAutoClose(block: context(AutoCloseScope) () -> Unit) {
    val scope = AutoCloseScopeImpl() // Not shown here
    try {
        with(scope) { block() }
    } finally {
        scope.close()
    }   
}

// usage
withAutoClose {
    val input = File("input.txt").open()
    val config = File("config.txt").open()
    // Work
    // All files are closed at the end
}

However I still cannot understand how this would work IRL.

1. What is scope.close()? I do not see that declared anywhere.
2. What is the purpose of AutoCloseScope.defer? That is not used anywhere.

The way I imagine using this example is

interface AutoCloseScope {
    fun defer(closeBlock: () -> Unit)
	fun close() // This does not exist in the example
}

context(AutoCloseScope)
fun File.open(): InputStream {
	defer { [email protected]() } // this defer must be added for every different type that we want to autoclose, correct?
	return [email protected]() // Didn't work with files a lot. Please accept this pseudocode
}

fun withAutoClose(block: context(AutoCloseScope) () -> Unit) {
    val scope = AutoCloseScopeImpl() // Shown below
    try {
        with(scope) { block() }
    } finally {
        scope.close()
    }   
}

class AutoCloseScopeImpl : AutoCloseScope {
    private val closeables = mutableListOf<() -> Unit>()

    override fun defer(closeBlock: () -> Unit) {
        closeables += closeBlock
    }

    override fun close() = closeables.asReversed().forEach { it.invoke() }
}

// usage
withAutoClose {
    val input = File("input.txt").open()
    val config = File("config.txt").open()
    // Work
    // All files are closed at the end
}

Is my understanding correct? Do you think of a different alternative?

[elizarov]

@fvasco In the section VM ABI and Java compatibility, the Kotlin code .... ... Kotlin does not allows void return type, it should be Unit

The example in the text is correct. Unit-returning Kotlin functions are compiled to void functions on JVM. Unit is carried over in functional types, though.

[elizarov]

@altavir First of all, let me thank you for a superb (and long-awaited) proposal. As for the discussion, my primary concern is still the syntax.

Thanks a lot for bringing up the context(Ctx) vs context<Ctx> discussion. In fact, it was one of our biggest design discussions earlier in the design process, but we've failed to include it into the resulting text. I've added the corresponding section to the document. Please find the detailed answer in the Parentheses vs angle brackets section.

[elizarov]

@altavir ... It could be called as with(a,b){ doSomething}, but in some cases, we also can have:

interface C: A, B

c.doSomething()

This seems to be an idiomatic example of using multiple receivers and I think, it should be covered in the proposal.

That's a very interesting extension to the call resolution algorithm that we did not even consider during our design discussions. I'd love to see more specific examples in the actual code-base. So far, we've been trying to narrowly constrain the set of interfaces that are "appropriate to use as context" and it seemed to us to be pretty distinct from the set of interfaces that are "appropriate to use as an object (qualifier) of the call". However, it does not mean that the intersection is zero, and it might turn out to be useful to start the greedy resolution of the context parameters with the qualifier of the call (c in your example).

[elizarov]

@fluidsonic context Syntax

I've answered on parentheses vs angle brackets above to @altavir. See the new Parentheses vs angle brackets section for answers.

Builders The use case Creating JSONs with JSONObject and custom DSL is basically a builder DSL. However in Contexts and coding style you write "Context receivers shall not be used for such builders". I'd say it depends on the DSL's purpose whether or not a context makes more sense.

Thanks for noting that. In fact, @ilya-g had noticed it, too, in the pre-publication review, but we failed to update the text to correct it. I've now added a clarification to the Kotlin builders section.

[altavir]

@elizarov thanks for the clarification (about brackets), The point about adding named arguments does not seem to be valid to me. You are introducing a new syntax anyway, something like context<name: Type> is as possible as context(name: Type), both will introduce new syntax, but the learning curve will be smoother for the first one. If we are talking about not blocking future possibilities while experimenting, won't it be better to use initially proposed annotation-like syntax @with<A, B> which does not require new entities? We can decide on replacement syntax later, when we better understand all the use cases.

As for the intersection of behaviors, it was discussed a lot inside the initial KEEP-176 proposal. The case in mathematics is a simple one. Consider that you want to do some higher-level operations on matrices. You want addition, subtraction defined in MatrixAlgebra, but you also want inversion operation which could be done in different ways on the same algebra in DecompositionOps. Basically, you pass to different contexts - one for algebra and one for decomposition, but in some cases, matrix algebra has one dedicated way of inverting, or even better algebra inherits DecompositionOps for the default inversion method. Then I want to pass a single context that will suit both type requirements.

A similar situation arises in other cases. For example, consider that we have a context-bound operation, that requires coroutine scope. In general, you require two receivers - a context and a coroutine scope. But the context could be a coroutine scope itself if it is an application scope or whatever. A recommendation to use an intersecting interface seems meaningless in this case since it could be the same object and could be not. Or even better, it is possible that you want to use application scope by default like:

with(application){
  doInContext()
}

but in some cases you want to substitute the coroutinscope:

with(application){
  withContext(Dispatchers.IO){
    doInContext()
  }
}

Such usage could produce some level of ambiguity in the resolution of receivers, but it was also discussed a lot in KEEP-176 and so far I see no problem because the receiver type is bound to the first appropriate receiver type it finds up the context tree. The resolution is context-aware and contexts are explicit in the function signature, so it seems fine.

[elizarov]

@BenWoodworth I might've missed something, but why not put the contexts between <T> and the function's receiver/name? Noise/parsing issues?

fun <T> context(Monoid<T>) List<T>.sum(): T = ...

It is about visual noise, not about parsing. We strongly feel that contexts belong to the realm of "additional annotations" for a function. They are not explicitly passed on the call site and should not obscure the reading and understanding of the regular function's signature that lists all its explicit parameters that should be mentioned on the call site. That is why we recommend formatting the context(Ctx) modifier as a separate line and we could not find a way to make it look nice if it is written after the fun. However, with the "before the fun" modifier it all plays out nicely:

context(Monoid<T>) // additional modifier, like @Transactional or @Logged
fun <T> List<T>.sum(): T = ...
//  ^^^^^^^^^^^^^^^^^ the actual call signature a reader should be primarily concerned with

[elizarov]

@nunocastromartins Syntax wise, in the spirit of solving the "usage of generic before its definition" I'd like to propose something perhaps slightly more verbose but that I think reads quite well and I didn't see suggested:

context fun <T> with Monoid<T> List<T>.sum(): T = ...

See the answer above. We've considered something along these lines (albeit failed to mention it in the text) and rejected it because it does not lend itself to the nice multi-line formatting.

[elizarov]

@TheBestPessimist I'm looking at the example for AutoCloseScope

1. What is scope.close()? I do not see that declared anywhere.
2. What is the purpose of AutoCloseScope.defer? That is not used anywhere.

It is all declared and used inside AutoCloseScopeImpl which is not shown for the conciseness of the example.

The way I imagine using this example is... Is my understanding correct? Do you think of a different alternative?

Yes. That is the way we envision it, too.

[mcpiroman]

If context was annotation-like (@context), couldn't it be declared for, say, all functions in a file at once? 🤔

[fvasco]

@nunocastromartins I consider more readable the parameters declaration, like the context's one, before the return type.

However, we need/prefer some kind of brackets? Should a comma separated list enough?

context TimeSource, TransactionContext, LoggingContext
fun doSomeTopLevelOperation() { ... }

[altavir]

@mcpiroman I think we need this functionality with any syntax, it is quite useful and I think that there are hints to it in the proposal. The problem with annotation-like markers is that they are not annotations and therefore could confuse people. Still, I am for annotation likeness since it helps a lot with my primary concern of readability and does not introduce new syntax in the language. It also is in line with what Compose does (the @Composable annotation is also a context definition). The only rule we need to introduce is not all that starts with @ is annotation and it is fine by me.

[altavir]

@fvasco

context TimeSource, TransactionContext, LoggingContext
fun doSomeTopLevelOperation() { ... }

And what about lambdas? You need always remember lambdas. And they could have additional modifiers like inline or suspend.

[fvasco]

@altavir sorry, I miss the point. Can you explain better?