Value for serializer ContextDescriptor(...) should always be non-null when trying to serialize null (`Nothing?`)

[SerVB]

Describe the bug

The following code fails in runtime with:

Exception in thread "main" java.lang.IllegalArgumentException: Value for serializer ContextDescriptor(kClass: class java.lang.Object (Kotlin reflection is not available), original: kotlinx.serialization.Polymorphic(type: kotlin.String, value: kotlinx.serialization.Polymorphic<Any>)) should always be non-null. Please report issue to the kotlinx.serialization tracker.
 at kotlinx.serialization.json.internal.StreamingJsonEncoder.encodeSerializableValue (StreamingJsonEncoder.kt:249) 
 at kotlinx.serialization.encoding.AbstractEncoder.encodeSerializableElement (AbstractEncoder.kt:80) 
 at Response$OK.write$Self$web_module (File.kt:7) 

To Reproduce

Playground: https://pl.kotl.in/SHyoGHTDb.

import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json

@Serializable
sealed interface Response {
    @Serializable
    data class OK<T>(val data: T) : Response
    @Serializable
    data class Error(val error: String) : Response
}

@Serializable
class Payload(val foo: Int)

object Playground {
    @JvmStatic
    fun main(args: Array<String>) {
    	println(Json.encodeToString(Response.serializer(), Response.OK(null)))
    }
}

Expected behavior

It's serialized successfully either to {type: ok, data: null} or {type: ok}.

Environment

• Kotlin version: 2.0.20
• Library version: playground
• Kotlin platforms: JVM
• Gradle version: playground

[SerVB]

So the following additional constructor method solves the problem: https://pl.kotl.in/s13AMnKpB. It's a workaround for me now

[pdvrieze]

I've played around a bit with it. The problem is that you are using polymorphic serialization with a parameterised child type (T). If you leave out Response.serializer() it works, because you are then not in polymorphic mode. Note also that it doesn't work either if you serialize an OK<String>("foo") or OK<Payload> for the same reason, it needs to find the child serializer.

In polymorphic mode the system needs to find out the type of the child parameter (which is even harder in decode mode). The way to deal with this requires the following:

• Use a custom serializer for Response (it might also work for OK)
• In the OK class, store the actual serializer to be used (this is technically an optimization, but it also fixes working with nulls - the error is that contextual serializer is not working for null - and using Nothing? is pointless as it only has null as instance)
• Then when serializing instances of OK use the stored serializer to serialize the value
• Add an inline factory function to create OK instances with reified content type.
• In serializing/deserializing the response you also need to record the content serializer.

I've created an implementation of this: https://pl.kotl.in/RfavV2alG

But you may actually make this work better by having the type parameter be part of the response (so it works without most tricks and without polymorphic modules, even for reading):

@Serializable
sealed interface Response<out T> {

    @Serializable
    data class Success<T>(val data: T) : Response<T>

    @Serializable
    data class Error(val message: String): Response<Nothing>
}

[sandwwraith]

You can achieve the same workaround without invoke, just by using val data: T? instead of non-nullable T. The reason for this is described here: https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md#polymorphism-and-generic-classes. When using generic subclasses, we cannot know in advance what T is, so we use PolymorhicSerializer(Any::class) for it, which generally doesn't permit nulls. In case you want to deserialze any other data with it, you need to register its serializer as a polymorphic subclass of Any (see the link for details)

[sandwwraith]

See also #2555, #2729