kotlinx.serialization icon indicating copy to clipboard operation
kotlinx.serialization copied to clipboard

Question: How to create ser/des for generics with List and Map?

Open ArcherEmiya05 opened this issue 1 year ago • 3 comments

I cannot find any sample converting Map<K,V> and List<T> to JSON and vice-versa with generics. Planning to migrate from Moshi to KotlinX Serialization as part of adapting KMM. Currently this is what we had.

class MoshiJsonParserRepositoryImpl(private val moshi: Moshi) : JsonParserRepository {

    override fun <T> fromJsonString(jsonString: String?, type: Class<T>): T? {

        if (jsonString.isNullOrBlank()) {
            return null
        }

        return try {
            moshi.adapter(type).fromJson(jsonString)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <T> fromJsonStringToList(jsonString: String?, type: Class<T>): List<T>? {

        if (jsonString.isNullOrBlank()) {
            return null
        }

        return try {
            moshi.adapter<List<T>>(Types.newParameterizedType(List::class.java, type)).fromJson(jsonString)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <K, V> fromJsonStringToMap(
        jsonString: String?,
        keyType: Class<K>,
        valueType: Class<V>
    ): Map<K, V>? {

        if (jsonString.isNullOrBlank()) {
            return null
        }

        val jsonAdapter: JsonAdapter<Map<K, V>> = moshi.adapter(
            Types.newParameterizedType(
                Map::class.java,
                keyType,
                valueType
            ))

        return try {
            jsonAdapter.fromJson(jsonString)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <T> toJsonString(obj: T, type: Class<T>): String? {

        return try {
            moshi.adapter(type).toJson(obj)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <T> toJsonString(obj: List<T>, type: Class<T>): String? {

        return try {
            moshi.adapter<List<T>>(Types.newParameterizedType(List::class.java, type)).toJson(obj)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <K, V> toJsonString(value: Map<K, V>, keyType: Class<K>, valueType: Class<V>): String? {

        val jsonAdapter: JsonAdapter<Map<K, V>> = moshi.adapter(
            Types.newParameterizedType(
                Map::class.java,
                keyType,
                valueType
            ))

        val buffer = Buffer()
        val jsonWriter: JsonWriter = JsonWriter.of(buffer)
        jsonWriter.serializeNulls = true
        jsonAdapter.toJson(jsonWriter, value)

        return try {
            buffer.readUtf8()
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

}

Attempt to work with simple data type like data class.

    override fun <T> fromJsonString(jsonString: String?, type: Class<T>): T? {

        if (jsonString.isNullOrBlank()) {
            return null
        }

        return try {
            @Suppress("UNCHECKED_CAST")
            Json.decodeFromString(Json.serializersModule.serializer(type) as KSerializer<T>, jsonString)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <T> toJsonString(obj: T, type: Class<T>): String? {

        return try {
            @Suppress("UNCHECKED_CAST")
            Json.encodeToString(Json.serializersModule.serializer(type) as KSerializer<T>, obj)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

ArcherEmiya05 avatar Jun 17 '24 17:06 ArcherEmiya05

See https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md#constructing-collection-serializers

Normally, you shouldn't handle generic types manually since json.decodeFromString<List<MyDataClass>>(jsonString) works just fine. In rare case you need manual construction of serializer, use factory functions, e.g. ListSerializer(serializer(type)). You may also find serializer( kClass: KClass<*>, typeArgumentsSerializers: List<KSerializer<*>>, isNullable: Boolean) overload (https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-core/kotlinx.serialization/serializer.html) helpful.

sandwwraith avatar Jun 18 '24 10:06 sandwwraith

See https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md#constructing-collection-serializers

Normally, you shouldn't handle generic types manually since json.decodeFromString<List<MyDataClass>>(jsonString) works just fine. In rare case you need manual construction of serializer, use factory functions, e.g. ListSerializer(serializer(type)). You may also find serializer( kClass: KClass<*>, typeArgumentsSerializers: List<KSerializer<*>>, isNullable: Boolean) overload (https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-core/kotlinx.serialization/serializer.html) helpful.

Does this mean that KotlinX Serialization is not generic friendly? Always needing to specify data types will result on bunch of boilerplate.

ArcherEmiya05 avatar Jun 18 '24 10:06 ArcherEmiya05

Does this mean that KotlinX Serialization is not generic friendly?

It means the opposite. kotlinx-serialization has first-class support for generics, without the need for type adapters

sandwwraith avatar Jun 18 '24 11:06 sandwwraith

Does this mean that KotlinX Serialization is not generic friendly?

It means the opposite. kotlinx-serialization has first-class support for generics, without the need for type adapters

Does this mean we no longer need to create the above interface? What if we want to create an extension function?

So far this is what we ended up doing (not tested yet)

    override fun <T> fromJsonString(jsonString: String?, type: Class<T>): T? {

        if (jsonString.isNullOrBlank()) {
            return null
        }

        return try {
            @Suppress("UNCHECKED_CAST")
            Json.decodeFromString(Json.serializersModule.serializer(type) as KSerializer<T>, jsonString)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <T> fromJsonStringToList(jsonString: String?, type: Class<T>): List<T>? {

        if (jsonString.isNullOrBlank()) {
            return null
        }

        return try {
            Json.decodeFromString<List<T>>(jsonString)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <K, V> fromJsonStringToMap(
        jsonString: String?,
        keyType: Class<K>,
        valueType: Class<V>
    ): Map<K, V>? {

        if (jsonString.isNullOrBlank()) {
            return null
        }

        return try {
            Json.decodeFromString(serializer<Map<K, V>>(), jsonString)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <T> toJsonString(obj: T, type: Class<T>): String? {

        return try {
            @Suppress("UNCHECKED_CAST")
            Json.encodeToString(Json.serializersModule.serializer(type) as KSerializer<T>, obj)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <T> toJsonString(obj: List<T>, type: Class<T>): String? {

        return try {
            Json.encodeToString(serializer<List<T>>(), obj)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

    override fun <K, V> toJsonString(value: Map<K, V>, keyType: Class<K>, valueType: Class<V>): String? {

        return try {
            Json.encodeToString(serializer<Map<K, V>>(), value)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

But we are still manually handling types here.

ArcherEmiya05 avatar Jul 05 '24 12:07 ArcherEmiya05

Just tested the code above and we are already getting exception at

override fun <K, V> toJsonString(value: Map<K, V>, keyType: Class<K>, valueType: Class<V>): String? {

        return try {
            Json.encodeToString(serializer<Map<K, V>>(), value)
        }
        catch (e: Exception) {
            e.printStackTrace()
            null
        }

    }

java.lang.IllegalStateException: Captured type parameter K from generic non-reified function. Such functionality cannot be supported as K is erased, either specify serializer explicitly or make calling function inline with reified K

ArcherEmiya05 avatar Jul 05 '24 13:07 ArcherEmiya05

Use MapSerializer(serializer(keyType), serializer(valueType)).

I can also see that you are using java.lang.Class. It won't be available in common multiplatform code. I don't know where you are getting it from, but migrate to kotlin.reflect.KType if possible. Since KType also contains generic arguments, there wouldn't be any need in different overloads for Map/List/value. See example:

fun foo(): Map<String, Int> {
    return mapOf("a" to 1)
}

fun getKType(): KType = typeOf<Map<String, Int>>()

fun <T> toJson(kType: KType, value: T): String = Json.encodeToString(serializer(kType), value)

@Test
fun serializationExample() {
    // Prints {"a":1}
    println(toJson(getKType(), foo()))
}

sandwwraith avatar Jul 05 '24 13:07 sandwwraith

Use MapSerializer(serializer(keyType), serializer(valueType)).

I can also see that you are using java.lang.Class. It won't be available in common multiplatform code. I don't know where you are getting it from, but migrate to kotlin.reflect.KType if possible. Since KType also contains generic arguments, there wouldn't be any need in different overloads for Map/List/value. See example:

fun foo(): Map<String, Int> {
    return mapOf("a" to 1)
}

fun getKType(): KType = typeOf<Map<String, Int>>()

fun <T> toJson(kType: KType, value: T): String = Json.encodeToString(serializer(kType), value)

@Test
fun serializationExample() {
    // Prints {"a":1}
    println(toJson(getKType(), foo()))
}

Sorry but may I know if this approach will work with multiplatform as well? Thanks!

ArcherEmiya05 avatar Jul 05 '24 13:07 ArcherEmiya05

Yes, typeOf / KType are multiplatform functions. Reflection, though, is available only on JVM

sandwwraith avatar Jul 05 '24 15:07 sandwwraith