kotlinx.serialization icon indicating copy to clipboard operation
kotlinx.serialization copied to clipboard

Request: a tuple descriptor, so JSON messages can be smaller (currently requires InternalSerializationApi)

Open aSemy opened this issue 3 years ago • 6 comments

What is your use-case and why do you need this feature?

I need to use JSON to encode and decode 2d arrays of x/y coordinates

  @Serializable
  data class Coordinates(
    val x: Int,
    val y: Int,
  )

By default, the JSON encoding is not size-optimal

{ "x": 1, "y": 2 }

Encoding a coordinate as a tuple (a fixed-length array, with fixed types per position, see the TypeScript docs) would make the messages more compact, and save a lot of space and improve processing speeds.

[1, 2]

There are other data structures I'd like to encode as well. For example, some information for each coordinate.

  @Serializable
  data class CoordinatesDetails(
    val x: Int,
    val y: Int,
    private val colour: String,
    private val active: Boolean,
  ) {
    ...
  }

Again, this would save a lot of space if I could en/decode it as a tuple

[1, 2, "red", true]
{ "x": 1, "y": 2, "colour": "red", "active": true }

Describe the solution you'd like

I've created a 'tuple descriptor builder' https://github.com/adamko-dev/kotlinx-serialization-typescript-generator/blob/main/docs/tuples.md

// tuple descriptor builder example
  @Serializable(with = Coordinates.Serializer::class)
  data class Coordinates(
    val x: Int,
    val y: Int,
  ) {
    object Serializer : TupleSerializer<Coordinates>(
      "Coordinates",
      {
        element(Coordinates::x)
        element(Coordinates::y)
      }
    ) {
      override fun tupleConstructor(elements: Iterator<*>): Coordinates {
        val x = requireNotNull(elements.next() as? Int)
        val y = requireNotNull(elements.next() as? Int)
        return Coordinates(x, y)
      }
    }
  }

This is quite finickity though, and also it's bugged. Under the hood it requires buildSerialDescriptor, because buildClassSerialDescriptor doesn't allow setting the structure kind to be LIST.


My custom tuple-serializer would be much easier if I could

  1. retain the plugin-generated serializer https://github.com/Kotlin/kotlinx.serialization/issues/1169
  2. override the serial kind of the plugin-generated descriptor and change it from CLASS to LIST.

Alternatively, it would be even easier if I could tell the compiler plugin to generate a descriptor as usual, except just change the 'kind' of the descriptor.

  @Serializable(kind = StructureKind.LIST)
  data class Coordinates(
    ...

Finally, another option is to add an arg to buildClassSerialDescriptor

public fun buildClassSerialDescriptor(
    serialName: String,
    serialKind: StructureKind = StructureKind.CLASS, // new arg, with a default 
    vararg typeParameters: SerialDescriptor,
    builderAction: ClassSerialDescriptorBuilder.() -> Unit = {}
):
...

aSemy avatar Apr 18 '22 20:04 aSemy

Yeah, this is indeed long-standing problem. I think we can pull buildSerialDescriptor out of internal API, since there is a demand for it.

sandwwraith avatar Apr 19 '22 11:04 sandwwraith

buildSerialDescriptor can work for my situation, but it's quite inconvenient and not very friendly to use. It requires that (at best) each 'tuple' class defines a custom serializer with manual casting. I imagine this will quite quickly get out of sync.

      override fun tupleConstructor(elements: Iterator<*>): Coordinates {
        val x = requireNotNull(elements.next() as? Int)
        val y = requireNotNull(elements.next() as? Int)
        return Coordinates(x, y)
      }

aSemy avatar Apr 19 '22 16:04 aSemy

While answering this StackOverflow question, I realized it would indeed be useful to have something like a JsonTupleSerializer based on the JsonTransformingSerializer.

Here is an example of how it could look like:

/**
 * Serializes objects of type [T] into tuples of their properties, in the order defined by their serializer.
 */
open class JsonTupleSerializer<T : Any>(
    private val serializer: KSerializer<T>,
) : JsonTransformingSerializer<T>(serializer) {

    private val propertyNames = serializer.descriptor.elementNames.toList()

    override fun transformDeserialize(element: JsonElement): JsonElement =
        if (element is JsonArray) tupleToObject(element) else element

    private fun tupleToObject(element: JsonArray): JsonObject {
        require(element.size == propertyNames.size) {
            "Unexpected tuple length ${element.size} for ${serializer.descriptor.serialName}, " +
                "expected ${propertyNames.size} values for properties $propertyNames"
        }
        return buildJsonObject {
            propertyNames.forEachIndexed { index, propName ->
                put(propName, element[index])
            }
        }
    }

    override fun transformSerialize(element: JsonElement): JsonElement =
        if (element is JsonObject) objectToTuple(element) else element

    private fun objectToTuple(element: JsonObject) = buildJsonArray {
        propertyNames.forEach { propName ->
            // some properties might not be serialized in object form (e.g. default values)
            // but tuples are positional, so we need explicit nulls
            add(element[propName] ?: JsonNull)
        }
    }
}

joffrey-bion avatar Aug 27 '22 16:08 joffrey-bion