kotlinx.serialization icon indicating copy to clipboard operation
kotlinx.serialization copied to clipboard

Expose whether decoder / encoder supports binary formats

Open 3v1n0 opened this issue 4 years ago • 4 comments

What is your use-case and why do you need this feature?

I'm writing a library where I only want to depend on serialization-core and so not have to check for decoder is JsonDecoder or similar, but I'd like to have different strategies depending whether the decoder natively supports binary format or not (for example, to encode an image in base64 or just use a byteArrray).

Describe the solution you'd like

Ideally the decoder and encoder would expose an abstract function with the flags they support about the specifics of the format we're going to build, so that the encoding format can change, without having to manually check the type of the encoder used.

3v1n0 avatar Jan 12 '21 12:01 3v1n0

Binary encoding is handled with ByteArraySerializer and the format should automatically "optimize" it if applicable. So this feature works by you specifying what you need, and the format doing something with that. Looking at the code, it is supported by the cbor and protobuf formats, but not by json (or xml). In case it is not supported it will serialize as an array of values even where base64 would be a better choice. This would be a worthwhile feature to implement (behind a feature flag as it would result in a format incompatibility).

I can see the point of having a custom serializer that does bytearray to base64 conversion, but I can't see how you would do so except using some sort of hardcoded whitelist (note that you can compare the class name without needing to have that).

pdvrieze avatar Jan 12 '21 14:01 pdvrieze

note that you can compare the class name without needing to have that

Yeah, something I thought about doing, but not really a fan of using strings comparisons in such case (also would be: val useJson = encoder.javaClass.interfaces.find { it.name == "kotlinx.serialization.json.JsonEncoder" } != null)

For example, I'm using this custom serializer and structure for serializing an Android bitmap into a PNG:

@Serializable
data class BitmapSerialDescriptor(
    val width : Int,
    val height : Int,
    val hashCode : Int,
    val base64 : String?
    val byteArray : ByteArray?)

object BitmapBase64Serializer : KSerializer<Bitmap> {
    override val descriptor: SerialDescriptor = BitmapSerialDescriptor.serializer().descriptor
    override fun serialize(encoder: Encoder, value: Bitmap) {
        val stream = ByteArrayOutputStream()
        val usingJson = encoder is JsonEncoder
        value.compress(Bitmap.CompressFormat.PNG, 90, stream)
        stream.toByteArray().also { byteArray ->
            BitmapSerialDescriptor(value.width, value.height, value.hashCode(),
                if (usingJson) Base64.encodeToString(byteArray, Base64.DEFAULT) else null,
                if (!usingJson) byteArray else null).also {
                encoder.encodeSerializableValue(serializer(), it)
            }
        }
    }
    override fun deserialize(decoder: Decoder): Bitmap {
        decoder.decodeSerializableValue(BitmapSerialDescriptor.serializer()).also {
            val byteArray = if (decoder is JsonEncoder) Base64.decode(it.base64, 0) else it.byteArray!!
            return BitmapFactory.decodeByteArray(byteArray, 0, byteArray.size)
        }
    }
}

So picking the right data flag depending on the case (I may just use one generic one with some more manual labor, but that's not the point), but I'd like to be able to figure this out better, without having to go for decoder specializations.

Even though, it seems that I'm still quite forced to do if I want to use @ByteString to optimize the Cbor case even more...

3v1n0 avatar Jan 12 '21 14:01 3v1n0

I think the similar approach will be usefull for UUID serialization, where it should be string in Json and XML by ByteArray in protobuf or CBOR.

Is it possible now to make different types of serilization depending on target plugin?

EMaksymenko avatar May 08 '24 18:05 EMaksymenko

@ComBatVision The way to handle different formats is to check the encoder/decoder type using an instance check where the else case gives the default behaviour. Then you can do something specific for each encoder/decoder type, this could be calling a delegate serializer to do the work (just remember the type serialname uniqueness rule).

pdvrieze avatar May 08 '24 19:05 pdvrieze