kotlinx.serialization icon indicating copy to clipboard operation
kotlinx.serialization copied to clipboard

Add more flexibility to BinaryFormat.encodeToByteArray

Open jfontsaballs opened this issue 4 months ago • 3 comments

What is your use-case and why do you need this feature?

I want to append additional data to the byte array generated by encodeToByteArray without having to copy all data into a larger array unnecessarily. I need to send this ByteArray over websockets but not all data that needs to be sent can be serialized.

Describe the solution you'd like

Option 1

Add additional parameters to encodeToByteArray to request empty space at the beginning and end of the array the this function instantiates.

Option 2

Receive a lambda that received the required space and returns a ByteArray on which to write serialized data and the starting index.

jfontsaballs avatar Aug 17 '25 21:08 jfontsaballs

Related to #3064

jfontsaballs avatar Aug 17 '25 21:08 jfontsaballs

Normally, I would say you should use slices for that, but the .slice function copies an array.

sandwwraith avatar Aug 18 '25 13:08 sandwwraith

The options given don't really work as encodeToByteArray doesn't actually know the needed array size ahead of time (due to the architecture of the library). It creates a dynamically sized byte buffer (e.g. ByteArrayOutputStream / ByteChannel) and writes to that before returning a byte buffer afterwards.

As such the ideal solution would be a to expose the serialization to a byte appendable type (Appendable for bytes or ByteArrayOutputStream etc.). There is no standard multiplatform type for this, and you really want to use a standard type (and a encodeToByteAppendable etc.).

While kotlinx.io provides such primitives this is not a dependency of the core serialization library, so a standard function to provide access is tricky.

pdvrieze avatar Aug 23 '25 18:08 pdvrieze