kotlinx.serialization icon indicating copy to clipboard operation
kotlinx.serialization copied to clipboard

Support definite-length Arrays and Maps in CBOR

Open wverdese opened this issue 3 years ago • 4 comments

What is your use-case and why do you need this feature? I want to use this library to communicate with an API that uses CBOR for payloads. This API is strictly requesting Arrays and Maps to be encoded with a definite length, because of / lack of dynamic memory allocation. Encoding a Kotlin Array or a Kotlin Map will always result in their indefinite-length counterpart, so the API will fail when decoding my payload.

Describe the solution you'd like This line:

Cbor.encodeToByteArray(arrayOf(mapOf(\* stuff *\)))

now serializes to:

9F                                  # array(*)
   BF                               # map(*)
      /* stuff */
      FF                            # primitive(*)
   FF                               # primitive(*)

I'd like to be able to, e.g., pass a parameter:

Cbor.encodeToByteArray(arrayOf(mapOf(\* stuff *\)), Lengths.DEFINITE)

to be able to change the serialization to:

81                          # array(1)
   A1                       # map(1)
      /* stuff */

wverdese avatar May 30 '22 15:05 wverdese

Well, any CBOR decoder should support indefinite lengths, otherwise it does not satisfy the standard. But probably some tuning may help here

sandwwraith avatar May 30 '22 20:05 sandwwraith

I agree, but there's also the efficiency aspect to consider. For instance, I'm using CBOR to send/receive messages over BLE. Using indefinite lengths in a context where the length is very well known will result in two extra bytes to exchange (the list+map delimiters) that are not necessary.

wverdese avatar May 31 '22 11:05 wverdese

This also informs decoder allocation efficiency: a consumer (in languages where it matters or can well take advantage of it) can allocate the exact size data structure needed when given a definite length, and without requiring any buffering to count the overall length before decoding and storing the data in memory.

extemporalgenome avatar Aug 01 '23 21:08 extemporalgenome

see #2412

JesusMcCloud avatar Aug 18 '23 14:08 JesusMcCloud