retrofit2-kotlinx-serialization-converter icon indicating copy to clipboard operation
retrofit2-kotlinx-serialization-converter copied to clipboard

Decode from stream instead of string

Open VincentJoshuaET opened this issue 3 years ago • 17 comments

class FromString(override val format: StringFormat) : Serializer() {
    override fun <T> fromResponseBody(loader: DeserializationStrategy<T>, body: ResponseBody): T {
      val string = body.string()
      return format.decodeFromString(loader, string)
    }

    override fun <T> toRequestBody(contentType: MediaType, saver: SerializationStrategy<T>, value: T): RequestBody {
      val string = format.encodeToString(saver, value)
      return RequestBody.create(contentType, string)
    }
  }

to

class FromStream(override val format: StringFormat) : Serializer() {
    override fun <T> fromResponseBody(loader: DeserializationStrategy<T>, body: ResponseBody): T {
      val stream = body.byteStream()
      return format.decodeFromStream(loader, stream)
    }

    override fun <T> toRequestBody(contentType: MediaType, saver: SerializationStrategy<T>, value: T): RequestBody {
      val stream = ByteArrayOutputStream()
      format.encodeToStream(saver, value, stream)
      return RequestBody.create(contentType, stream.toByteArray())
    }
  }

VincentJoshuaET avatar Sep 30 '21 02:09 VincentJoshuaET

I don't see much reason to change the encoding since both end up buffering. We can switch the decoding, however. Want to send a PR?

JakeWharton avatar Sep 30 '21 02:09 JakeWharton

This is blocked on:

  • [ ] Streaming being available for all format types, not just a single one
  • [ ] The streaming API being stable

JakeWharton avatar Oct 01 '21 13:10 JakeWharton

This is blocked on:

* [ ]  Streaming being available for all format types, not just a single one

* [ ]  The streaming API being stable

Is there any update on this or where can we track this?

FunkyMuse avatar Dec 11 '21 14:12 FunkyMuse

On the kotlinx.serialization repo. I'm not sure if there are bugs for either, but you could create them.

JakeWharton avatar Dec 11 '21 21:12 JakeWharton

I believe streaming support has landed in kotlinx.serialization 1.3.0: https://github.com/Kotlin/kotlinx.serialization/issues/204.

mthmulders avatar May 20 '22 06:05 mthmulders

Yes, but only for JSON. This issue was filed after 1.3.0 was released.

JakeWharton avatar May 24 '22 02:05 JakeWharton

@JakeWharton I see that 1.4.0 now supports Okio BufferedSource and BufferedSink. Is this converter gonna be updated with that or is Retrofit maybe planning on releasing an official converter? Thanks.

gajicm93 avatar Aug 03 '22 10:08 gajicm93

No, nothing has really changed. Streaming still only works for JSON not all text formats. And the APIs this library uses are still marked as unstable which means we cannot upstream it yet.

JakeWharton avatar Aug 03 '22 12:08 JakeWharton

@JakeWharton Will the usage of Okio BufferedSource and BufferedSink APIs reduce memory footprint during serialization/deserialization?

ahulyk avatar Aug 18 '22 08:08 ahulyk

If the response if absolutely huge it would, yes.

JakeWharton avatar Aug 18 '22 13:08 JakeWharton

@JakeWharton cool, thanks

ahulyk avatar Aug 18 '22 14:08 ahulyk

@JakeWharton we have stable kotlinx.serialization v1.4.0 could we expect the support of Okio in the near future?

ahulyk avatar Aug 26 '22 09:08 ahulyk

No. It's only supported for Json and not arbitrary StringFormats.

JakeWharton avatar Aug 26 '22 13:08 JakeWharton

I suppose we could add a Json-specific overload of the factory function for now to enable it.

JakeWharton avatar Aug 27 '22 23:08 JakeWharton

Although Json is in a separate library so it would require a separate artifact...

JakeWharton avatar Aug 28 '22 00:08 JakeWharton

@JakeWharton given that you are considering separate artifact for JSON - do you think it is realistic to have an "official" retrofit-converter for JSON with kotlinx-serialization? As far as I see, JSON support seams to be stable in kotlinx-serialization now.

AlexTrotsenko avatar Sep 15 '22 10:09 AlexTrotsenko

No, the APIs we use are still unstable for any format.

JakeWharton avatar Sep 15 '22 12:09 JakeWharton