elm-concurrent-task icon indicating copy to clipboard operation
elm-concurrent-task copied to clipboard

Support decoding the whole bytes response without knowing its size

Open mpizenberg opened this issue 8 months ago • 9 comments

Currently in version 1.1.1, the expectBytes function has the following signature:

expectBytes : Decoder a -> Expect a

Since the Elm bytes library has no decoder to get the whole bytes as Bytes without providing a byte length, it is impossible to just retrieve the whole bytes response.

In the regular elm/http library, the expectBytes function has the same issue, but we can circumvent that issue by using instead the expectBytesResponse function.

expectBytesResponse :
    (Result x a -> msg)
    -> (Response Bytes -> Result x a)
    -> Expect msg

It seems to me that elm-concurrent-task doesn’t have such alternative, which is a blocker for me. What’s your view on adding something that enables us to get the whole bytes response instead and deal with it on our own.

mpizenberg avatar Mar 18 '25 19:03 mpizenberg

Hi @mpizenberg - I'm afraid I don't really have the capacity to tackle this at the moment, but if you'd like to experiment it's possible to write your own http module + adapter (have a look at the docs here https://package.elm-lang.org/packages/andrewMacmurray/elm-concurrent-task/latest/ConcurrentTask-Http)

andrewMacmurray avatar Mar 26 '25 09:03 andrewMacmurray

@mpizenberg did you ever get a chance to look at this? Re-looking at it now would something like this help?

expectMetadata : (Metadata -> Expect a) -> Expect a

So you could do something like:

expectBytesWithLength : Expect Bytes
expectBytesWithLength =
    expectMetadata
        (\meta ->
            meta.headers
                |> Dict.get "content-length"
                |> Maybe.andThen String.toInt
                |> Maybe.withDefault 0
                |> Bytes.Decode.bytes
                |> expectBytes
        )

andrewMacmurray avatar Jul 10 '25 16:07 andrewMacmurray

I was somewhat lucky that I didn’t really have a need for ConcurrentTask for my requests needing to read the full bytes response, so I just used the regular http requests for these, and used concurrent tasks for my other requests.

I’m not super familiar with http headers, but what you are showing makes sense.

mpizenberg avatar Jul 10 '25 17:07 mpizenberg

In your case how would you determine the size of the bytes response in regular elm/http?

expectBytesResponse :
    (Result x a -> msg)
    -> (Response Bytes -> Result x a)
    -> Expect msg

The Metadata type is the same as the type in elm/http https://package.elm-lang.org/packages/elm/http/latest/Http#Metadata

andrewMacmurray avatar Jul 15 '25 09:07 andrewMacmurray

In your case how would you determine the size of the bytes response in regular elm/http?

Well I don’t need to determine the size of the bytes response because I need the whole bytes. But if I needed, I’d call the Bytes.width : Bytes -> Int function: https://elm.dmy.fr/packages/elm/bytes/latest/Bytes?q=wid#width

mpizenberg avatar Jul 15 '25 11:07 mpizenberg

I see, so you're just taking the raw value from Response Bytes. In that case using Metadata wouldn't really work.

andrewMacmurray avatar Jul 15 '25 12:07 andrewMacmurray

Yes indeed I need the raw bytes for something I cannot deal with directly in Elm (cannot decode it, and don’t know the size in advance)

mpizenberg avatar Jul 15 '25 12:07 mpizenberg

What about this? https://github.com/andrewMacmurray/elm-concurrent-task/commit/8b1e13c0ec35a67482423570694ed5302c3453d0

andrewMacmurray avatar Jul 15 '25 18:07 andrewMacmurray

It looks perfect, at least for my use cases

mpizenberg avatar Jul 19 '25 10:07 mpizenberg