File uploads / content-type: multipart/form-data

[knubie]

At the moment it doesn't seem possible to send files as multipart/form-data as described here: Using the Fetch API - Web APIs | MDN.

For example with js/fetch we could do something like this:

(-> (js/fetch "https://example.com/profile/avatar"
              #js{:method "POST"
                  :body (doto (js/FormData.) (.append "file" blob file-name))})

and js/fetch will automatically set the Content-Type header to multipart/form-data and encode the body properly.

However in lambdaisland/fetch the body is always encoded according to the :content-type option before it's passed to js/fetch (unless it's a string):

https://github.com/lambdaisland/fetch/blob/1fe9fab75b47f5a245dfa105a87ffe8da539118d/src/lambdaisland/fetch.cljs#L102-L104

There isn't really a way to leave the body decoded without passing a string.

I think the simplest workaround would probably be to add a new :raw or nil option for :content-type that just passes the :body through to js/fetch decoded.

It's possible to bypass the encoding today by using :default as the :content-type, but that will also explicitly set the Content-Type header to nil, since it's not in the content-types map and is really just a hack for the encoding multimethod.

https://github.com/lambdaisland/fetch/blob/1fe9fab75b47f5a245dfa105a87ffe8da539118d/src/lambdaisland/fetch.cljs#L80-L81

[plexus]

Thanks @knubie for reporting this. I think there's a few sides to this. Currently we accept a string body as being already encoded, but js/fetch also accepts a FormData, and (I assume?) a Blob, so it makes sense to detect those and not further try to encode them. It doesn't really make sense to encode a FormData or Blob to Transit anyway. That seems like a no-brainer change to me, it's potentially a breaking change, but I don't see how anyone could be meaningfully relying on the current behavior for FormData or Blob as body (or am I missing something? are there cases where this would make sense? perhaps people have set up custom Transit writers/readers for these types, that could happen...)

I'm also not against introducing something like :raw as an escape hatch to opt-out of certain processing behavior. We can't anticipate every use case, and sometimes it's better to just get out of the way and let the user handle things.

I think it also makes sense for people to want to separately decide the encoding, and the content-type header. E.g. serializing as json, but have a more specific content-type like application/x-my-app-data. I guess you can do this by setting both :content-type and :headers {"Content-Type" ...}? not sure, would have to do some testing.

Ideally a high level library like this would relieve people from having to deal with FormData objects directly. POSTing a file/blob seems common enough that we can support it directly. I'm thinking fetch could introduce a :multipart-form-data content-type, allowing one to specify the content-type, content-disposition, and content of each part.

(fetch/post "/foo" {:content-type :multipart-form-data
                    :body [{:content-type "application/json" ;; string or keyword, so e.g. :json or :transit+json also works
                            :content-disposition "my_file.json"
                            :content blob-or-string}]})

Does that all make sense @knubie ? Patches for these things are welcome, as is further comments to make sure what we're planning to do makes sense and doesn't have adverse side effects for existing users.

Thanks!

[knubie]

@plexus

I think I agree with pretty much everything you've said here. I know that js/fetch accepts a Blob, but I'm not sure how that works or what the use cases are for passing a Blob there, so I don't know what the expectation would be for how this library should handle that.