blitz icon indicating copy to clipboard operation
blitz copied to clipboard

Form Multipart Management/Resolvers should accept Files

Open ryardley opened this issue 3 years ago • 11 comments

What do you want and why?

In application development to upload a file is a really common thing to do. (Say for a user avatar upload uploading a video etc). Often especially in a serverless context, we then need to stream it on to a third party asset storage solution such as S3 or cloudinary. Some services provide unsigned options but this can lead to security issues. In any case there is a fair bit management required to successfully accept a file in the browser send the file to the server or serverless function parse it to a stream then send it elsewhere. Currently to accomplish this in Blitz we need to create a custom api route and because fetch has no progress event use XmlHTTPRequest or a lib that is based on it to manage the upload process.

On the clientside it might be normal to use something like React Dropzone to get the file ready in the browser or you might use a native file input and assemble it for form submission using the FormData API. In anycase you would manipulate file blobs using the browsers File API.

On the server side You need to use a library such as formidable or busboy to handle parsing the file to a stream and then piping that stream to S3 or Cloudinary or to disk. Really what we need to be able to do is write a mutation that can accept file objects in the browser and receive them as streams on the server.

Possible implementation(s)

The browser File API is a Blob which has a stream method so we might want to have a similar API on the server and provide any files passed as a mutation's argument properties to be re assembled as File-like objects on the Node end.

import uploadFile from 'app/projects/mutations/uploadFile'

const {getRootProps, getInputProps} = useDropzone({onDrop: (acceptedFiles) => {
  // an instant upload in this case
  uploadFile({file:acceptedFiles[0]}, {
    onUploadProgress(event) {
      const uploadedBytes = event.loaded / event.total;
      console.log(`Uploaded ${uploadedBytes} bytes`);
    }
  }) 
}})
export default async function uploadFile({ file }: UploadFileInput) {
  const [uploadStream, donePromise] = getCloudinaryUploadStreamWithDonePromise(file.name)
  file.stream().pipe(uploadStream)
  await donePromise
  return 'ok'
}

To do this we should probably use busboy on the server as that does not do any intermediate file storage. See article

ryardley avatar Aug 08 '20 01:08 ryardley

Related https://github.com/blitz-js/blitz/issues/841

ryardley avatar Aug 08 '20 01:08 ryardley

Sounds like a good idea!

flybayer avatar Aug 08 '20 15:08 flybayer

Using React Dropzone on the client-side is a great idea.

On the server side it would be useful to have the option to pipe the file to S3/Cloudinary and/or to disk (for non-serverless context) as cloud photo storage services can get expensive pretty quickly.

cr101 avatar Aug 09 '20 01:08 cr101

Hey @flybayer. I want to lobby Blitz in our small team and this feature will be handy in my arguments. Any thoughts when it can be done?

mirus-ua avatar Aug 13 '20 17:08 mirus-ua

@Miruzz I definitely won't get to it soon, but maybe @ryardley will?

flybayer avatar Aug 14 '20 21:08 flybayer

@flybayer this is a huge dealbreaker :(

kitze avatar Sep 29 '20 11:09 kitze

@kitze yeah we'll get it added. The workaround is to use a regular API route, same as you would have to do without Blitz.

flybayer avatar Sep 29 '20 13:09 flybayer

Yeah, I ended up using:

  • react-dropzone for dropping the image
  • cloudinary.utils.sign_request for getting a signed url (via blitz query)
  • upload the images on the client-side

kitze avatar Oct 01 '20 04:10 kitze

That's definitely the approach I recommend (upload straight to cloudinary, s3, etc).

flybayer avatar Oct 01 '20 18:10 flybayer

@flybayer do you have an example of uploading to s3 by any chance?

ranihorev avatar Dec 28 '21 22:12 ranihorev

@ranihorev In case this is still relevant for you, or anyone else that may come across this. We ended up using the source of https://github.com/ryanto/next-s3-upload but changing it to make it work with a Blitz query, and adding some other stuff we needed.

dbrxnds avatar Jun 01 '22 22:06 dbrxnds

@dbrxnds can you please share the example of how to make it work with Blitz query?

muco-rolle avatar Dec 10 '22 00:12 muco-rolle