serverless-image-handler icon indicating copy to clipboard operation
serverless-image-handler copied to clipboard

Give options to fallback to a alternative distribution with s3 bucket as the origin to get rid of the 6M limitaion

Open ensean opened this issue 3 years ago • 17 comments

Is your feature request related to a problem? Please describe.

Lambda's 6M payload limitation causing 413 error

Describe the feature you'd like

Give the user an option to fallback to an alternative distribution when image is too large, the alternative distribution use s3 as origin

Additional context

Maybe we can use LE bound to origin response, and redirect to the original image when api gw returns 413 error?

ensean avatar Jan 08 '22 14:01 ensean

This is a duplicate of #35 which started in 2018. Wonder why the original is closed, while this duplicate is open?

IMHO a 6 MB image size limit is not just a detail needing "enhancement", but major architectural flaw and actual bug.

klausbadelt avatar Feb 10 '22 22:02 klausbadelt

Hello. :smile:

So, how about we provide a function to retry by requesting a parameter?

heowc avatar Mar 16 '22 09:03 heowc

I agree with @klausbadelt. Competitive technologies like cloudinary are easy to integrate, and do not have this limitation. As an awful work around with SIH, the S3 webserver could be enabled on the image hosting bucket, and the application can pull raw from the bucket when it gets this error from the SIH cloudfront distro. One could argue that no UI img tags should ever be serving images which blow-out this limit.... but that should not preclude backend services from requesting whatever they need.

deanshelton913 avatar Mar 23 '22 20:03 deanshelton913

Hey team — adding a +1 to this thread. Overall, this solution has been fantastic and really easy to integrate, but not being able to send large images back to the client is limiting the potential use cases. Right now my customers are frequently uploading 3-8mb images (e.g. a screenshot of a 30" monitor), and not being able to return that or fall back to the original bucket source image is a blocker.

Any advice or workarounds for falling back to the original bucket origin would be awesome to have here.

brianlovin avatar Jun 12 '22 16:06 brianlovin

@brianlovin My solution would be to run a light compression on the browser side on a file that is 6 MB or larger before sending it to your S3 bucket. Because that would ensure it still gets piped through SIH and accessed via Cloudfront delivery easily. It is a workaround for now until Amazon fixes the 6MB Lambda limit.

aligajani avatar Jun 14 '22 23:06 aligajani

Makes sense, thanks for the idea. In my case, it's important I preserve the original asset for the user to download again in the future at its original size/resolution, so this might not work for my case at the moment.

brianlovin avatar Jun 20 '22 19:06 brianlovin

Still facing the same issue with Image greater than 5MB, can we get a workaround for this one? or do we need to restrict users to only upload images below certain file size?

mzahidriaz avatar Jul 01 '22 12:07 mzahidriaz

We are researching methods to resolve the 6mb issue and will update this once we have a resolution.

dougtoppin avatar Oct 31 '22 15:10 dougtoppin

Hi @fisenkodv Is there an update for this features. Really looking forward,

karthickeyang avatar Dec 18 '22 09:12 karthickeyang

Just remember even if you get by the 6MB Lambda payload limit, there is still the 10MB gateway payload limit.

CodeZeno avatar Dec 18 '22 10:12 CodeZeno

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

github-actions[bot] avatar Mar 19 '23 00:03 github-actions[bot]

Can we please fix this issue? we are really looking forward to have a workaround, if server cannot process the image, instead of throwing Internal Server Error can't we send the original image?

This issue is almost 1 and half year old now.

It's causing blockers for us

mzahidriaz-tr avatar Apr 20 '23 08:04 mzahidriaz-tr

Even with the image resizing, some images are too big as response for the lambda payload limit.

Would changing the lambda payload invocation from request/response to streaming be an option?

In particular, I was thinking of updating the handler function to stream responses by wrapping it with the streamifyResponse() decorator.

geraldinefiser avatar Jul 14 '23 04:07 geraldinefiser

Hi all, any update on this one?

hoantran-it avatar Mar 06 '24 13:03 hoantran-it

Hi all, any update on this one?

crowforkotlin avatar Mar 16 '24 14:03 crowforkotlin

Hi @hoantran-it, @crowforkotlin and @ensean,

While we don't currently have plans to implement this feature, we are working on providing a deployment parameter which will cause SIH to use an alternative infrastructure backed by S3 Object Lambda rather than API Gateway. This would remove the 6MB response size limit, restricting us solely to what can be processed by the Lambda within 30 seconds. We hope to release this feature in the next major release.

Thank you for your interest in SIH, Simon

simonkrol avatar Mar 18 '24 19:03 simonkrol

Thank you for the update! We are looking for that! 🙏 @simonkrol

For other people, who's encountering this issue. This is a workaround I suggest:

  • Do not config fallback image for SIH
  • Utilize onerror of the img tag (onError in case you use React)
  • When onerror is triggered, fetch the image again to classify the error. If it is 413 (image too large for this issue), call an API to create signedURL from S3 to show the image. If it is 404, show the fallback image

Hope it helps!

hoantran-it avatar Mar 19 '24 00:03 hoantran-it