parabol icon indicating copy to clipboard operation
parabol copied to clipboard

Support multi-part uploading for CDNs

Open mattkrick opened this issue 6 months ago • 0 comments

Predecessor to #9262.

When we do a heap dump, it consumes so much memory that we get an OOM error & the pod restarts. The right way to do a dump is to upload the chunks to S3 or GCS. That way, we don't have to go into the pod & fish it out.

To do this, we need to support multi-part uploads. For example, here's how to do it for S3 https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/s3/command/CreateMultipartUploadCommand/

mattkrick avatar Dec 12 '23 18:12 mattkrick