bootc-image-builder icon indicating copy to clipboard operation
bootc-image-builder copied to clipboard

AWS AMI upload to S3 failed : TotalPartsExceeded

Open ivanbaldo opened this issue 1 year ago • 0 comments

Hello. Running this:

podman run \
  --rm \
  -ti \
  -v /var/lib/containers/storage:/var/lib/containers/storage \
   \
   \
  --privileged \
  --net host \
  --pull newer \
  -v ./build:/output \
  -v ./build/store:/store \
  -v /dev/null:/root/.aws/credentials:ro \
   \
  quay.io/centos-bootc/bootc-image-builder \
   \
   \
  --type ami \
  --chown 0:0 \
  --aws-ami-name Netlabs-RHELAI-2024-06-24.x86_64 \
  --aws-bucket rhelai.netlabs.com.uy \
  --aws-region us-east-1 \
  --local \
  quay.io/ai-lab/nvidia-bootc:latest

Failed with:

Uploading image/disk.raw to rhelai.netlabs.com.uy:7b242daf-87f1-4335-a73b-fab5878c9247-disk.raw
48.83 GiB / 85.39 GiB [------------------------------------------------------------->______________________________________________] 57.19% 136.88 MiB p/s
Error: cannot upload AMI: MultipartUpload: upload multipart failed
	upload id: 1CREpYyflDLAT4In7LX0KKXAksg2w_0hVhZSeEmQdBT40JG67Z0ArNukPdQOtPQ5F7fsf.W.VpSOEQsN.PB0_t7xatmAtTPABhsr2usrPPP.mxq_6k30cWz.KXfu.D7tr59jdLjPnwmTXqzMlKoveg--
caused by: TotalPartsExceeded: exceeded total allowed configured MaxUploadParts (10000). Adjust PartSize to fit in this limit
2024/06/24 22:26:31 error: cannot upload AMI: MultipartUpload: upload multipart failed
	upload id: 1CREpYyflDLAT4In7LX0KKXAksg2w_0hVhZSeEmQdBT40JG67Z0ArNukPdQOtPQ5F7fsf.W.VpSOEQsN.PB0_t7xatmAtTPABhsr2usrPPP.mxq_6k30cWz.KXfu.D7tr59jdLjPnwmTXqzMlKoveg--
caused by: TotalPartsExceeded: exceeded total allowed configured MaxUploadParts (10000). Adjust PartSize to fit in this limit

Since the image is already saved locally we should be able to get it's size and use an appropriate PartSize. Thanks!!!

ivanbaldo avatar Jun 24 '24 22:06 ivanbaldo