workers-sdk icon indicating copy to clipboard operation
workers-sdk copied to clipboard

🐛 BUG: R2 Multipart upload doesn't return custom metadata

Open YannicEl opened this issue 1 year ago • 3 comments

Which Cloudflare product(s) does this pertain to?

R2

What version(s) of the tool(s) are you using?

3.57.2 Wrangler

What version of Node are you using?

20.12.2

What operating system and version are you using?

Windows 11

Describe the Bug

Observed behavior

Custom metadata is missing when completing a R2 multipart upload. However when fetching the head of the r2Object the metadata is there. Strangely enough when running locally with wrangler pages dev the example works as expected. Only once you deploy your site the custom metadata is missing.

const upload = await env.BUCKET.createMultipartUpload(key, {
  customMetadata: {
    hallo: 'zwallo',
  },
});

const uploadPart = await upload.uploadPart(1, request.body);
    
const r2Object = await upload.complete([uploadPart]);
console.log(r2Object.customMetadata) // logs an empty object: {}

const r2Head = await env.BUCKET.head(key);
console.log(r2Head.customMetadata) // logs the correct metadata: {hallo: "zwallo"}

Expected behavior

multipartUpload.complete() should return an R2Object with the field customMetadata populated

Steps to reproduce

  1. clone repository
  2. run cd multipart-upload-missing-metadata && npm i
  3. run npm run dev for a working example
  4. run npm run deploy for the example to fail on the deployed site

Please provide a link to a minimal reproduction

https://github.com/YannicEl/cloudflare_worker_bugs/tree/main/multipart-upload-missing-metadata

Please provide any relevant error logs

No response

YannicEl avatar May 29 '24 17:05 YannicEl

I've pinged the R2 team, who should be able to take a look at this (cc @jonesphillip)

penalosa avatar Aug 19 '24 13:08 penalosa

By design, R2 does not return custom metadata on complete multipart. The "bug" is that Miniflare assumes that we do, which you can see here: https://github.com/cloudflare/workers-sdk/blob/main/packages/miniflare/src/workers/r2/bucket.worker.ts#L442

This would explain why it works locally, but not when you deploy. Locally, it's using this Miniflare instance which seems to not match R2 behavior 1:1. Our team will need to see if not returning custom metadata is by design or not internally. Might be something we overlooked, but will try and get you an answer.

Schachte avatar Aug 19 '24 17:08 Schachte

We discussed this and ultimately just need to prioritize plumbing this through. Will throw it on the roadmap to cleanup so we have parity with Miniflare.

Schachte avatar Aug 25 '24 18:08 Schachte