Receiving: {"result":{"permit":false},"id":885,"status":"ok"} and non-resolving Promise.all while fetching from emulator storage

[braadworst]

[REQUIRED] Environment info

firebase-tools: 9.22.0

Platform: macOs

[REQUIRED] Test case

Doesn't apply

[REQUIRED] Steps to reproduce

Right so this is a bit of a guesswork thing, but I am loading a bunch off files from the storage emulator via a Promise.all call. These are geojsons that I need to process further. But if the amount of files I load exceeds a certain threshold (I have no clue what the threshold is) it starts to return:

{"result":{"permit":false},"id":885,"status":"ok"}
{"result":{"permit":false},"id":886,"status":"ok"}
{"result":{"permit":false},"id":887,"status":"ok"}

And never resolves the Promise.all. If I play around with the files, meaning load less this goes away and everything is working as expected. The problem is not the files themselves, loading them individually is all fine, just a certain amount becomes troublesome. Is there some kind of data size limit?

[REQUIRED] Expected behavior

I expect, that no matter how many files/file size I throw at Promise.all, it will resolve and not give me any issues

[REQUIRED] Actual behavior

Never resolves, and as a consequence times my cloud function out

[braadworst]

By the way, changing the Promise.all to a for loop, and loading the files one by one give the same issue. It seems the storage emulator is limiting in some way the requests maybe?

[skam22]

having the same issue here, the promise doesn't resolve, my console outputs:

{"result":{"permit":true},"id":1692,"status":"ok"} {"result":{"permit":true},"id":1693,"status":"ok"} {"result":{"permit":true},"id":1694,"status":"ok"} {"result":{"permit":true},"id":1695,"status":"ok"}

those four console only outputs when a promise doesn't resolve. @braadworst are you seeing the same issue with firebase itself?

[LB-Digital]

Can confirm also having this issue when using full emulator suite, not tested on prod. Occurred for me with 7 getDownloadUrl promises in a Promise.allSettled() call. Sometimes all will resolve, sometimes 2 or 3 will fail.

@braadworst try using Promise.allSettled(), so you can confirm that only 3 are sometimes failing.

[weixifan]

I have filed an internal issue b/211308355 to track.

[braadworst]

I can confirm all the things you mention @skam22, @LB-Digital thanks for the info, I can see already how many resolve, I just need the missing ones to do so as well :-) @weixifan any progress on this?

[jelizarovas]

Is this just emulator issue or persists live?

[skam22]

Emulator only for me, working fine on production.

[jelizarovas]

I'm getting the same issue on emulator, When I'm uploading 3+ files also with promises, it never returns and gets frozen on the front end. Even though the emulator functions execute fully and create thumbnails for the files, I can never complete the upload callback.

upload task gets stuck after first or second upload. this is my upload function if it helps at all

    dispatch({ type: "IN_PROGRESS" });
    if (files.filter((f) => !f.uploaded).length === 0) {
      return dispatch({ type: "ERROR", payload: "Nothing selected. Please attach files" });
    }

    function uploadFileAsPromise(file) {
      return new Promise(async function (resolve, reject) {
        const fileId = file.id || cuid();
        console.log({
          file,
          fileId,
          storagePath,
          documentPath,
          documentKey,
        });
        const uploadTask = uploadFileToFirebaseStorage({
          file,
          fileId,
          storagePath,
          documentPath,
          documentKey,
        });
        dispatch({ type: "CANCEL_FN_ARRAY", payload: { id: fileId, cancelUpload: () => uploadTask.cancel() } });

        uploadTask.on(
          "state_changed",
          (snapshot) => {
            const progress = (snapshot.bytesTransferred / snapshot.totalBytes) * 100;
            dispatch({ type: "CURRENT_FILE_UPLOAD_PROGRESS", payload: { progress, fileId } });
          },
          (error) => {
            dispatch({ type: "ERROR", payload: error.message });
            reject(error);
          },
          async () => {
            console.log(`${file.name} uploaded.`);
            const downloadURL = await uploadTask.snapshot.ref.getDownloadURL();
            console.log(`${file.name} download url is ${downloadURL}.`);
            dispatch({
              type: "UPDATE_TOTAL_PROGRESS",
              payload: { downloadURL, id: file.id },
            });

            return resolve({
              id: file.id,
              fileName: file.name,
              size: file.size,
              type: file.type,
              url: downloadURL,
              storageRef: storagePath,
              thumbs: [],
            });
          }
        );
      });
    }

    Promise.all(files.filter((f) => !f.uploaded).map(uploadFileAsPromise))
      .then(async (uploadedFiles) => {
        console.log("Adding file data to firestore");
        await addFileDataToFirestore({ documentPath, documentKey, uploadedFiles });
        // console.log(uploadedFiles);
        dispatch({ type: "COMPLETE" });
      })
      .catch((error) => dispatch({ type: "ERROR", payload: error }));
  }

[skam22]

seeing the error on production now - the items are taking a long time to download from firestore storage..

[abhis3]

Hi all, the fix for this issue has been merged and should be in the next release of the emulator.

If you're still running in to issues please feel free to reopen this issue

[braadworst]

Works like a charm! Thanks for picking this up, much appreciated

[royniels]

Sorry to pull up an old issue, but this is back in 13.0.2, uploading multiple files in parellel will generate the issue

[nt-tuguldur]

Same here, firebase-tools 13.5.2 have same issue