tusd icon indicating copy to clipboard operation
tusd copied to clipboard

Re-uploading the same file will not trigger a hook event.

Open zxl777 opened this issue 4 years ago • 11 comments

I uploaded the file for the first time and received the json data of the hook event correctly. After I refreshed the page and uploaded the same file again, no hook event occurred.

zxl777 avatar Feb 02 '20 08:02 zxl777

I found a solution, which is to delete the file after processing it.

zxl777 avatar Feb 04 '20 06:02 zxl777

Hello,

We have the same problem. Given that deleting the file may not succeed (e.g. backend failure), it is necessary to re-emit this event when the file has been fully downloaded.

tuscland avatar Feb 06 '20 11:02 tuscland

I would also like to add that it is not possible to delete the file before uploading it because it would defeat the resumable nature of’ the upload...

tuscland avatar Feb 06 '20 13:02 tuscland

This is not a problem with the tus server but your client, which likely has a setting enabled that it will not reupload a file that it has already uploaded. For example, if you are using tus-js-client look at the resume option. Removing the file on the server works, because the client can then not find the upload anymore and will reupload it.

Acconut avatar Feb 06 '20 16:02 Acconut

Hi Marius,

Here are the settings we use:

{
    endpoint: this.endpoint,
    retryDelays: [0, 3000, 5000, 10000, 20000],
    metadata: {
        ...
    },
    onProgress: (bytesUploaded, bytesTotal) => {
        ...
    },
    onSuccess: () => {
        ...
    },
    onError: (error) => {
        ...
    }
}

The use case is the following:

  • A user uploads completely the file.
  • Tusd notifies the backend with post-finish hook
  • The backend fails for some reason
  • ...
  • Later, the same user tries to upload again
  • The upload is resumed
  • No event is sent, because the upload is still present, and the backend has no way to know.
  • The user can never complete the upload.

Thank you for your attention. Camille

tuscland avatar Feb 06 '20 17:02 tuscland

@tuscland Please look into the removeFingerprintOnSuccess: true option for tus-js-client. It should allow you to reupload the file again.

Acconut avatar Feb 06 '20 17:02 Acconut

Sounds great.. I will, thank you!

tuscland avatar Feb 06 '20 17:02 tuscland

While removeFingerprintOnSuccess does work, it would be great to have the ability to re-trigger the completion hooks in one way or another. When uploading 100gb files, it can be a touch bit demoralizing if an API failure forces a full restart of the upload.

Pontohi avatar Nov 21 '22 17:11 Pontohi

API failure forces a full restart of the upload.

How do you mean that? API failure in the sense that the API which consumes the uploaded file fails and needs to be retried?

Acconut avatar Nov 22 '22 14:11 Acconut

API failure forces a full restart of the upload.

How do you mean that? API failure in the sense that the API which consumes the uploaded file fails and needs to be retried?

Yep, that would be the case. TUS will have the full file with all bytes uploaded, but if the API fails to consume the hook we have no choice but to have the user restart since we can't get the hook to refire. Of course the API should be more reliable for consuming the hooks, but for files that take hours to days to upload, seems like it could stand to be more fail-tolerant

Pontohi avatar Nov 22 '22 15:11 Pontohi

TUS will have the full file with all bytes uploaded, but if the API fails to consume the hook we have no choice but to have the user restart since we can't get the hook to refire.

tusd will retry hooks if they fail. At least for HTTP and gRPC hooks, I think. So if you bubble up the error from inside your hooks, then it should be rare that an uploads fails due to this.

Acconut avatar Feb 11 '23 20:02 Acconut