Colossus: validate upload request params before upload is finished

[kdembler]

Background

For various reasons, a Colossus upload request may fail. It may be client failure (wrong params), operator misconfiguration or probably tons of other reasons we haven't seen yet. That will happen from time to time. However, a lot of errors that can occur on the operator side, don't depend on the file being uploaded itself, but on request params (storageBucketId, bagId, etc.), or external factors (sync status of operator QN, etc.). For this reason, the upload UX could be much improved, if the validation that happens after upload is finished, could happen immediately on request sent. This way, if the upload is doomed to fail anyway, user doesn't have to wait for lengthy upload and can try another operator right away.

Proposal

Our goal should be to allow upload request validation before the file upload is finished. There are 2 ways I see we could go about this:

1. Change the /api/v1/files endpoint - at the moment validation logic that happens here will start only after the file upload. Ideally, we modify the request handling logic to run before the full file is streamed, do validation and short-circuit if it fails and then optionally wait for the upload to finish before marking it as accepted.
2. Add additional endpoint that does the same validation as /api/v1/files but doesn't accept any file. This way, request params validation could be done and instruct whether it makes sense to start an actual upload. This could even accept additional params like (dataObjectSize and dataObjectIpfsHash) to validate whether that data (as seen by client) matches what the operator would expect.

I think option 1. works better overall as we don't require additional request roundtrip, but I'm not sure how easy that is on the technical side, to postpone awaiting file streaming to finish. I'm sure somebody solved this already though. My suggestion is to go for 1. if it doesn't include significant effort increase, otherwise just go with 2.

┆Issue is synchronized with this Asana task by Unito

[bedeho]

Adding to mainnet scope!

[bedeho]

Done.