httpwg
RESUMABLE: proposal to change the design to support parallel uploads
@gstrauss provided the following feedback on https://lists.w3.org/Archives/Public/ietf-http-wg/2025JulSep/0094.html wrt the Introduction section
"Unlike ranged downloads, this protocol does not support transferring an upload as multiple requests in parallel." That is an application limitation due to over-specified implementation details and flaws in the resumable upload protocol design. The client should manage the state of the resource through successful upload of chunks. If a chunk fails for any reason, the chunk should be resent, perhaps in multiple smaller chunks.
This is a proposal to change the technical design established during the development of the document.
The ability to upload in parallel is a nice feature, but is not the primary point I was trying to make here
The primary point I am trying to make about this is that if the client manages the temporary resource, much of the current resumable upload complexity can be removed. Upload-Offset would not be needed to be sent in responses by server. The client would upload chunks of data. If uploading a chunk of data was unsuccessful, the client would re-upload the whole chunk, perhaps in a set of multiple smaller chunks. In contrast, the current extra round trips required for resumable upload recovery can exhibit worst-case behavior more frequently than not, and the recommendations for the server to write as much partial data as possible before being killed by a different request (?!) is suboptimal and messy. That is not in the spirit of independent HTTP requests.
Were the client managing the temporary resource, and the client would always know the state of the temporary resource. There would be no need for the very, very questionable recommendation in resumable uploads that an idempotent request for temporary resource state should kill a different, already in progress non-idempotent PATCH which may have hung. Instead, the client would ignore the hung request and re-upload the data to the same offset in the temporary resource. Even if not hung, the previous request would write the same data to that offset, and so there would not be any file corruption in the temporary resource.
When you say "manage", what do you mean? Do you mean choosing the URL?
The design is as it is because it is the server's URL space to manage, so it needs to be the one to allocate resources and resource identifiers for the upload process.
When you say "manage", what do you mean? Do you mean choosing the URL?
I mean not relying on the server to tell the client Upload-Offset or other state of the temporary resource. The client should know which chunks it uploaded successfully and should know when the temporary resource has been completely uploaded.
A client can know this and not rely on the server's representation of this state, but only if requests succeed. The design needs to allow for cases where connections break and requests fail. How else can a client recover in that case?
The design needs to allow for cases where connections break and requests fail. How else can a client recover in that case?
If the client is uploading in reasonably-sized chunks, that should be sufficient. If there is more loss, the client can use smaller chunks to progress. (This is much more reasonable when the client can upload many chunks, filling the bandwidth, rather than resumable upload requiring serial upload)
To my prior post, a client updating a temporary resource -- and not restricted by resumable uploads as drafted -- could send an HTTP/1.1 pipeline or multiple HTTP/2 streams loading chunks serially or in parallel (does not matter). If a request or connection fails, reconnect and send the chunks for which client has not received a successful HTTP response. A client can abandon a connection if the client deems it hung or too much loss, and can resend chunks on a different network path. All this is possible when the client is responsible for uploading the temporary resource and the client is able to do so across multiple requests and without having to ask the server for state.
it is the server's URL space to manage, so it needs to be the one to allocate resources and resource identifiers for the upload process.
I fully agree that the server should control creation/allocation of the temporary resource and resource identifiers.
Most of the discussion is besides the point of the issue I created. Please stay on topic. The proposal is to change the design and that would allow parallel upload.
Authors need to hear from the WG to understand if this is a property that they want or not.