emscripten
emscripten copied to clipboard
Fix progress reporting for compressed downloads in fetchRemotePackage
When a server uses compression (e.g., Brotli (br) or Gzip (gzip)), the Content-Length header in the HTTP response represents the compressed size of the data, not the uncompressed size. The fetchRemotePackage function, however, tracks the uncompressed size of the downloaded data (via value.length of each chunk after decompression). This leads to a discrepancy where the reported totalLoaded (uncompressed) exceeds the Content-Length (compressed), causing inaccurate progress reporting.
The Fetch API does not provide direct access to the compressed stream’s size for each chunk, as decompression is handled transparently. The updated code addresses this by aligning the total and loaded values with the compressed size from Content-Length when the content is compressed. Specifically:
- The contentEncoding is checked to determine if compression (gzip or br) is used.
- If the content is not compressed and Content-Length is available, total is set to the Content-Length value. Otherwise, it falls back to packageSize.
This ensures progress reporting reflects the compressed size, preventing totalLoaded from exceeding total.
Can you update the PR description and title now that we have changed the PR content?
Do you know why we cannot just use packageSize in all cases? Are there some cases where packageSize is not known?
Do you know why we cannot just use
packageSizein all cases? Are there some cases wherepackageSizeis not known?
just my guess, packageSize is calculated while doing packaging, one can still change the file with different size on the server without doing packaging. in that case, content-length will be correct value,
It looks like someone else just ran into this in #24529. Any chance you could update this PR and we can land it?