Differential downloader not working when Bitbucket is redirecting to Amazon S3 for downloading

[mleister97]

Node Version: v18.16.0 Target: Windows (will probably affect all target platforms) Notes on the repository: I am using a private repository with BitBucket.

Important modules in package.json

"electron": "^26.1.0",
"electron-builder": "^24.6.3",
"electron-devtools-installer": "^3.2.0",
"electronmon": "^2.0.2",

Description: When using a private repository in conjunction with Bitbucket for differential downloads, the process fails to complete successfully. Initially, it appears to work as expected, but subsequently, Bitbucket redirects the process to Amazon S3. At this point, I suspect that the authorization token sent to Bitbucket is being passed along to Amazon S3. As a result, Amazon S3 checks for the presence of this token and immediately returns a 400 error. It's important to note that the correct authorization information for Amazon is indeed included in the request.

Steps to Reproduce:

1. Set up a private repository on Bitbucket.
2. Initiate a differential download process from the repository.
3. Observe the redirection to Amazon S3 and the subsequent 400 error.

Console output

Found version 27.0.0 (url: myproject-27.0.0.exe)
Downloading update from myproject-27.0.0.exe
updater cache dir: C:\Users\micha\AppData\Local\myproject
disableWebInstaller is set to false, you should set it to true if you do not plan on using a web installer. This will default to true in a future version.
Download block maps (old: "https://api.bitbucket.org/2.0/repositories/xxx/myproject/downloads/myproject-5.0.0.exe.blockmap", new: https://api.bitbucket.org/2.0/repositories/xxx/myproject/downloads/myproject-27.0.0.exe.blockmap)
 x49Tc7GXT1FLdkzenA0GBjNP duplicated in blockmap (same size), it doesn't lead to broken differential downloader, just corresponding block will be skipped)
File has 112 changed blocks
[] // Here are all blocks listet, working as expected
Full: 72,088.2 KB, To download: 2,254.01 KB (3%)
download range: bytes=0-15449

====================================================
THIS IS WHERE THE PROBLEM BEGINS
====================================================
Redirect to https://bbuseruploads.s3.amazonaws.com/b40f0432-c97d-40a9-92e6-bcde2ad80606/downloads/f933d4bb-091b-4da3-8e4c-f317f9573b3f/myproject-27.0.0.exe
Cannot download differentially, fallback to full download: HttpError: 400 Bad Request

at createHttpError (C:\Users\micha\AppData\Local\Programs\myproject\resources\app.asar\dist\main\webpack:\node_modules\builder-util-runtime\out\httpExecutor.js:16:12)
at ClientRequest.<anonymous> (C:\Users\micha\AppData\Local\Programs\myproject\resources\app.asar\dist\main\webpack:\node_modules\electron-updater\out\differentialDownloader\DifferentialDownloader.js:205:59)
    at ClientRequest.emit (node:events:513:28)
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:2:49930)
    at SimpleURLLoaderWrapper.emit (node:events:513:28)

It seems to me like the error occurs somewhere in the following lines. I haven't been able to debug this successfully. Even if I comment out the line with the configuration of the redirect URL, the redirect works, which seems to indicate that I don't fully understand the context. I have also attempted to directly modify the request options using the configureRequestOptions method, but unfortunately, without success.

electron-updater/out/differentialDownloader/DifferentialDownloder.js

 request.on("redirect", (statusCode, method, redirectUrl) => {
  this.logger.info(`Redirect to ${removeQuery(redirectUrl)}`);
  actualUrl = redirectUrl;
  (0, builder_util_runtime_1.configureRequestUrl)(new url_1.URL(actualUrl), requestOptions);
  request.followRedirect();
});

These lines in the httpExecutor have indicated to me that my assumption is correct. However, they are not executed in the DifferentialDownloader.js. The fix could be relatively straightforward for someone who understands the context or the flow. The solution would be to clear the authorization header (delete headers.authorization) even for a differential download in case a redirect to the URL occurs. The provided code snippet actually describes quite well the missing part.

builder-util-runtime/out/httpExecutor.js

static prepareRedirectUrlOptions(redirectUrl, options) {
  const newOptions = configureRequestOptionsFromUrl(redirectUrl, { ...options });
  const headers = newOptions.headers;
  if (headers === null || headers === void 0 ? void 0 : headers.authorization) {
    const parsedNewUrl = new url_1.URL(redirectUrl);
    if (parsedNewUrl.hostname.endsWith(".amazonaws.com") || parsedNewUrl.searchParams.has("X-Amz-Credential")) {
        delete headers.authorization;
    }
  }
  return newOptions;
}

How can I solve this issue? Even during a differential download, the authorization header must be cleared in the event of a redirect to .amazonaws.com. I would greatly appreciate any assistance with this.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[mleister97]

Issue is not stale

[BaraoVlask]

Same problem here Any solution?

[mmaietta]

Looking into this, but I think this may be why you're not able to override the request options https://github.com/electron-userland/electron-builder/blob/4497e865ca653d1986ef2871b3db4e3e61f5167a/packages/electron-updater/src/electronHttpExecutor.ts#L83-L86

The downside is that there are write streams in progress, so I can't just provide a new fdList: [] to doDownloadFile(...) AFAICT?

I'm really out of my element here, not familiar with how this area of the code works (differential downloads) or how the caveats of Electron.ClientRequest play in

Just to play around, it could be worth trying to add that special delete options.authorization logic into the DifferentialDownloader, but I'm not sure if it's possible to start a new request. I'm not seeing any other functions accepting a request object and based on the code comment, it's not possible to use followRedirect in this case, it must be aborted and a new one created

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 30 days with no activity.