feat: add support differential update on macOS

[beyondkmp]

Fix #7547

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: cb40ebe7a537a3b04a002316f8a2bc69cdbcb344

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
electron-updater	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[netlify[bot]]

✅ Deploy Preview for car-park-attendant-cleat-11576 ready!

Name	Link
🔨 Latest commit	cb40ebe7a537a3b04a002316f8a2bc69cdbcb344
🔍 Latest deploy log	https://app.netlify.com/sites/car-park-attendant-cleat-11576/deploys/65dcb878e1237f000724bc77
😎 Deploy Preview	https://deploy-preview-7709--car-park-attendant-cleat-11576.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mmaietta]

Thanks for contributing!

Please see build error:

Module '"builder-util-runtime"' has no exported member 'CURRENT_MAC_APP_ZIP_FILE_NAME'.

[beyondkmp]

Thanks for contributing!

Please see build error:

Module '"builder-util-runtime"' has no exported member 'CURRENT_MAC_APP_ZIP_FILE_NAME'.

Fixed

[beyondkmp]

@mmaietta I will fix these unit tests, but it might take some time as I've been quite busy lately.

[mmaietta]

Sounds great @beyondkmp! Really appreciate you picking this up!

[github-actions[bot]]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[petem24]

Any update on this?

[mmaietta]

I think this is the final remaining comment/step to get this ready for merge.

https://github.com/electron-userland/electron-builder/pull/7709#discussion_r1309102399

[mmaietta]

Hmmm, are you able to run the unit tests? Seems it already had the issue (at least on my machine) even without your changes applied: TEST_FILES=differentialUpdateTest pnpm -w run test

> @electron-builder/monorepo@ test /Users/michael.maietta/Development/electron-builder
> node ./test/out/helpers/runTests.js skipArtifactPublisher

Test files: differentialUpdateTest
custom opt: skipArtifactPublisher

FAIL src/updater/differentialUpdateTest.ts
  ● Test suite failed to run

    Jest encountered an unexpected token

SyntaxError: /Users/michael.maietta/Development/electron-builder/test/src/updater/differentialUpdateTest.ts: Cannot transform the imported binding "TmpDir" since it's also used in a type annotation. Please strip type annotations using @babel/preset-typescript or @babel/preset-flow.
      32 |   let outDirs: Array<string> = []
      33 |
    > 34 |   async function buildApp(version: string, tmpDir: TmpDir) {
         |                                                    ^^^^^^

I need to get that running on the CI before we can merge otherwise there's no visibility into the test you wrote

[beyondkmp]

I also cannot run the unit tests in my computer.

[mmaietta]

I'll continue investigating the unit test issue and get back to you as soon as possible so we can get your work merged in 🙂

[mmaietta]

Alright, I got the differential test suite fixed. It was an issue with circular references/imports and then I had to fix the windows+linux tests themselves w/ a small refactor. Just updated your PR with latest master

[mmaietta]

Tests are passing but there's an error when I tested it out using a localhost server

[error] Cannot download differentially, fallback to full download: TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at validateString (internal/validators.js:124:11)
    at Object.join (path.js:1039:7)
    at MacUpdater.differentialDownloadInstaller (/electron-builder-test/dist/electron-quick-start-typescript.app/Contents/Resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:586:31)
    at Object.task (/electron-builder-test/dist/electron-quick-start-typescript.app/Contents/Resources/app.asar/node_modules/electron-updater/out/MacUpdater.js:83:32)
    at MacUpdater.executeDownload (/electron-builder-test/dist/electron-quick-start-typescript.app/Contents/Resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:548:31)

[beyondkmp]

Tests are passing but there's an error when I tested it out using a localhost server

[error] Cannot download differentially, fallback to full download: TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at validateString (internal/validators.js:124:11)
    at Object.join (path.js:1039:7)
    at MacUpdater.differentialDownloadInstaller (/electron-builder-test/dist/electron-quick-start-typescript.app/Contents/Resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:586:31)
    at Object.task (/electron-builder-test/dist/electron-quick-start-typescript.app/Contents/Resources/app.asar/node_modules/electron-updater/out/MacUpdater.js:83:32)
    at MacUpdater.executeDownload (/electron-builder-test/dist/electron-quick-start-typescript.app/Contents/Resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:548:31)

I have moved CURRENT_MAC_APP_ZIP_FILE_NAME into MacUpdater.ts to fix it. You can try it in latest commit.

[mmaietta]

Testing this further, I can't get the differential download to work correctly, I keep receiving a checksum mismatch for some reason.

Cannot download differentially, fallback to full download: Error: sha512 checksum mismatch, expected TuNB4sWHMz/8a8H+g32IlXpet9aFS1SlDAhbkWy9QuSkPlPW8+1TgsNdh8jLo+DVEfUtjBvnoJbw8dr6HMEHcA==, got vhSVmETkWgobSMa3biddjNeo20QePxDgG2FTRAqvWUkHyrU2T+pGa4KgyQoOpxF31qSpOuD2ils900HYhFFVCg==

Are you able to get the differential download working? You can use a minio server locally to set up a localhost "s3" config, on MacOS, it's as easy as using brew install minio + the setup with the instructions below https://www.electron.build/tutorials/test-update-on-s3-locally

[beyondkmp]

@mmaietta I can. Here's logs

2024-02-26T14:15:25.599+0800 DEBUG: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: /4cFBaLj9grwTU0jv6OTGeJe duplicated in blockmap (same size), it doesn't lead to broken differential downloader, just corresponding block will be skipped)
2024-02-26T14:15:25.599+0800 DEBUG: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: o0MSgzsL7wr02KddiDQOx64N duplicated in blockmap (same size), it doesn't lead to broken differential downloader, just corresponding block will be skipped)
2024-02-26T14:15:25.600+0800 DEBUG: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: 15e/AHUeiZeB/8kP4WUstznD duplicated in blockmap (same size), it doesn't lead to broken differential downloader, just corresponding block will be skipped)
2024-02-26T14:15:25.600+0800 DEBUG: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: gRHNaDm9LqR4/gc0CPRVe+Iv duplicated in blockmap (same size), it doesn't lead to broken differential downloader, just corresponding block will be skipped)
2024-02-26T14:15:25.601+0800 DEBUG: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: /lOV1D7Lk3AdOYM457wkehlD duplicated in blockmap (same size), it doesn't lead to broken differential downloader, just corresponding block will be skipped)
2024-02-26T14:15:25.608+0800 INFO: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: File has 2371 changed blocks
2024-02-26T14:15:25.640+0800 DEBUG: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: {"truncation":"[\n  {\n    \"kind\": 1,\n    \"start\": 0,\n    \"end\": 32768\n  },\n  {\n    \"kind\": 0,\n    \"start\": 32768,\n    \"end\": 60026699\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60026699,\n    \"end\": 60054775\n  },\n  {\n    \"kind\": 0,\n    \"start\": 60054775,\n    \"end\": 60066471\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60066471,\n    \"end\": 60153556\n  },\n  {\n    \"kind\": 0,\n    \"start\": 60153556,\n    \"end\": 60164856\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60164856,\n    \"end\": 60250146\n  },\n  {\n    \"kind\": 0,\n    \"start\": 60250146,\n    \"end\": 60265982\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60265982,\n    \"end\": 60332341\n  },\n  {\n    \"kind\": 0,\n    \"start\": 60332341,\n    \"end\": 60341828\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60341828,\n    \"end\": 60429566\n  },\n  {\n    \"kind\": 0,\n    \"start\": 60429566,\n    \"end\": 60439715\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60439715,\n    \"end\": 60577813\n  },\n  {\n    \"kind\": 0,\n    \"start\": 60577813,\n    \"end\": 60591229\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60591229,\n    \"end\": 60715152\n  },\n  {\n    \"kind\": 0,\n    \"start\": 60715152,\n    \"end\": 60732527\n  },\n  {\n    \"kind\": 1,\n    \"start\": 60732527,\n    \"end\": 61051932\n  },\n  {\n    \"kind\": 0,\n    \"start\": 61051932,\n    \"end\": 61063648\n  },\n  {\n    \"kind\": 1,\n    \"start\": 61063648,\n    \"end\": 61072282\n  },\n  {\n    \"kind\": 0,\n    \"start\": 61072282,\n    \"end\": 61082528\n  },\n  {\n    \"kind\": 1,\n    \"start\": 61082528,\n    \"end\": 61092268\n  },\n  {\n    \"kind\": 0,\n    \"start\": 61092268,\n    \"end\": 61101250\n  },\n  {\n    \"kind\": 1,\n    \"start\": 61101250,\n    \"end\": 61165773\n  },\n  {\n    \"kind\": 0,\n    \"start\": 61165777,\n    \"end\": 61602555\n  },\n  {\n    \"kind\": 1,\n    \"start\": 61602551,\n    \"end\": 61646474\n  },\n  {\n    \"kind\": 0,\n    \"start\": 61646483,\n    \"end\": 61691685\n  },\n  {\n    \"kind\": 1,\n    \"start\": 61691676,\n    \"end\": 61734449\n  },\n  {\n    \"kind\": 0,\n    \"start\": 61734448,\n    \"end\": 65104586\n  },\n  {\n    \"kind\": 1,\n    \"start\": 65104587,\n    \"end\": 65132246\n  },\n  {\n    \"kind\": 0,\n    \"start\": 65132245,\n    \"end\": 65173775\n  },\n  {\n    \"kind\": 1,\n    \"start\": 65173776,\n    \"end\": 65291050\n  },\n  {\n    \"kind\": 0,\n    \"start\": 65291048,\n    \"end\": 66289993\n  },\n  {\n    \"kind\": 1,\n    \"start\": 66289995,\n    \"end\": 66355079\n  },\n  {\n    \"kind\": 0,\n    \"start\": 66355073,\n    \"end\": 67861542\n  },\n  {\n    \"kind\": 1,\n    \"start\": 67861548,\n    \"end\": 67911805\n  },\n  {\n    \"kind\": 0,\n    \"start\": 67911793,\n    \"end\": 67983818\n  },\n  {\n    \"kind\": 1,\n    \"start\": 67983830,\n    \"end\": 68016598\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68016586,\n    \"end\": 68134804\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68134816,\n    \"end\": 68167349\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68167337,\n    \"end\": 68272040\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68272052,\n    \"end\": 68304820\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68304808,\n    \"end\": 68398862\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68398874,\n    \"end\": 68409099\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68409087,\n    \"end\": 68535215\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68535227,\n    \"end\": 68550445\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68550433,\n    \"end\": 68643985\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68643997,\n    \"end\": 68674273\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68674261,\n    \"end\": 68711753\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68711765,\n    \"end\": 68744533\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68744521,\n    \"end\": 68810057\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68810069,\n    \"end\": 68842837\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68842825,\n    \"end\": 68940766\n  },\n  {\n    \"kind\": 1,\n    \"start\": 68940778,\n    \"end\": 68956370\n  },\n  {\n    \"kind\": 0,\n    \"start\": 68956358,\n    \"end\": 69058342\n  },\n  {\n    \"kind\": 1,\n    \"start\": 69058354,\n    \"end\": 69073589\n  },\n  {\n    \"kind\": 0,\n    \"start\": 69073577,\n    \"end\": 69175852\n  },\n  {\n    \"kind\": 1,\n    \"start\": 69175864,\n    \"end\": 69198717\n  },\n  {\n    \"kind\": 0,\n    \"start\": 69198705,\n    \"end\": 69307015\n  },\n  {\n    \"kind\": 1,\n    \"start\": 69307027,\n    \"end\": 69323452\n  },\n  {\n    \"kind\": 0,\n    \"start\": 69323440,\n    \"end\": 69405"}
2024-02-26T14:15:27.276+0800 INFO: ELECTRON,[UPDATE],[UpdateController]],[ElectronUpdater]: Full: 330,805.67 KB, To download: 50,556.52 KB (15%)

The error you encountered is due to updater.zip not being the current version. The reason for this is as follows: the initial download for the update was successful, and it was copied to the cache directory. However, if you did not exit the installation of the new version, and the server subsequently updated to another version, attempting to download the incremental update would result in this error because the original updater.zip no longer matches.

[mmaietta]

Nice work! I've added an additional check for when the update file hasn't existed yet due to app's first install/update. It also includes the override downloadUpdateOptions.disableDifferentialDownload property in case the developer has explicitly disabled it

const canDifferentialDownload = () => {
  if (!pathExistsSync(cachedFile)) {
    log.info("Unable to locate previous update.zip for differential download (is this first install?), falling back to full download")
    return false
  }
  return !downloadUpdateOptions.disableDifferentialDownload
}

[mmaietta]

Tests are passing.

@beyondkmp I think all necessary steps for implementation are complete and ready for merge. Can you please confirm when you have a chance?

[beyondkmp]

Thanks for fixing the UT, I have no issues now. You can help merge it.

[mmaietta]

Thanks for your contribution!

[ThatKawaiiSam]

@mmaietta @beyondkmp I was testing this within my application this morning - seems as if the first time it can't differentially download it never actually falls back to download the full bundle which is a little concerning...

Happy to help diagnose over discord if needed (thatkawaiisam)

[mmaietta]

That's seriously concerning. Will revert this PR and debug after

[mmaietta]

Took another go and it and this should fix it: https://github.com/electron-userland/electron-builder/pull/8093