Size of Releases folder getting too big

[chldbwnstm]

Squirrel version(s) Which version(s) of the project are you using? 4.9.1 Description When I started with version 1.0.0 50MB as my application size was 50MB. Then I upgraded to version 1.0.1 then did --releasify 1.0.1, so Releases folder became around 100MB because it keeps 1.0.0 and 1.0.1 and the delta file between 1.0.1. Until here ok..

Then, I releasify 1.0.2 then releases folder now is 150MB, having 1.0.0(50MB), 1.0.1(50MB), 1.0.2(50MB) inside the folder.. , Then I do a small code update and do 1.0.3 where it now comes 200MB... then 1.0.4 where it becomes 250MB because it has 1.0.0,1.0.1,1.0.2,1.0.3,1.0.4. Is this intended? because I am now afraid to do a small update to my program as I know even if its only a small update, the releases folder will get so much bigger which decreases my CI/CD potentials.

With the above example, if I ONLY keep 1.0.0 and 1.0.4 in the s3 update repository, will it be enough? will people with 1.0.0 and 1.0.2 both update to 1.0.4 successfully? Or is it intented to keep ALL previous full versions of the files like above.

So my real questions is, in the example above, say that I have below files in releases folder.

-MyApp-1.0.0-full.nupkg 50MB (initial package) -MyApp-1.0.0-delta.nupkg 60KB -MyApp-1.0.1-full.nupkg 50MB -MyApp-1.0.1-delta.nupkg 60KB -MyApp-1.0.2-full.nupkg 50MB -MyApp-1.0.2-delta.nupkg 60KB -MyApp-1.0.3-full.nupkg 50MB -MyApp-1.0.3-delta.nupkg 60KB

In the Update folder in the network (e.g. S3) is it sufficient to have just below to customers? : -MyApp-1.0.0-full.nupkg 50MB (initial package) -MyApp-1.0.0-delta.nupkg 60KB -MyApp-1.0.1-delta.nupkg 60KB -MyApp-1.0.2-delta.nupkg 60KB -MyApp-1.0.3-full.nupkg 50MB (latest full) -MyApp-1.0.3-delta.nupkg 60KB

[bzuillsmith]

I'm no expert of this library. If I'm reading the code properly, it appears the heuristic that determines whether to download a full release, or the Delta's, is rather simple. It simply downloads whatever is smaller. [See here] (https://github.com/Squirrel/Squirrel.Windows/blob/develop/src/Squirrel/UpdateInfo.cs). Either all the Delta's up to the latest version OR the latest full release will be downloaded. What this MIGHT mean (test because I haven't tried it yet) is you can remove full releases, just not the latest of course. Alternatively, the check updates function will let you ignore Delta's (it's a parameter) and you can always just make it download the latest full release in which case you theoretically only need one package on the server.

[kylclrk]

Can someone please confirm with confidence what @bzuillsmith is saying? I would love to be able to remove old full nuget packages.

[anaisbetts]

@kylclrk Yep he got it right :) In general, you're free to remove any release older than 2+ releases or so - even if a user is really far behind and you removed some of the delta releases they need, the updater will just fall back to the latest full release.

[kylclrk]

That's great news, thanks for the same-day response! And thanks for this valuable library :)

[bzuillsmith]

Yep. In fact, my colleagues and I found it better in our particular scenario to only support the few last deltas because even though they were small size-wise, it takes much longer to apply say 10 deltas of a few MB than for the user to download the latest full release and have it install right over everything. Something about the processing of the deltas takes a long time for us when there are lots of little changes.

This is something to consider, but you must decide if that is the best for you product or not. It won't be the same for every case.

[kylclrk]

Interesting, we'll have to keep that in mind. Thanks for the info

[anaisbetts]

it takes much longer to apply say 10 deltas of a few MB than for the user to download the latest full release and have it install right over everything

While this is true, you should be running this in the background so that users don't particularly notice how long it takes - run your app updates when your app is running but idle

[bzuillsmith]

True, if it meets your requirements, and I'm sure it does for most people. However the server deployment tool doesn't download the latest update and so a requirement of our software is to force an update on startup, otherwise the server users will always open the program to whenever it was deployed on their server. We also make breaking API changes occasionally because our dev team is small and it's sometimes expensive to make upgrades without such a change.

[caesay]

Applying deltas does take a very long time, and unfortunately the way this library determines whether to apply deltas vs. the full version is purely based on size. If your full package is 200mb and your deltas are 5mb, this library will choose to apply up to 40!! deltas instead of downloading the full version. If you are building releases via CI, this can easily happen.

Now why does applying delta's take so long? it's because of a few things. The general approach in this library is: take a full.nupkg, unzip it, apply a single delta, then zip it again into nextfull.nupkg. Do this again for X deltas. Zipping 200mb takes a very long time, especially when you need to do it 40 times, even with good hardware. And meanwhile, it's taking up lots of disk and CPU cycles.. not a very good user experience even if it is in the background! The second reason it's slow is because actually applying the deltas with msdelta is slow.

I have tried to improve/fix all of those things in my maintained fork of the library here.

In my fork, we're firstly a little more cautious about whether to download deltas or a full release, based on a heuristic we try to decide what will be faster, and we will never ever download more than 10. Next, we do not unzip/zip for every delta. We just do this once - unzip the full.nupkg, apply each delta, then re-zip once. This has the potential to 40x the speed of your delta update. Lastly, we use a faster diff algorithm.