Download the OTA update as individual files

[colenso]

Would help if the OTA updates could be downloaded as individual files rather than one big .zip file. Especially helpful with app having a bigger OTA update size and users downloading the OTA on Mobile Data.

[anbraten]

As the client apps are sending there current version to the server there could also be some system generating diff zips. So in case the user had for example the previous version and now updates to the latest version he could just get a to file large diff containing the two updated files. If another user is just installing the app he would get the full zip. The diffed zips could be cached based on from and to version identifiers.

Or something like a diff -ruaBN previous new > test1.diff

[riderx]

@anbraten since user can be in any current previous version, that would mean produce a diff zip for all previous version, plus Capgo will need to understand what is a diff in your build who seems very complex since project can be react/svelte/angular/vue/js/etc a believe for now a easier upgrade would be using better the background job feature from iOS and android. to have more than 30 sec in background to finish the work.

[anbraten]

I guess creating the diff would be somehow possible without knowing what the details are, similar to how git, diff work, but sure the server side "calculation" whats the difference between two version would require some computation power even if the result of an upgrade / downgrade between two versions would be cached.

Doing that in a background job seems to be a nice idea in general.

One benefit of sending something like a diff, could be that you save a lot of network bandwidth which would be nice for the client, but also for your server if you really have a lot of clients.

Just wanted to do some brain dump in general as I read this issue 😃. I really like your lib and system.

[riderx]

I totally agree if i found a easy way to do diff i would love to have that, sending duplicated file is not nice

[csotiMatyas]

A comment on this, i was also thinking about sending just some files or folders in the bundle folder. My problem is having an asssets folder being around 1mb, a total waste to send on every download. So i was looking around in the code trying to figure out how the bundle swap happens, so maybe instead of a bundle "replace" i could just make a "copy and replace all" so i could just send what i want, and just override the needed folders/ files. But i can't really find the place where it happens on android.

[riderx]

the files are saved in a new folder and then the capgo define the capacitor web view new default location to it, so for your idea to work, it needs to copy previous app in new folder and then override certain files, the problem is: It's difficult to be sure to have one solution to fit all, since some people in web use unique file name to flush cache

[riderx]

/bounty $500

[algora-pbc[bot]]

💎 $500 bounty • Capgo

~~## 💎 $100 bounty • Capgo~~

Steps to solve:

1. Start working: Comment /attempt #119 with your implementation plan
2. Submit work: Create a pull request including /claim #119 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Additional notes:

ℹ️ If something is not clear ask before working on it, otherwise your chance to rework it is high 🎥 To claim you need to provide in your PR a demo video of the change 👨‍👩‍👧‍👦 Join the Discord to get help 📏 Check all Bounty rules

Thank you for contributing to Cap-go/capacitor-updater!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @ayewo	Aug 10, 2023, 4:31:21 PM	#119
🟢 @WcaleNieWolny	Mar 10, 2024, 6:21:53 PM	WIP

[0x0elliot]

hey, i would like to help out with this issue. would you mind giving me some context and assigning me to it? :)

[riderx]

Hey @0x0elliot i did assign you ! Currently, the updates are sent from the CLI as a single zip and downloaded by Android and iOS devices as single file. This is causing issue for big update (more than 10 MB). So, we have a high level of fail update from 10% to 50% for app used in location where internet is less stable.

What needs to be done:

• Update the CLI to send multiple zip/files instead of one to store.
• Update the backend to understand multi files by update.
• Update the plugin code in android to download multi files updates.
• Update the plugin code in IOS to download multi files updates.

To note:

• There is a current end-to-end encryption system who makes this job a bit harder

The job is pretty big, and I believe it has to be split to be done. Are you interested in doing any of this task?

[0x0elliot]

i am interested. if you would be comfortable, we can split the bounty into two tasks and 4 PRs:

Part 1

• Update the backend to understand multi files by update.
• Update the CLI to send multiple zip/files instead of one to store.

Part 2

• Update the plugin code in android to download multi files updates.
• Update the plugin code in IOS to download multi files updates.

In chronological order.

I am of course very new to this. Would snoop around in the discord to figure it out properly.

Immediate bottlenecks

To begin with, some questions, which project exactly is the backend?

to help save some time, would you mind helping me figure out which files should i look into? i am thinking of creating another API endpoint (or the equivalent in this case) for this feature as that's the cleaner way to proceed further.

[riderx]

yes, of course, I can adjust the bounty accordingly. you are right, the backend is in the repo https://github.com/Cap-go/capgo in the Supabase folder. I have to create a reproducible env i believe for this task

[0x0elliot]

okay. let me know of the .env file then. why would you require it?

[riderx]

if you want to be able to try end to end, otherwise you can run in local

[0x0elliot]

been trying to set it up locally. it has surely been a bottleneck to set it up. would be less of one if you can manage to create one with some instructions so that i can just focus on the issue.

[0x0elliot]

having quite a difficult time with migrations here :') @riderx this should be a one time effort help though!

[Tallyb]

Some notes: Diff should be made on the md5 of the file, as this is a reliable way to know it has changes. Lost of modern tools use hash when building web output, e.g. webpack, for caching reasons, so file name is likely to be different.

[riderx]

Hey @0x0elliot sorry for the delay it's now way more easy to have local env, i made the seeb db and migration file. https://github.com/Cap-go/capgo#start-local-supabase-db

[ayewo]

Hey @riderx

Still open? I'm thinking I might want to take a stab at this ...

[riderx]

It's still open yes !

[ayewo]

@riderx Great!

I would like to tenatively adopt the same approach proposed above:

i am interested. if you would be comfortable, we can split the bounty into two tasks and 4 PRs:

Part 1

• Update the backend to understand multi files by update.
• Update the CLI to send multiple zip/files instead of one to store.

Part 2

• Update the plugin code in android to download multi files updates.
• Update the plugin code in IOS to download multi files updates.

Before I start work on Part 1, I'd like to assess if the scope is commensurate with the bounty on offer. Once I'm done with my initial assessment, I'll ping you here or in Discord so you can officially assign me to the issue.

[ayewo]

... or negotiate the terms before being assigned.

[riderx]

ok no problem mate, just to let you know as the same time someone else contacted me in private and is also doing a proposal on how to do the work

[ayewo]

@riderx Haha, no problem. Thanks for the heads up. (Sounds like that person was subscribed to be notified of any activity on this issue 😀).

Quick question

When I tried to stand up the DB using supabase start as per your README instructions, I get the following error related to migrations:

Applying migration 20230321023005_create_base_db.sql...
ERROR: schema "supabase_migrations" already exists (SQLSTATE 42P06)
At statement 242: --
-- Name: supabase_migrations; Type: SCHEMA; Schema: -; Owner: postgres
--

CREATE SCHEMA supabase_migrations

Seesm like the latest Supabase docker image now ships with the supabase_migrations schema already in place which is why there's a conflict?

[ayewo]

@riderx Nevermind, I figured it out 😎

You can assign me to the issue 🙂.

[AitorG]

@ayewo could be nice, as a first version, to update only those files included in zip and do not delete others in order to save memory and transfer data (it could be setted by config value), just saying :)

[ayewo]

@AitorG yea it would nice, but I haven't been assigned to this issue, yet 😀.

Someone else proposed something similar above to which @riderx replied with the following comment:

the files are saved in a new folder and then the capgo define the capacitor web view new default location to it, so for your idea to work, it needs to copy previous app in new folder and then override certain files, the problem is: It's difficult to be sure to have one solution to fit all, since some people in web use unique file name to flush cache

[ayewo]

@riderx

I've taken a good look and I now understand what needs to be done here. Essentially, will need to update 3 repos:

• the cli to support multiple zip files;
• the capgo backend and the supabase DB;
• the capacitor-updater native code for each of iOS and Android.

Then there is the encryption which also needs to be handled for this task to be complete.

For comparison, this issue #119 requires significantly more effort than #72 where only the capacitor-updater repo has to be updated but you priced both issues the same 🤷.

[riderx]

Yes you are right, I didn’t price it well. I will update it 

Martin

Maker of Capgo https://capgo.app/ Capacitor Live updates Podcast host at SOLOS https://solos.ventures/ Maker of Captime https://captime.app Crossfit timer app Mobile app Builder at Forgr https://forgr.ee/ 

On August 9, 2023, Martin DONADIEU @.***> wrote:

@riderx https://github.com/riderx

I've taken a good look and I now understand what needs to be done here.  Essentially, will need to update 3 repos:

• the cli https://github.com/Cap-go/CLI to support multiple zip files;
• the capgo https://github.com/Cap-go/capgo backend and the supabase DB;
• the capacitor-updater <https://github.com/Cap-go/capacitor- updater> native code for each of iOS and Android.

Then there is the encryption which also needs to be handled for this task to be complete.

For comparison, this issue #119 <https://github.com/Cap-go/capacitor- updater/issues/119> requires significantly more effort than #72 https://github.com/Cap-go/capacitor-updater/issues/72 where only the capacitor-updater repo has to be updated but you priced both issues the same 🤷.

— Reply to this email directly, view it on GitHub <https://github.com/Cap- go/capacitor-updater/issues/119#issuecomment-1671209104>, or unsubscribe <https://github.com/notifications/unsubscribe- auth/AA7FGL2UB5T5U5PM5D6G5OLXUN437ANCNFSM6AAAAAASAQXRBY>. You are receiving this because you were mentioned.Message ID: <Cap- @.***>

[tobihans]

@riderx Haha, no problem. Thanks for the heads up. (Sounds like that person was subscribed to be notified of any activity on this issue 😀).

Quick question

When I tried to stand up the DB using supabase start as per your README instructions, I get the following error related to migrations:

Applying migration 20230321023005_create_base_db.sql...
ERROR: schema "supabase_migrations" already exists (SQLSTATE 42P06)
At statement 242: --
-- Name: supabase_migrations; Type: SCHEMA; Schema: -; Owner: postgres
--

CREATE SCHEMA supabase_migrations

Seesm like the latest Supabase docker image now ships with the supabase_migrations schema already in place which is why there's a conflict?

@ayewo I'm facing the same issue, would you mind giving me a tip on how you solved it? Thanks