plugins: initial postmarketOS support

[calebccff]

This PR adds support for flashing postmarketOS to supported devices, using images hosted on https://images.postmarketos.org. The images are indexed via https://images.postmarketos.org/bpo/index.json using the BPO schema: https://gitlab.com/postmarketOS/build.postmarketos.org/-/blob/master/test/testdata/index.schema.json

As it stands it's possible to pick the release channel (edge, v21.12, etc) and user interface of choice (phosh, plasma mobile etc), the images download successfully, but I'm having some trouble decompressing the .xz files.

For now, this will only be able to support some Android devices from the main or community categories: https://wiki.postmarketos.org/wiki/Devices e.g. the OnePlus 6 and PocoPhone F1. Devices like the PinePhone require extra functionality for flashing.

My WIP device configs can be found here: https://github.com/calebccff/installer-configs/tree/postmarketos

Finally, the BPO schema gives a list of all the images for a given interface instead of sorting by build job, making it impossible to know how many images are actually needed for a particular device (some might just need the rootfs and boot images, others might have extra images), hopefully we can upgrade the schema to handle this properly, for now I just fetch all the images with the same timestamp as the latest one.

[NeoTheThird]

Ohhh, interesting! Thank you! I will take a look as soon as i have a second.

[calebccff]

Thanks! Sorry for the messy code :x

If you have the chance could you take a look at the action__decompress() function? Full disclosure this is basically the first time I've touched nodeJS, and I just can't get my head around async / streams, for some reason it just fails immediately to decompress the file.

[NeoTheThird]

Sorry it took me a while to get back to this.

Are you sure you sure you need your own decompress function? It is a pretty generic feature. What we already have is the core:unpack action. It uses 7z, which should support .xz. Plugins can interface with each other by resolving steps containing actions for any plugin (as you're already doing with core:download). Here's the docs: https://github.com/ubports/installer-configs/blob/70fb56f596bce91548d9cb3d25da345883d4f582/v2/schema/action.schema.yml#L381

I'd much rather use a generic function. If we need to tweak core:unpack, that might also be possible. Just trying to reduce feature redundancy, because it will create trouble in the future.

[amartinz]

Hey Caleb, shall we pick this up? :)

[codecov[bot]]

Codecov Report

Merging #2511 (5463ab0) into master (5463ab0) will not change coverage. The diff coverage is n/a.

:exclamation: Current head 5463ab0 differs from pull request most recent head 75a8675. Consider uploading reports for the commit 75a8675 to get more accurate results

@@           Coverage Diff           @@
##           master    #2511   +/-   ##
=======================================
  Coverage   82.72%   82.72%           
=======================================
  Files          29       29           
  Lines         990      990           
=======================================
  Hits          819      819           
  Misses        171      171           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[amartinz]

Alright, this should be working now and good to go after tests have been implemented.

[amartinz]

@calebccff would it be possible for you to give this a try on enchilada once you got some spare time? :)

Artifacts can be found on the latest CI build: https://github.com/ubports/ubports-installer/suites/7900605565/artifacts/335992568

[amartinz]

Tested on enchilada (found one after thinking i lost it).

I do not like the warning messages when unpacking/renaming: warn: aborting run...

Full log:

ubports-installer [master] $> npm run start -- --verbose -f /tmp/enchilada.yml 

> [email protected] start
> ./start.sh "--verbose" "-f" "/tmp/enchilada.yml"

rollup v2.79.0
bundles src/svelte.mjs → public/build/bundle.js...
LiveReload enabled
created public/build/bundle.js in 4.2s

[2022-09-07 14:11:08] waiting for changes...
info: Welcome to the UBports Installer version 0.9.6-beta!
verbose: Versions: {"node":"16.15.0","v8":"10.4.132.22-electron.0","uv":"1.43.0","zlib":"1.2.12","brotli":"1.0.9","ares":"1.18.1","modules":"107","nghttp2":"1.47.0","napi":"8","llhttp":"6.0.4","openssl":"1.1.1","cldr":"41.0","icu":"71.1","tz":"2022a","unicode":"14.0","electron":"20.1.1","chrome":"104.0.5112.102"}
verbose: Reading config file: /tmp/enchilada.yml
info: Installing postmarketOS on your OnePlus 6/6T (enchilada)
info: configuring...
info: settings: {"interface":"phosh","release":"edge"}
verbose: running step {"actions":[{"adb:reboot":{"to_state":"bootloader"}}],"fallback":[{"core:user_action":{"action":"bootloader"}}]}
verbose: running adb action reboot
verbose: running core action user_action
verbose: running step {"actions":[{"fastboot:wait":null}]}
verbose: running fastboot action wait
verbose: running step {"actions":[{"postmarketos:download":null},{"fastboot:erase":{"partition":"dtbo"}},{"fastboot:flash":{"partitions":[{"partition":"userdata","file":"rootfs.img","group":"postmarketOS"},{"partition":"boot","file":"boot.img","group":"postmarketOS"}]}},{"fastboot:continue":null}]}
verbose: running postmarketos action download
verbose: running step {"actions":[{"core:download":{"group":"postmarketOS","files":[{"url":"https://images.postmarketos.org/bpo/edge/oneplus-enchilada/phosh/20220821-1158/20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada.img.xz","checksum":{"sum":"94c005327aa338aa10da5a0a1dd48372099131a1379645685f4a14e6a5a15a3f","algorithm":"sha256"}},{"url":"https://images.postmarketos.org/bpo/edge/oneplus-enchilada/phosh/20220821-1158/20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada-boot.img.xz","checksum":{"sum":"bde5886fe4b933ac06ebc482f193e6172ba233d6574965a35a4603e855ef9d1d","algorithm":"sha256"}}]}},{"core:unpack":{"group":"postmarketOS","files":[{"url":"https://images.postmarketos.org/bpo/edge/oneplus-enchilada/phosh/20220821-1158/20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada.img.xz","checksum":{"sum":"94c005327aa338aa10da5a0a1dd48372099131a1379645685f4a14e6a5a15a3f","algorithm":"sha256"},"archive":"20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada.img.xz","dir":"."},{"url":"https://images.postmarketos.org/bpo/edge/oneplus-enchilada/phosh/20220821-1158/20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada-boot.img.xz","checksum":{"sum":"bde5886fe4b933ac06ebc482f193e6172ba233d6574965a35a4603e855ef9d1d","algorithm":"sha256"},"archive":"20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada-boot.img.xz","dir":"."}]}},{"postmarketos:rename_unpacked_files":{"group":"postmarketOS","files":[{"url":"https://images.postmarketos.org/bpo/edge/oneplus-enchilada/phosh/20220821-1158/20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada.img.xz","checksum":{"sum":"94c005327aa338aa10da5a0a1dd48372099131a1379645685f4a14e6a5a15a3f","algorithm":"sha256"}},{"url":"https://images.postmarketos.org/bpo/edge/oneplus-enchilada/phosh/20220821-1158/20220821-1158-postmarketOS-edge-phosh-18-oneplus-enchilada-boot.img.xz","checksum":{"sum":"bde5886fe4b933ac06ebc482f193e6172ba233d6574965a35a4603e855ef9d1d","algorithm":"sha256"}}]}}]}
verbose: running core action download
info: Downloading 2 files
info: Downloaded file 1 of 2
info: Downloaded file 2 of 2
verbose: running core action unpack
warn: aborting run...
verbose: running postmarketos action rename_unpacked_files
warn: aborting run...
verbose: running fastboot action erase
verbose: running fastboot action flash
verbose: running fastboot action continue
verbose: running step {"actions":[{"core:end":null}]}
verbose: running core action end
info: All done! Your device will now reboot and complete the installation. Enjoy exploring Ubuntu Touch!