OTA Updates?

[kiwisibk]

What's about OTA updates? My "Chromebook" is always saying that I'm on the latest version (71)...

Thanks

[anupanup2001]

Same with me. it says installing update. then it says it's latest without updating.

[MuntashirAkon]

It should be clear to you by now that (native) OTA update will broke your installation. You'll have to update manually using a Linux distro (or the Chrome OS itself, but requires more technical knowledge) using the process described here (if you've used Method 3).

In case you're applied TMP1.2 patch to create chromium_os.img, you cannot update to the latest version (if you do, you will not be able to log in).

Possible solution

The only (easy-to-understand) way I can think of is to create a chromium_os.img using the latest recovery update and copy the partition 3 (ROOT-A) of this image to the partition 3 (the ROOT-A parition, can vary if you've multibooted) of your HDD.

[MuntashirAkon]

I've made some progress. You can check it here.

[b1t0y]

@MuntashirAkon in your github what do you mean Install in HDF with cd ~/Downloads/ChromeOS/ && sudo bash install_chromium.sh chromiumos_image.img sda4 sda5 sda6. Do I use exactly this command then replace later? Or do I replace it right away during install_chromium.sh?

[imperador]

This could be done in some ways: 1 - Two partition system with a shell script that runs at boot. It changes the main partition and applies an update to the other partitions. The problem with this approach is that it would require two partitions with 4GB+ each. But that is how it's done on the original chrome os. This is also how @MuntashirAkon is doing. We made something like this, but this approach won't work with dualboot as it is now.

2 - A minimal partition only for updates. The same approach as above but with a custom CLI-only linux distro on a small partition that only has the essential commands. It would require powerwashing the first partition (data) every time because you still need a space to store the files and it is the best candidate.

3 - A very difficult approach that only replaces essential files. It has great potential to solve the problem without consuming much space or adding unnecessary partitions, but also has great potential to mess everything up. (I am currently researching about this approach, as it should solve most update problems without creating new ones)

[snmcmillan]

Another way is to implement an EFI-bootloader-based VBOOT and Depthcharge stub to convince ChromeOS that it's running on real chromeos hardware (like Clover in the Hackintosh scene), and use NVRAM to store variables such as current rootfs/boot partition. Likely outside most of the dev's skill set (and not in my skill set either).

If we want to take it even further, various laptops already have coreboot ports, it's just a matter of configuring VBOOT and getting proper Flashmaps for depthcharge, and pretend that our device is a given chromeos device, which would give us all of the advantages of ChromeOS, such as native recovery mode, native recovery and native devmode toggle without having to pass a kernel argument.

[Crescendo-BLYAT]

This could be done in some ways: 1 - Two partition system with a shell script that runs at boot. It changes the main partition and applies an update to the other partitions. The problem with this approach is that it would require two partitions with 4GB+ each. But that is how it's done on the original chrome os. This is also how @MuntashirAkon is doing. We made something like this, but this approach won't work with dualboot as it is now.

leave the dualboot users out for a while & let this approach work properly, then you can create another script just for dualboot users... 4GB is small in current generation of hardware, so just create it for the sake of proper live update...

just my opinion.... ;)

[Crescendo-BLYAT]

@imperador somehow updating the installation is difficult coz the live OS (Debian 9, Ubuntu 19.04, latest Gparted Live) always unable to mount sda5 said that it contains newer features....

[dragon788]

@SebastianMcMillan even without those tools it might be possible with efimanager or similar to look at the current ROOT and set the other ROOT instance as the next boot target during an update, but that would probably be easiest in a multi-boot scenario while your method would work best if you were only booting a Chromefy system.

[MuntashirAkon]

@imperador I'm porting the update_engine repo. Hopefully we can update our OS like a regular Chromebook (with a small catch).

[MuntashirAkon]

@imperador @arnoldthebat do you know how to verify and/or decrypt signed updates?

[TCU14]

Currently we do not know how, though we are working on trying to figure that out so that everyone can update using the official channels without issue. As of now it appears to be possible to update if you clean install ChromiumOS, but if you multiboot and installed ChromiumOS after another operating system and don't have your root partitions on 3/5, you won't be able to update as ChromeOS doesn't recognize the slot you are currently booted on as slot 0 or 1. Please update us here if you figure out how to apply a delta update locally

[MuntashirAkon]

@TCU14 I can download the delta/full updates, but I don't know how to apply the updates. The update files contain addresses of the disk where compressed files are to be uncompressed and replaced. This is what I found out so far.

[MuntashirAkon]

UPDATE: It appears that delta update may not be possible since the root partition is heavily modified. But whether the full update is possible is still in question.

Currently, I'm using the recovery file to update Chrome OS (and it's working fine for me). You can checkout the script here.

[MuntashirAkon]

Another way is to implement an EFI-bootloader-based VBOOT and Depthcharge stub to convince ChromeOS that it's running on real chromeos hardware (like Clover in the Hackintosh scene), and use NVRAM to store variables such as current rootfs/boot partition. Likely outside most of the dev's skill set (and not in my skill set either).

Unlike macOS, Chrome OS doesn't support dual or multi boot (we may able to add support for it but it'll require changing the partition numbers of the HDD since the disk format is hard coded). The purpose of Clover or OpenCore is to prevent modifying macOS as best as possible by providing some sort of HW emulation (with the help of FakeSMC or VirtualSMC and ACPI tables). For instance, I'm using macOS 10.14 on a Dell laptop without modifying anything in the OS. This may not be possible with Chrome OS (macOS doesn't require deleting any folder or replacing files, but Chrome OS does). Also, notice that we're converting Chromium OS to Chrome OS using a recovery file that means we're already doing (some sort of) emulation.

The NVRAM idea is nice and extensively used in Clover/OC. But (unlike macOS) this also requires heavy modification of the OS itself. If you look at the source code, you'll find that it doesn't use NVRAM to determine the rootfs/state/boot partitions.

Lastly, unlike macOS, the codebase of Chrome OS is ever changing. In future, many things that are hard coded right now may be generalised, and many operations that are currently carried out using Shell scripts will be converted into C++. This happened before and will happen in future as well. So, I don't think any dev will be interested in creating a custom bootloader for Chrome OS yet.

[kitakar5525]

Hello. Thanks to your work @MuntashirAkon, I can now get links to images from update server. (You can get only link to image and not perform update by doing like this, if you're interested)

omaha_cros_update.sh: add option --check-only to perform update check only

From 913fe31db0a3818744502700d48b662991a83e96 Mon Sep 17 00:00:00 2001
From: kitakar5525 <[email protected]>
Date: Tue, 17 Dec 2019 16:44:11 +0900
Subject: [PATCH] omaha_cros_update.sh: add option --check-only to perform
 update check only

---
 omaha_cros_update.sh | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/omaha_cros_update.sh b/omaha_cros_update.sh
index bf2fffa..3879fdd 100644
--- a/omaha_cros_update.sh
+++ b/omaha_cros_update.sh
@@ -3,6 +3,31 @@
 # NOTE: TPM 1.2 fix is adapted from the Chromefy project and
 # this copyright doesn't apply them.
 
+# Check if there's an update
+# Takes no params, just leave it to existing scripts.
+function check_for_update {
+    echo "Checking for update..."
+
+    . omaha_response_handler_action.sh
+
+    OmahaRequestAction_TransferComplete
+    # If no update available, the function exits with 1.
+    # So, no need to take care here.
+
+    # Show all available URLs
+    for (( i=0; i<${#ORA_payload_urls[@]}; i++ )); do
+        echo ${ORA_payload_urls[${i}]}
+    done
+
+    exit 0
+}
+
+# XXX: This action does not need root privilege. So, I placed it here.
+# I think there is better place to do this action, but for now
+# do this here.
+if [ "$1" == "--check-only" ]; then
+    check_for_update
+fi
 
 if [ $UID -ne 0 ]; then
     echo "This script must be run as root!"
-- 
2.24.1

I'm not sure if you're already aware, but you can extract kernel and root image from the downloaded signed image using paycheck.py from update_engine/scripts.

For example:

$ cd /path/to/chromeos_image
$ git clone --depth 1 https://android.googlesource.com/platform/system/update_engine

# (following command depends on python-protobuf and python-lzma)

# extract kernel partition image to kernel.img and
# root partition image to root.img
$ ./update_engine/scripts/paycheck.py chromeos_12607.58.0_eve_beta-channel_full_mp-v2.bin-76ce2f908fd905e85f0fd6a02d1a5f27.signed --out_dst_part_paths kernel.img root.img

Now, I can mount root.img. However, I cannot mount kernel.img yet and I can't figure out why. But maybe not a problem when we update existing chromeos.

[madebypixel02]

Hello. Thanks to your work @MuntashirAkon, I can now get links to images from update server. (You can get only link to image and not perform update by doing like this, if you're interested)

omaha_cros_update.sh: add option --check-only to perform update check only I'm not sure if you're already aware, but you can extract kernel and root image from the downloaded signed image using paycheck.py from update_engine/scripts.

For example:

$ cd /path/to/chromeos_image
$ git clone --depth 1 https://android.googlesource.com/platform/system/update_engine

# (paycheck.py depends on python-protobuf, install it if not installed)

# extract kernel partition image to kernel.img and
# root partition image to root.img
$ ./update_engine/scripts/paycheck.py chromeos_12607.58.0_eve_beta-channel_full_mp-v2.bin-76ce2f908fd905e85f0fd6a02d1a5f27.signed --out_dst_part_paths kernel.img root.img

Now, I can mount root.img. However, I cannot mount kernel.img yet and I can't figure out why. But maybe not a problem when we update existing chromeos.

Sounds interesting. Is it usable as it is right now? Using eve v79 now, and I'd be happy to report when I wish to update to v80 beta

[kitakar5525]

Sounds interesting. Is it usable as it is right now? Using eve v79 now, and I'd be happy to report when I wish to update to v80 beta

Yes, and I'm using the image now. (BTW, v80 Beta for chromeos is not released yet. Maybe it will be released after Thu, Dec 19, 2019: https://chromiumdash.appspot.com/schedule)

Basically, you need to do the following steps:

1. Mount the root.img
2. Cleanup ROOT-A (sudo rm -rf $ROOT-A/*)
3. Copy the contents to ROOT-A (cp -a to preserve permissions)
4. Do the same modifications manually that croissant.sh does. (modify write_gpt.sh, copy kernel, …)

[kitakar5525]

update_engine/scripts/test_paycheck.sh maybe a hint to do delta update: Example:

# depends on python-protobuf, python-lzma and bsdiff

./update_engine/scripts/test_paycheck.sh \
# old_full
chromeos_12607.34.0_atlas_beta-channel_full_mp.bin-03239934eda5483a2ab4df2cd2f000cd.signed \
# delta
chromeos_12607.34.0-12607.47.0_atlas_beta-channel_delta_mp.bin-1b9dee3e55f1219957d27bf1637dc8cf.signed \
# new_full
chromeos_12607.47.0_atlas_beta-channel_full_mp.bin-3e809e14e01f25ad49a95e6858978a75.signed

If I understand correctly, the script tests if the result of old_full+delta is the same as new_full or not.

Link to the script: https://android.googlesource.com/platform/system/update_engine/+/refs/heads/master/scripts/test_paycheck.sh

[TCU14]

When I was experimenting with paycheck a couple minutes ago, it failed to extract the signed payload to kernel.img and root.img, I got the following output:

./update_engine/scripts/paycheck.py /mnt/chromeos/MyFiles/Downloads/chromeos_12499.66.0-12607.58.0_atlas_stable-channel_delta_mp.bin-56475df13c8407d30d4ae2d89c8d6b27.signed --out_dst_part_paths kernel.img root.img' 'Error: trying to apply a delta update without src partition(s) kernel, root

I did dd to get my hands on both kernel.img and root.img from an Atlas recovery image. I'm wondering if you see any obvious problems with what I did

[kitakar5525]

Sorry, I should have clarified that the example was for full payload. I can't figure out how to apply delta payload yet.

When I look into apply_delta_payload() in the test_paycheck.sh script, it seems that --dst_part_paths argument takes ${NEW_FULL_KERN_PART} and "${NEW_FULL_ROOT_PART}". I'm not sure why the function needs those new_full_kern and new_full_root image.

I guess we don't need the argument. So, the command applying delta payload to old images may be like this, but failed:

# old_kern.img and old_root.img are images extracted from full payload beforehand
$ ./update_engine/scripts/paycheck.py \
        chromeos_12607.56.0-12607.58.0_atlas_beta-channel_delta_mp.bin-e1160ca5008dca9f676b730b76e88cd0.signed \
        --src_part_paths old_kern.img old_root.img \
        --out_dst_part_paths new_kern.img new_root.img
bspatch: usage: bspatch oldfile newfile patchfile

Traceback (most recent call last):
  File "./update_engine/scripts/paycheck.py", line 304, in <module>
    sys.exit(main(sys.argv))
  File "./update_engine/scripts/paycheck.py", line 274, in main
    payload.Apply(out_dst_parts, **dargs)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/payload.py", line 332, in Apply
    helper.Run(new_parts, old_parts=old_parts)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 681, in Run
    new_part_info[name], old_parts.get(name, None), old_part_info[name])
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 605, in _ApplyToPartition
    new_part_file, new_part_info.size)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 557, in _ApplyOperations
    new_part_file)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 447, in _ApplyDiffOperation
    subprocess.check_call(bspatch_cmd)
  File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bspatch', '/dev/fd/8', '/dev/fd/7', '/tmp/tmpRA3x66', '2699264:4096', '2699264:4096']' returned non-zero exit status 1

And when I run the test_paycheck.sh, it also fails at applying delta payload with the same error:

log

$ ./update_engine/scripts/test_paycheck.sh \
        chromeos_12607.56.0_atlas_beta-channel_full_mp.bin-08a842276af262634375e3f32add2ed4.signed \
        chromeos_12607.56.0-12607.58.0_atlas_beta-channel_delta_mp.bin-e1160ca5008dca9f676b730b76e88cd0.signed \
        chromeos_12607.58.0_atlas_beta-channel_full_mp.bin-c5929e276ce9ea4ba455d62fc0b8cf79.signed
Checking payloads...

real    0m10.893s
user    0m9.322s
sys    0m1.066s

real    0m10.290s
user    0m9.261s
sys    0m0.954s

real    0m23.666s
user    0m23.451s
sys    0m0.099s
Done
Initiating application of payloads at /tmp/test_paycheck.wUmeQth7
Applying old full payload...

real    1m5.694s
user    1m4.196s
sys    0m1.379s
Done
Applying new full payload...

real    1m8.385s
user    1m6.479s
sys    0m1.776s
Done
Applying delta payload to old partitions...
bspatch: usage: bspatch oldfile newfile patchfile

Traceback (most recent call last):
  File "./update_engine/scripts/paycheck.py", line 304, in <module>
    sys.exit(main(sys.argv))
  File "./update_engine/scripts/paycheck.py", line 274, in main
    payload.Apply(out_dst_parts, **dargs)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/payload.py", line 332, in Apply
    helper.Run(new_parts, old_parts=old_parts)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 681, in Run
    new_part_info[name], old_parts.get(name, None), old_part_info[name])
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 605, in _ApplyToPartition
    new_part_file, new_part_info.size)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 557, in _ApplyOperations
    new_part_file)
  File "/home/kitakar5525/Downloads/delta_test/update_engine/scripts/update_payload/applier.py", line 447, in _ApplyDiffOperation
    subprocess.check_call(bspatch_cmd)
  File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bspatch', '/dev/fd/8', '/dev/fd/7', '/tmp/tmpDco5og', '2699264:4096', '2699264:4096']' returned non-zero exit status 1

real    0m7.738s
user    0m7.388s
sys    0m0.330s

[MuntashirAkon]

I'm not sure if you're already aware, but you can extract kernel and root image from the downloaded signed image using paycheck.py from update_engine/scripts.

For example:

$ cd /path/to/chromeos_image
$ git clone --depth 1 https://android.googlesource.com/platform/system/update_engine

# (following command depends on python-protobuf and python-lzma)

# extract kernel partition image to kernel.img and
# root partition image to root.img
$ ./update_engine/scripts/paycheck.py chromeos_12607.58.0_eve_beta-channel_full_mp-v2.bin-76ce2f908fd905e85f0fd6a02d1a5f27.signed --out_dst_part_paths kernel.img root.img

Now, I can mount root.img. However, I cannot mount kernel.img yet and I can't figure out why. But maybe not a problem when we update existing chromeos.

Thanks, I'll check it out.

[MuntashirAkon]

However, I cannot mount kernel.img yet and I can't figure out why.

Sorry, but how to mount Kernel partitions in first place? I wasn't able to mount the partition, let alone the img.

[MuntashirAkon]

Sounds interesting. Is it usable as it is right now? Using eve v79 now, and I'd be happy to report when I wish to update to v80 beta

Yes, and I'm using the image now. (BTW, v80 Beta for chromeos is not released yet. Maybe it will be released after Thu, Dec 19, 2019: https://chromiumdash.appspot.com/schedule)

Basically, you need to do the following steps:

1. Mount the root.img

2. Cleanup ROOT-A (`sudo rm -rf $ROOT-A/*`)

3. Copy the contents to ROOT-A (`cp -a` to preserve permissions)

4. Do the same modifications manually that croissant.sh does. (modify write_gpt.sh, copy kernel, …)

Seems to be very neat and typical process, but I intend to do the update in ROOT-B partition in my script until the process is bug-free.

[kitakar5525]

Sorry, but how to mount Kernel partitions in first place? I wasn't able to mount the partition, let alone the img.

Ah, I see. I've confused KERN-A with EFI-SYSTEM which contains vmlinuz.A.

Seems to be very neat and typical process, but I intend to do the update in ROOT-B partition in my script until the process is bug-free.

Yes. I want to update chromeos within chromeos, and to do so, I need to do A/B update eventually. But for now, I'm not so sad with updating chromeos from Linux.

[MuntashirAkon]

Hi, I've updated my repo to support Omaha updates. Just download the repo and follow the readme to install the script.

Not documented in the repo: you can switch between various channels by exporting CUSTOM_RELEASE_TRACK before running the updater through terminal. The values of CUSTOM_RELEASE_TRACK can be {dev|canary|beta|stable}-channel. I haven't checked it yet though, so please use with caution.

[MuntashirAkon]

update_engine/scripts/test_paycheck.sh maybe a hint to do delta update: Example:

# depends on python-protobuf, python-lzma and bsdiff

./update_engine/scripts/test_paycheck.sh \
# old_full
chromeos_12607.34.0_atlas_beta-channel_full_mp.bin-03239934eda5483a2ab4df2cd2f000cd.signed \
# delta
chromeos_12607.34.0-12607.47.0_atlas_beta-channel_delta_mp.bin-1b9dee3e55f1219957d27bf1637dc8cf.signed \
# new_full
chromeos_12607.47.0_atlas_beta-channel_full_mp.bin-3e809e14e01f25ad49a95e6858978a75.signed

If I understand correctly, the script tests if the result of old_full+delta is the same as new_full or not.

Link to the script: https://android.googlesource.com/platform/system/update_engine/+/refs/heads/master/scripts/test_paycheck.sh

Sorry, I didn't notice this comment before. Yes, you are correct. But the problem with this process is that to save around 500 MB bandwidth, one have to waste more than 1 GB of space!

[kitakar5525]

And anyway we may not be able to do delta update if we modify ROOT as you mentioned before. I wish we could use overlay or something for ROOT instead of actually modifying ROOT…

[MuntashirAkon]

And anyway we may not be able to do delta update if we modify ROOT as you mentioned before. I wish we could use overlay or something for ROOT instead of actually modifying ROOT…

This is a problem with chromebooks as well when developer mode is enabled.

[MuntashirAkon]

Delta update is not a big thing to worry about. Android users are quite accustomed to this. On android, delta update isn't possible even with a "systemless" root. One way would be to dynamically add/remove/replace libraries and drivers before system starts but the benefits are much less than the costs (space to keep the files and increased startup time).