scripts icon indicating copy to clipboard operation
scripts copied to clipboard

jenkins/kola/do: use gzipped image file

Open jepio opened this issue 3 years ago • 10 comments

jenkins/kola/do: use gzipped image file

The bzipped image file sometimes results in a timeout during decompression on the remote side. Gzipped images are supposed to be better supported by DigitalOcean, so use those.

How to use

[ describe what reviewers need to do in order to validate this PR ]

Testing done

[Describe the testing you have done before submitting this PR. Please include both the commands you issued as well as the output you got.]

  • [ ] Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)

jepio avatar Mar 28 '22 08:03 jepio

TODO: update https://github.com/flatcar-linux/flatcar-docs/blob/main/docs/installing/cloud/digitalocean.md as soon as the gzipped DigitalOcean images make it to all the channels.

jepio avatar Mar 28 '22 09:03 jepio

Maybe we should backport the change now to align the build system in all channels. So far we had no problems coming up from the change, and misalignment always causes inconsistencies (e.g., the gzip change gets documented but LTS doesn't have it) and cherry-pick troubles later.

pothos avatar May 06 '22 01:05 pothos

The changes to compression handling? Yeah, we can do that. The reason this PR is not merged is because when I started testing it, the import still failed with a timeout several times in a row. So I lost some faith in whether bzip/gzip makes a real difference. Had to bump up the timeout instead (https://github.com/flatcar-linux/mantle/commit/d48d25de1cd3ba932ca63e7ef1f03a260f524762).

jepio avatar May 06 '22 07:05 jepio

I think it was even a problem without any timeout constraints but maybe it doesn't happen often anymore because the hardware running the decompression job got upgraded as well or has less load than before, or lbzip2 is used there now…

I'm still for merging but think that if we do so we should first cherry-pick everywhere.

pothos avatar May 06 '22 07:05 pothos

We can merge this now and cherry pick

pothos avatar Nov 17 '22 16:11 pothos

Merging this as is now is pointless as this updates scripts in jenkins directory, which are rather unused.

krnowak avatar Nov 17 '22 16:11 krnowak

Ah, we now need to change DIGITALOCEAN_IMAGE_URL_TEMPLATE in ci-automation/ci-config.env

pothos avatar Nov 17 '22 16:11 pothos

I'm still not convinced this helps. When testing this I found that there were still occasional timeouts during the upload, same as when using bzip2. If no one objects I'd rather close.

jepio avatar Nov 17 '22 17:11 jepio

The main benefit would be a faster image import when running the tests

pothos avatar Nov 17 '22 17:11 pothos

If it were faster I would agree, but I don't remember seeing any difference. I still encourage someone else to give it a try.

jepio avatar Nov 17 '22 17:11 jepio