builderv2: do `package init-cache` automatically

[marmarek]

How to file a helpful issue

The problem you're addressing (if any)

Normally builder creates the whole build env from scratch each time. Twice for a given package ("prep" and "build" stages). That takes time.

The solution you'd like

Automatically do init-cache stage if cache isn't there already. Skip the automatic init-cache, if it's created already. Do not skip it, if user explicitly requested to re-create the cache. The re-creating can require extra option, but the message about skipping init-cache should include info how to force re-create.

The value to a user, and who that user might be

Faster build times.

Completion criteria checklist

(This section is for developer use only. Please do not modify it.)

[marmarek]

For similar UX reasons, maybe it should not remove old cache if init-cache fails?

[DemiMarie]

Which packages should go in the cache?

[marmarek]

This ticket is only about doing plain qb package init-cache (base chroot) automatically, without extra lists of packages to include.

[DemiMarie]

Ah, I thought that init-cache was useless without such a list.

[marmarek]

There is an issue with the current approach: it calls init-cache for all configured dists, even if no package is going to be built. For example ./qb -c core-admin package all will init cache for all VM dists too, even though only host one is needed...

[fepitre]

There is an issue with the current approach: it calls init-cache for all configured dists, even if no package is going to be built. For example ./qb -c core-admin package all will init cache for all VM dists too, even though only host one is needed...

Yes the approach is not good. To be honest, the best approach is to leverage our already built-in job dependency resolution by marking explicitly init-cache in the stages to be done. The automatic dependency resolution will mark init-cache job for each component mandatory for prep. But, this is not done automatically unless calling package all. Adding some magic is possible but should this really happen or calling all is enough?

[fepitre]

I checked that's also not sufficient. init-cache will be called even if no package has to be built later on. I guess we still need some specific handling somewhere. I'll think about it.

[marmarek]

Some examples to consider:

1. qb package all
2. qb -d host-fc41 package all
3. qb -d host-fc41 package init-cache
4. qb package init-cache
5. qb -c core-admin package all
6. qb -c core-admin package init-cache
7. qb -o components+core-admin package init-cache (should be equivalent to 6)

The number 4 is not obvious, but IMO for consistency it should also init cache only for dists that are needed for any package, so if your builder.yml has only host packages, it should skip vm dists.