New generator for distgen working with minimal images

[frenzymadness]

Python replacement for the black magic in https://github.com/sclorg/container-common-scripts/blob/233496946a028218958d8fbb1857f921ad8a99ef/common.mk#L147-L164 and in https://github.com/sclorg/container-common-scripts/blob/master/generate.sh

It might be possible to merge some of the files specific for the minimal container to the original one but that can be done in the future. This does not apply to README.md which is completely different.

[frenzymadness]

[test]

[pkubatrh]

I am wondering, can we add the black magic replacements to container-common-scripts, or are they too specific for the python repository to be used elsewhere?

[frenzymadness]

I am wondering, can we add the black magic replacements to container-common-scripts, or are they too specific for the python repository to be used elsewhere?

I'd like to replace what we have in the common scripts with this after some testing here. How many images use distgen? I know only about postgres.

[torsava]

Meta thought from meeting with Lumir: Currently in multispec it's necessary to exclude version/distro combinations that are unwanted. How about switching it around and only declaring version/distro combinations that are wanted? It feels more natural and increases readability.

[frenzymadness]

Meta thought from meeting with Lumir: Currently in multispec it's necessary to exclude version/distro combinations that are unwanted. How about switching it around and only declaring version/distro combinations that are wanted? It feels more natural and increases readability.

This is not possible in distgen, see: https://github.com/devexp-db/distgen/blob/main/docs/spec_multispec.rst#multispecs

But, we might be able to implement it outside of distgen. The current workflow takes all the possible combinations from multispec.yaml (generating all of them, excluding specified). But we use them one by one only for files for which we need them for so we might also come with some external yaml config containing all the wanted combinations and remove the matrix section from distgen. Or, we might try to implement this in distgen itself.

[torsava]

First review pass finished. Overall it's looking much better than the previous machinery, I like it!

[torsava]

Meta thought from meeting with Lumir: Currently in multispec it's necessary to exclude version/distro combinations that are unwanted. How about switching it around and only declaring version/distro combinations that are wanted? It feels more natural and increases readability.

This is not possible in distgen, see: https://github.com/devexp-db/distgen/blob/main/docs/spec_multispec.rst#multispecs

But, we might be able to implement it outside of distgen. The current workflow takes all the possible combinations from multispec.yaml (generating all of them, excluding specified). But we use them one by one only for files for which we need them for so we might also come with some external yaml config containing all the wanted combinations and remove the matrix section from distgen. Or, we might try to implement this in distgen itself.

Good to know. That'd be a great idea for the future, but no need to burden this PR with it.

[frenzymadness]

Meta thought from meeting with Lumir: Currently in multispec it's necessary to exclude version/distro combinations that are unwanted. How about switching it around and only declaring version/distro combinations that are wanted? It feels more natural and increases readability.

This is not possible in distgen, see: https://github.com/devexp-db/distgen/blob/main/docs/spec_multispec.rst#multispecs

But, we might be able to implement it outside of distgen. The current workflow takes all the possible combinations from multispec.yaml (generating all of them, excluding specified). But we use them one by one only for files for which we need them for so we might also come with some external yaml config containing all the wanted combinations and remove the matrix section from distgen. Or, we might try to implement this in distgen itself.

Good to know. That'd be a great idea for the future, but no need to burden this PR with it.

I've filled an issue in distgen and it seems that they're open to this idea. Will see how hard it'll be.

[frenzymadness]

See the new commits - some of them are fixup but it's better for review. [test]

[frenzymadness]

Rebased. I'm happy with how the commit history looks now.

[frenzymadness]

[test]

[phracek]

This pull request affects also https://github.com/sclorg/container-common-scripts/pull/275 and varnish-container and postgresql-container which uses manifest.sh

[torsava]

I've done a sanity check of all the commits again. All the previous discussions were successfully addressed. I've pinpointed one typo, but besides that all looks good to me!

However, be warned that if you remove CCCP, Putin might attack you in the next ~30 years. Beware! :smiley: