WIP: implement `packit build in-image-builder`

[TomasTomecek]

TODO:

• [ ] Write new tests or update the old ones to cover new functionality. (at least 2 test cases for the new code)
• [ ] Update doc-strings where appropriate. (at minimim the 2 new API methods and the new vm_image_build module)
• [ ] agree on design of the CLI interface

$ packit build in-image-builder --help
Usage: packit build in-image-builder [OPTIONS] IMAGE_NAME [PATH_OR_URL]

  Create a VM image in Image Builder.

  Packit loads image build configuration from your packit.yaml file.

  When `--job-config-index` is not specified, the job configuration is loaded
  from your .packit.yaml and the first matching vm_image_build job is used.

  IMAGE_NAME is the name of the image to be created. Please pick something
  unique so it's easy to identify for you in the Image Builder interface and
  can be well associated with the image content.

  [PATH_OR_URL] argument is a local path or a URL to the upstream git
  repository, it defaults to the current working directory

Options:
  --job-config-index INTEGER  Use N-th job definition to load configuration
                              for the image build. The type needs to be
                              vm_image_build.
  --wait / --no-wait          Wait for the build to finish
  -h, --help                  Show this message and exit.

Context: https://github.com/packit/research/tree/main/image-builder

RELEASE NOTES BEGIN

Packit CLI can now submit VM images in Red Hat Image Builder.

RELEASE NOTES END

[softwarefactory-project-zuul[bot]]

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. Warning: Error merging github.com/packit/packit for 1666,fdbeda8b49a6176cbba9c99082913c6f6466a22d

[softwarefactory-project-zuul[bot]]

Build succeeded.

:heavy_check_mark: pre-commit SUCCESS in 2m 03s :heavy_check_mark: packit-tests-rpm SUCCESS in 14m 13s :heavy_check_mark: packit-tests-pip-deps SUCCESS in 13m 58s :heavy_check_mark: packit-tests-git-main SUCCESS in 14m 38s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 3m 22s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 3m 05s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 3m 51s

[softwarefactory-project-zuul[bot]]

Build failed.

:x: pre-commit FAILURE in 1m 34s :x: packit-tests-rpm FAILURE in 12m 41s :x: packit-tests-pip-deps FAILURE in 12m 25s :x: packit-tests-git-main FAILURE in 12m 36s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 2m 40s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 2m 48s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 3m 28s

[softwarefactory-project-zuul[bot]]

Build failed.

:heavy_check_mark: pre-commit SUCCESS in 1m 44s :x: packit-tests-rpm FAILURE in 13m 26s :x: packit-tests-pip-deps FAILURE in 13m 40s :x: packit-tests-git-main FAILURE in 13m 45s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 2m 48s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 2m 55s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 3m 39s

[TomasTomecek]

Thank you @mfocko and @csomh for your feedback!

I updated my original post with the new simplified CLI interface. I made the vm_image_build schema even more explicit.

[packit-as-a-service[bot]]

Failed to load packit config file:

Cannot parse package config: ValidationError({'jobs': {9: {'job': ['Invalid enum member vm_image_build'], 'image_type': ['Unknown field.'], 'image_account_id': ['Unknown field.'], 'image_architecture': ['Unknown field.'], 'image_distribution': ['Unknown field.'], 'packages_to_install': ['Unknown field.']}}}).

For more info, please check out the documentation: https://packit.dev/docs/packit-service or contact us - Packit team

[softwarefactory-project-zuul[bot]]

Build failed.

:heavy_check_mark: pre-commit SUCCESS in 1m 38s :x: packit-tests-rpm FAILURE in 12m 33s :x: packit-tests-pip-deps FAILURE in 13m 24s :x: packit-tests-git-main FAILURE in 13m 12s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 2m 43s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 2m 46s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 3m 34s

[packit-as-a-service[bot]]

Failed to load packit config file:

Cannot parse package config: ValidationError({'jobs': {9: {'job': ['Invalid enum member vm_image_build'], 'packages_to_install': ['Unknown field.'], 'image_type': ['Unknown field.'], 'image_distribution': ['Unknown field.'], 'image_architecture': ['Unknown field.'], 'image_account_id': ['Unknown field.']}}}).

For more info, please check out the documentation: https://packit.dev/docs/packit-service or contact us - Packit team

[csomh]

Thank you @mfocko and @csomh for your feedback!

I updated my original post with the new simplified CLI interface. I made the vm_image_build schema even more explicit.

I like this direction to read the arguments from a file. I'm a little bit concerned forcing this file to be .packit.yaml, which to me is a file with a slower change rate. I imagine a user who would use this command would need to change .packit.yaml way to often. Would it make sense to be able to pass content from stdin or any other file?

[TomasTomecek]

/packit build

[TomasTomecek]

Thank you @mfocko and @csomh for your feedback! I updated my original post with the new simplified CLI interface. I made the vm_image_build schema even more explicit.

I like this direction to read the arguments from a file. I'm a little bit concerned forcing this file to be .packit.yaml, which to me is a file with a slower change rate. I imagine a user who would use this command would need to change .packit.yaml way to often. Would it make sense to be able to pass content from stdin or any other file?

I share your concern and want to elaborate on it: with the number of customizations users can do now to images (and will probably do even more in the future), it doesn't sound feasible for us to define schema for all these attributes.

Example: https://console.redhat.com/#components-schemas-ComposeRequest (see ComposeRequest if it doesn't open properly)

In the documentation I saw that the composer CLI works with a file where the image is defined - hopefully we could reuse it. I'll reach out to the team and see what they think about this.

[softwarefactory-project-zuul[bot]]

Build failed.

:x: pre-commit FAILURE in 1m 39s :heavy_check_mark: packit-tests-rpm SUCCESS in 13m 08s :heavy_check_mark: packit-tests-pip-deps SUCCESS in 13m 47s :heavy_check_mark: packit-tests-git-main SUCCESS in 13m 21s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 2m 40s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 2m 55s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 3m 26s

[TomasTomecek]

@ondrejbudai had a good suggestion on the internal chat about the structure. He suggested to passthough all the image customizations from packit.yaml into Image Builder API.

- job: vm_image_build 
    trigger: pull_request
    packit_instances: ["stg"]
    image_distribution: rhel-8
    image_type: aws
    image_architecture: x86_64
    image_account_id: "727920394381"
    packages_to_install: [packit]
    owner: packit
    project: packit-dev
    targets: [epel-8]
    customizations:
       filesystems:
            - /app
       packages:
            - osbuild

I'd probably go even one step further:

- job: vm_image_build 
  trigger: pull_request
  packit_instances: ["stg"]
  owner: packit
  project: packit-dev
  targets: [epel-8]
  image_distribution: rhel-8
  image_customizations:
    filesystems:
    - /app
    packages:
     osbuild
  image_requests:
    architecture: x86_64
    image_type: aws
    upload_request: 
      options:
        share_with_accounts": ["123456789012"]
      type: aws

[softwarefactory-project-zuul[bot]]

Build failed.

:x: pre-commit FAILURE in 2m 17s :x: packit-tests-rpm FAILURE in 1m 58s :x: packit-tests-pip-deps FAILURE in 2m 46s :x: packit-tests-git-main FAILURE in 2m 59s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 3m 09s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 3m 19s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 3m 55s

[softwarefactory-project-zuul[bot]]

Build succeeded.

:heavy_check_mark: pre-commit SUCCESS in 1m 53s :heavy_check_mark: packit-tests-rpm SUCCESS in 13m 07s :heavy_check_mark: packit-tests-pip-deps SUCCESS in 12m 57s :heavy_check_mark: packit-tests-git-main SUCCESS in 14m 26s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 3m 21s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 3m 34s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 3m 46s

[softwarefactory-project-zuul[bot]]

Build succeeded.

:heavy_check_mark: pre-commit SUCCESS in 1m 53s :heavy_check_mark: packit-tests-rpm SUCCESS in 11m 22s :heavy_check_mark: packit-tests-pip-deps SUCCESS in 12m 26s :heavy_check_mark: packit-tests-git-main SUCCESS in 12m 54s :heavy_check_mark: packit-tests-pip-deps-sess-rec SUCCESS in 2m 59s :heavy_check_mark: packit-tests-git-main-sess-rec SUCCESS in 3m 19s :heavy_check_mark: reverse-dep-packit-service-tests SUCCESS in 2m 50s

[softwarefactory-project-zuul[bot]]

Build succeeded (gate pipeline).

:heavy_check_mark: pre-commit SUCCESS in 1m 55s