assisted-service OKD deployment using podman

I am following the instructions for a podman deployment for okd. When I put in my pull secret either one pulled from redhat or the fake I get the below error

pull secret for new cluster is invalid: pull secret must contain auth for "registry.ci.openshift.org"

I tried putting a fake entry in the pull secret for registy.ci.openshift.org but then it fails to pull. Are we not able to install okd with the assisted installer?

Jul 11 '24 13:07 matuck

the versions below are known to work:


{"openshift_version":"4.12","cpu_architecture":"x86_64","url":"https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/37.20221127.3.0/x86_64/fedora-coreos-37.20221127.3.0-live.x86_64.iso","version":"37.20221127.3.0"}
,

{"openshift_version":"4.12-scos","cpu_architecture":"x86_64","url":"https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/37.20221127.3.0/x86_64/fedora-coreos-37.20221127.3.0-live.x86_64.iso","version":"37.20221127.3.0"}
]'
RELEASE_IMAGES: '[

{"openshift_version":"4.12","cpu_architecture":"x86_64","cpu_architectures":["x86_64"],"url":"quay.io/openshift/okd:4.12.0-0.okd-2023-04-01-051724","version":"4.12.0-0.okd-2023-04-01-051724","default":true}
,

{"openshift_version":"4.12-scos","cpu_architecture":"x86_64","cpu_architectures":["x86_64"],"url":"quay.io/okd/scos-release:4.12.0-0.okd-scos-2023-03-23-213604","version":"4.12.0-0.okd-scos-2023-03-23-213604","default":false}
]'

@mlorenzofr do you know if there are other OKD/fedora combinations that we support in assisted installer?

Jul 12 '24 14:07 rccrdpccl

I tested an install of the Stable Branch of OKD SCOS with Secure Boot Disabled: https://amd64.origin.releases.ci.openshift.org/#4-scos-stable

I ran into an issue with stable OKD FCOS and DNS systemd-resolve being disabled on rendezvous once bootstrap sequence starts

REF: https://github.com/okd-project/okd/discussions/1967#discussioncomment-10034165

This also relates to this issue: https://github.com/openshift/assisted-service/issues/6552

Jul 15 '24 21:07 ArthurVardevanyan

These versions have been tested with assisted-installer and work to install OKD versions 4.13, 4.14 and 4.15:

OS_IMAGES: '[
    {
      "openshift_version":"4.13-scos",
      "cpu_architecture":"x86_64",
      "url":"https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.13/4.13.10/rhcos-4.13.10-x86_64-live.x86_64.iso",
      "version":"413.92.202308210212-0"
    },
    {
      "openshift_version":"4.14-scos",
      "cpu_architecture":"x86_64",
      "url":"https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-x86_64-live.x86_64.iso",
      "version":"414.92.202310210434-0"
    },
    {
      "openshift_version":"4.15-scos",
      "cpu_architecture":"x86_64",
      "url":"https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.15/4.15.0/rhcos-4.15.0-x86_64-live.x86_64.iso",
      "version":"415.92.202402130021-0"
    }
]'

RELEASE_IMAGES: '[
    {
      "openshift_version":"4.13-scos",
      "cpu_architecture":"x86_64",
      "cpu_architectures":["x86_64"],
      "url":"quay.io/okd/scos-release:4.13.0-0.okd-scos-2023-10-19-111256",
      "version":"4.13.0-0.okd-scos-2023-10-19-111256",
      "default":false
    },
    {
      "openshift_version":"4.14-scos",
      "cpu_architecture":"x86_64",
      "cpu_architectures":["x86_64"],
      "url":"quay.io/okd/scos-release:4.14.0-0.okd-scos-2024-01-30-032525",
      "version":"4.14.0-0.okd-scos-2024-01-30-032525",
      "default":false
    },
    {
      "openshift_version":"4.15-scos",
      "cpu_architecture":"x86_64",
      "cpu_architectures":["x86_64"],
      "url":"quay.io/okd/scos-release:4.15.0-0.okd-scos-2024-01-18-223523",
      "version":"4.15.0-0.okd-scos-2024-01-18-223523",
      "default":false
    }
]'

Jul 16 '24 15:07 mlorenzofr

And I have not tested it personally, but this should also work for version 4.16 (they are used in tests):

OS_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "url": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240322.3.1/x86_64/fedora-coreos-39.20240322.3.1-live.x86_64.iso",
        "version": "39.20240322.3.1"
    }
]'

RELEASE_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "cpu_architectures": ["x86_64"],
        "url": "registry.ci.openshift.org/origin/release:4.16",
        "version": "4.16.0-0.okd",
        "default": true,
        "support_level": "beta"
    }
]'

Jul 16 '24 15:07 mlorenzofr

And I have not tested it personally, but this should also work for version 4.16 (they are used in tests):

OS_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "url": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240322.3.1/x86_64/fedora-coreos-39.20240322.3.1-live.x86_64.iso",
        "version": "39.20240322.3.1"
    }
]'

RELEASE_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "cpu_architectures": ["x86_64"],
        "url": "registry.ci.openshift.org/origin/release:4.16",
        "version": "4.16.0-0.okd",
        "default": true,
        "support_level": "beta"
    }
]'

Does the test pipeline by chance have a proper pull secret to: registry.ci.openshift.org?

Jul 16 '24 16:07 ArthurVardevanyan

I have got past this error one of 2 ways now. The okd-configmap.yml file in deploy/podman needs to change some entries. The default that is there points to registry.ci.openshift.org. If I want to use that I need to change the line PUBLIC_CONTAINER_REGISTRIES: 'quay.io' to PUBLIC_CONTAINER_REGISTRIES: 'quay.io,registry.ci.openshift.org'. The other way I have got past this error is to change the OS_IMAGE line and RELEASE_IMAGE line to use ones that are quay as suggested here. I suggest changing the default values in that file or maybe updating the README in deploy/podman to note this. I have not done the full process yet but this seems to have worked and got past my error.

Jul 17 '24 13:07 matuck

related https://github.com/openshift/assisted-service/issues/6552 https://github.com/okd-project/okd/discussions/1964

Jul 17 '24 13:07 rccrdpccl

And I have not tested it personally, but this should also work for version 4.16 (they are used in tests):

OS_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "url": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240322.3.1/x86_64/fedora-coreos-39.20240322.3.1-live.x86_64.iso",
        "version": "39.20240322.3.1"
    }
]'

RELEASE_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "cpu_architectures": ["x86_64"],
        "url": "registry.ci.openshift.org/origin/release:4.16",
        "version": "4.16.0-0.okd",
        "default": true,
        "support_level": "beta"
    }
]'

Does the test pipeline by chance have a proper pull secret to: registry.ci.openshift.org?

Yes, it has. If you want to use the public images hosted on registry.ci.openshift.org you can use the PUBLIC_CONTAINER_REGISTRIES environment variable as explained by @matuck and assisted-service will pass the validation.

Jul 17 '24 16:07 mlorenzofr

And I have not tested it personally, but this should also work for version 4.16 (they are used in tests):
OS_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "url": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240322.3.1/x86_64/fedora-coreos-39.20240322.3.1-live.x86_64.iso",
        "version": "39.20240322.3.1"
    }
]'

RELEASE_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "cpu_architectures": ["x86_64"],
        "url": "registry.ci.openshift.org/origin/release:4.16",
        "version": "4.16.0-0.okd",
        "default": true,
        "support_level": "beta"
    }
]'
Does the test pipeline by chance have a proper pull secret to: registry.ci.openshift.org?
Yes, it has. If you want to use the public images hosted on registry.ci.openshift.org you can use the PUBLIC_CONTAINER_REGISTRIES environment variable as explained by @matuck and assisted-service will pass the validation.

But if the registry doesn't require a pull secret, shouldn't it already be included by default?

Jul 17 '24 17:07 ArthurVardevanyan

And I have not tested it personally, but this should also work for version 4.16 (they are used in tests):
OS_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "url": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240322.3.1/x86_64/fedora-coreos-39.20240322.3.1-live.x86_64.iso",
        "version": "39.20240322.3.1"
    }
]'

RELEASE_IMAGE='[
    {
        "openshift_version": "4.16",
        "cpu_architecture": "x86_64",
        "cpu_architectures": ["x86_64"],
        "url": "registry.ci.openshift.org/origin/release:4.16",
        "version": "4.16.0-0.okd",
        "default": true,
        "support_level": "beta"
    }
]'
Does the test pipeline by chance have a proper pull secret to: registry.ci.openshift.org?
Yes, it has. If you want to use the public images hosted on registry.ci.openshift.org you can use the PUBLIC_CONTAINER_REGISTRIES environment variable as explained by @matuck and assisted-service will pass the validation.
But if the registry doesn't require a pull secret, shouldn't it already be included by default?

Absolutely, we will add your suggestions:

The registry.ci.openshift.org entry will be added to the default values.
An additional paragraph will be added to the README explaining the PUBLIC_CONTAINER_REGISTRIES variable for custom registries.

Jul 18 '24 14:07 mlorenzofr