operator-sdk olm command cannot work with olm v0.19.0 in OpenShift

[dove-young]

Bug Report

What did you do?

• all commands do not work
  • operator-sdk olm status
  • operator-sdk olm install
  • operator-sdk olm uninstall

[root@st2 ~]# operator-sdk olm install --verbose
DEBU[0000] Debug logging is set
I0920 02:11:13.496934  435849 request.go:655] Throttling request took 1.047692939s, request: GET:https://api.bmy-host.com:6443/apis/cache.example.com/v1alpha1?timeout=32s
INFO[0002] Fetching CRDs for version "latest"
INFO[0002] Fetching resources for resolved version "latest"
FATA[0007] Failed to install OLM version "latest": detected existing OLM resources: OLM must be completely uninstalled before installation
[root@st2 ~]#
[root@st2 ~]# operator-sdk olm uninstall --verbose
DEBU[0000] Debug logging is set
I0920 02:11:28.447453  435877 request.go:655] Throttling request took 1.047298103s, request: GET:https://api.bmy-host.com:6443/apis/acme.cert-manager.io/v1?timeout=32s
FATA[0002] Failed to uninstall OLM: error getting installed OLM version (set --version to override the default version): no existing installation found
[root@st2 ~]#

• here is an olm v0.19.0 found

[root@st2 ~]# oc get csv,deploy -n openshift-operator-lifecycle-manager
NAME                                                       DISPLAY          VERSION   REPLACES   PHASE
clusterserviceversion.operators.coreos.com/packageserver   Package Server   0.19.0               Succeeded

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/catalog-operator         1/1     1            1           24h
deployment.apps/olm-operator             1/1     1            1           24h
deployment.apps/package-server-manager   1/1     1            1           24h
deployment.apps/packageserver            2/2     2            2           11m
[root@st2 ~]#

• retry with specific version and namespace

[root@st2 ~]# operator-sdk olm status --verbose --olm-namespace openshift-operator-lifecycle-manager --version 0.19.0
DEBU[0000] Debug logging is set
I0920 02:14:17.925849  435921 request.go:655] Throttling request took 1.046292814s, request: GET:https://api.bmy-host.com:6443/apis/policy/v1?timeout=32s
INFO[0002] Fetching CRDs for version "0.19.0"
INFO[0002] Fetching resources for resolved version "v0.19.0"
FATA[0004] Failed to get OLM status: failed to get resources: failed to fetch CRDs: request failed: failed GET 'https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.19.0/crds.yaml': unexpected status code 404, expected 200; manifests may not exist for this OLM release,please check https://github.com/operator-framework/operator-lifecycle-manager/releases for olm.yaml and crds.yaml
[root@st2 ~]#
[root@st2 ~]# operator-sdk olm uninstall --verbose --olm-namespace openshift-operator-lifecycle-manager --version 0.19.0
DEBU[0000] Debug logging is set
I0920 02:14:33.988904  435951 request.go:655] Throttling request took 1.046020435s, request: GET:https://api.bmy-host.com:6443/apis/acme.cert-manager.io/v1?timeout=32s
INFO[0002] Fetching CRDs for version "0.19.0"
INFO[0002] Fetching resources for resolved version "v0.19.0"
FATA[0004] Failed to uninstall OLM: failed to get resources: failed to fetch CRDs: request failed: failed GET 'https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.19.0/crds.yaml': unexpected status code 404, expected 200; manifests may not exist for this OLM release,please check https://github.com/operator-framework/operator-lifecycle-manager/releases for olm.yaml and crds.yaml
[root@st2 ~]#

Kubernetes cluster type:

• OpenShift 4.10.18

$ operator-sdk version

operator-sdk version: "v1.8.0-ocp", commit: "bf96b7604eadb82f173df1c60e6999c2d17695fb", kubernetes version: "v1.20.2", go version: "go1.16.6", GOOS: "linux", GOARCH: "amd64"

$ go version (if language is Go)

go version go1.18.2 linux/amd64 $ kubectl version

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:26:19Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5+3afdacb", GitCommit:"3c28e7a79b58e78b4c1dc1ab7e5f6c6c2d3aedd3", GitTreeState:"clean", BuildDate:"2022-05-10T16:30:48Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}

Possible Solution

Additional context

[dove-young]

Found a known issue https://bugzilla.redhat.com/show_bug.cgi?id=2021039

[joelanford]

A few things:

OLM is built into OpenShift, so there should be no need to use the operator-sdk olm tooling with an OpenShift cluster.

• operator-sdk olm install would be expected to fail because OLM will already be installed on OpenShift
• operator-sdk olm status might work, but it fetches upstream manifests to know what to check status about. Those upstream manifests do not necessarily align with what exists downstream
• operator-sdk olm uninstall might work temporarily. But even if it succeeds, OpenShift will likely re-install it.

If you're using operator-sdk olm upstream, there was an issue specifically with the v0.19.0 release that caused the release manifests not to be published. Since operator-sdk olm commands rely on OLM's github releases, it means you won't be able to use those commands with v0.19.0.

[tlwu2013]

+1 on @joelanford's reply for this issue. Close this issue and feel free to leave comments if any additional question, thanks!