installation without olm via helm or manifests

[travisghansen]

trafficstars

I have many clusters I would like to roll the operator out to where olm is undesirable...is there not a set of raw manifests or some way to generate a helm chart that would be compatible with installing the operator?

Thanks!

[tristantarrant]

Have you looked at https://github.com/infinispan/infinispan-helm-charts ?

[travisghansen]

Yes, that appears to not be the operator though correct? It’s a single install?

[ryanemerson]

Hi @travisghansen. It's possible to deploy the latest version of the Operator without OLM by logging into your k8s cluster and then calling make deploy. This will also install the cert-manager operator, if it's not already installed, as this is required in order for the Operator's webhooks to work as expected.

[travisghansen]

Awesome! I will look into that as it seems promising. Is it possible to just generate the yaml/manifests (ensuring no deps like cert-manager are included) without applying them? We manage everything via gitops so running that directly against a cluster isn’t really desirable either.

[travisghansen]

Digging into the Makefile to see if I can come up with something useful. I think I generally have a grip on running the kustomize commands to generate some files that could be used..it appears to rbac pieces are configured to handle a single namespace, is it not possible to make the operator function in a cluster-wide fashion? or if I fixup the rbac bit and set WATCH_NAMESPACE to "" in the deployment it should work?

[ryanemerson]

if I fixup the rbac bit and set WATCH_NAMESPACE to "" in the deployment it should work?

That should work :+1:

You can use a kustomize patch for setting the WATCH_NAMESPACE and rbac should just be a case of converting the role and rolebinding to clusterrole and clusterrolebinding respectively.

Is it possible to just generate the yaml/manifests (ensuring no deps like cert-manager are included) without applying them? We manage everything via gitops so running that directly against a cluster isn’t really desirable either.

You could update the deploy target in the Makefile to remove the kubectl apply -f - part of the configuration and also remove the cert-manager dependency, e.g.:

.PHONY: deploy
deploy: manifests kustomize
	cd config/manager && $(KUSTOMIZE) edit set image operator=$(IMG)
	cd config/default && $(KUSTOMIZE) edit set namespace $(DEPLOYMENT_NAMESPACE)
	$(KUSTOMIZE) build config/default

[travisghansen]

Thank you for the pointers, I will see what I can put together and report back.

[travisghansen]

Is there a canonical way to determine which IMG version should be used given a particular tag/checkout of the repo? It appears there isn't an alignment between app versions and operator versions..

[travisghansen]

It's crude but functional I believe, any feedback about outcomes is appreciated:

#!/bin/bash

# TODO: make cluster-wide optional
# TODO: make certs/issuer optional
# TODO: make `Role` / `ClusterRole` merging more sane with yq or similar

set -x
set -e

SCRIPT_DIR="${PWD}"
TMP="/tmp"
CHART="infinispan-operator"
NAME="${CHART}"
CLONE_URL="https://github.com/infinispan/infinispan-operator.git"
VERSION="2.4.3.Final"
IMG="quay.io/infinispan/operator:${VERSION}"
DEPLOYMENT_NAMESPACE="replace-me-operators-zzzzzzzzzzzzzzzzzzz"

# clean up
rm -rf "${TMP}/${CHART}"

# checkout correct version
git clone "${CLONE_URL}" "${TMP}/${CHART}"
cd "${TMP}/${CHART}"
git checkout "${VERSION}"


mkdir -p _chart/crds
mkdir -p _chart/templates

# fixup rbac for cluster-wide
# this takes all the rules from the `Role`  and appends them to the `ClusterRole`
yq -M -e 'select((.kind == "ClusterRole") and (.metadata.name == "manager-role"))' config/rbac/role.yaml > config/rbac/cluster-wide-role.yaml
yq -M -e 'select((.kind == "Role") and (.metadata.name == "manager-role")).rules' config/rbac/role.yaml | sed 's/^/  /' >> config/rbac/cluster-wide-role.yaml
# this effectively deletes the `Role`
mv config/rbac/cluster-wide-role.yaml config/rbac/role.yaml

# remove the RoleBinding
yq -i -e 'del(select(.kind == "RoleBinding"))' config/rbac/role_binding.yaml

cd config/manager && kustomize edit set image operator=${IMG}
cd -
cd config/default && kustomize edit set namespace ${DEPLOYMENT_NAMESPACE}
cd -

cat << 'EOF' >> config/manager/kustomization.yaml

patches:
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: controller-manager
        namespace: system
      spec:
        template:
          spec:
            containers:
            - name: manager
              env:
                - name: WATCH_NAMESPACE
                  $patch: delete
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: controller-manager
        namespace: system
      spec:
        template:
          spec:
            containers:
            - name: manager
              env:
                - name: WATCH_NAMESPACE
                  value: ""
EOF


kustomize build config/default > _chart/templates/operator.yaml
# crds are included in the above
# kustomize build config/crd > _chart/crds/crds.yaml

# move into asset dir
cd "${TMP}/${CHART}/_chart"

# remove docs before we create invalid yaml with gotmpl logic below
yq -i -e 'del(select(.kind == "Namespace"))' templates/operator.yaml

# undesirable given we have leader-election and the main app, leader-election
# does not need cluster-wide access so blanket rewrite is less than ideal
#sed -i 's/kind: Role/kind: ClusterRole/g' templates/operator.yaml
#sed -i 's/kind: RoleBinding/kind: ClusterRoleBinding/g' templates/operator.yaml


#sed -i "s/namespace: ${DEPLOYMENT_NAMESPACE}/namespace: {{ .Release.Namespace }}/g" templates/operator.yaml
sed -i "s/${DEPLOYMENT_NAMESPACE}/{{ .Release.Namespace }}/g" templates/operator.yaml


# prepare Chart.yaml
cat << EOF > Chart.yaml
apiVersion: v2
name: ${CHART}
description: A Helm chart for ${NAME}
type: application
version: ${VERSION%%.Final}
appVersion: "${VERSION%%.Final}"
EOF

cd "${SCRIPT_DIR}"
helm package "${TMP}/${CHART}/_chart"

[ryanemerson]

Is there a canonical way to determine which IMG version should be used given a particular tag/checkout of the repo? It appears there isn't an alignment between app versions and operator versions..

Each operator version supports a range of Infinispan server versions, which is why there's no direct alignment between the two. The operator image always has the format quay.io/infinispan/operator:<tag>, so to replicate a particular Operator release you could do something like:

1. TAG=2.4.3.Final
2. git checkout $TAG
3. IMG=quay.io/infinispan/operator:$TAG

You can view all release operator images at https://quay.io/repository/infinispan/operator?tab=tags

[travisghansen]

Thanks, I found those images later and have the exact logic you have suggested in the script.

The main hacky / tricky thing in the script is I can’t actually simply rename Role to ClusterRole etc as the names collide with the existing Cluster* assets that already exist. So I had to do a sort of crude merging for now which could certainly be improved if I knew yq better..but so far so good. Will test some deployments today but logs are currently clean so I expect that to go well.

[travisghansen]

Testing has gone well, and at a minimum basic functionality works. I haven’t tested all the different resources but an infinispan and cache resource both seem to work.

[ryanemerson]

The intention is to provide raw yaml manifests that will allow the Operator to be installed without OLM. The manifests will assume that cert-manager is installed as this is required in order for the webhooks to work as expected.

Manifests will be distributed as part of each release tag, with our docs updated to describe the process of installing via kubectl apply commands.

[Blarc]

I am also very interested in this feature, since I manage operators with ArgoCD and I need a way to specify raw manifests - either via helm chart or kustomization. Just for example, for keycloak I was able to do something like:

# yaml-language-server: $schema=https://json.schemastore.org/kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  # crds
  # renovate: datasource=github-releases depName=keycloak/keycloak-k8s-resources
  - https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.1.0/kubernetes/keycloaks.k8s.keycloak.org-v1.yml
  # renovate: datasource=github-releases depName=keycloak/keycloak-k8s-resources
  - https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.1.0/kubernetes/keycloakrealmimports.k8s.keycloak.org-v1.yml
  # operator
  # renovate: datasource=github-releases depName=keycloak/keycloak-k8s-resources
  - https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.1.0/kubernetes/kubernetes.yml

[Blarc]

So I've created a repository that follows the same principles as keycloak's k8s resources repository: https://github.com/Blarc/infinispan-k8s-resources.

The operator can now be installed with:

# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - https://raw.githubusercontent.com/Blarc/infinispan-k8s-resources/refs/tags/2.4.12.Final/kubernetes/backups.infinispan.org-v1.yml
  - https://raw.githubusercontent.com/Blarc/infinispan-k8s-resources/refs/tags/2.4.12.Final/kubernetes/batches.infinispan.org-v1.yml
  - https://raw.githubusercontent.com/Blarc/infinispan-k8s-resources/refs/tags/2.4.12.Final/kubernetes/caches.infinispan.org-v1.yml
  - https://raw.githubusercontent.com/Blarc/infinispan-k8s-resources/refs/tags/2.4.12.Final/kubernetes/infinispans.infinispan.org-v1.yml
  - https://raw.githubusercontent.com/Blarc/infinispan-k8s-resources/refs/tags/2.4.12.Final/kubernetes/restores.infinispan.org-v1.yml
  - https://raw.githubusercontent.com/Blarc/infinispan-k8s-resources/refs/tags/2.4.12.Final/kubernetes/kubernetes.yml

[DanSalt]

@travisghansen - was this what you finally settled on? We're in exactly the same situation (need to deploy Operator using GitOps and no access to OLM). We've been keeping a home-grown chart but it's problematic and hard to maintain, so interested to know if your approach was ultimately successful, and whether you can share?

[travisghansen]

Yeah, I’ve been using it successfully for some time now. Let me do a version bump today and see how it goes..

[travisghansen]

Yeah , did a version bump today with the same script and diff looks sane in argocd.