karmada-io
add ci for karmada operator
What type of PR is this? /kind feature
What this PR does / why we need it:
The current karmada-operator lacks code merge access control; this PR will introduce:
- A one-click script to install a Karmada instance through the
karmada-operator. - karmada-operator ci
Which issue(s) this PR fixes: Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
:warning: Please install the to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 40.85%. Comparing base (
5e7d836) to head (4f2b2f0).
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@ Coverage Diff @@
## master #5519 +/- ##
==========================================
- Coverage 40.86% 40.85% -0.01%
==========================================
Files 651 651
Lines 55259 55259
==========================================
- Hits 22580 22576 -4
- Misses 31239 31241 +2
- Partials 1440 1442 +2
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 40.85% <ø> (-0.01%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
test report:
# karmada.yaml
apiVersion: operator.karmada.io/v1alpha1
kind: Karmada
metadata:
name: karmada-demo
namespace: test
spec:
crdTarball:
httpSource:
url: https://github.com/karmada-io/karmada/releases/download/v1.11.0/crds.tar.gz
components:
etcd:
local:
imageRepository: registry.k8s.io/etcd
imageTag: 3.5.13-0
replicas: 1
volumeData:
# hostPath:
# type: DirectoryOrCreate
# path: /var/lib/karmada/etcd/karmada-demo
volumeClaim:
metadata:
name: etcd-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
karmadaAPIServer:
imageRepository: registry.k8s.io/kube-apiserver
imageTag: v1.29.6
replicas: 1
serviceType: NodePort
serviceSubnet: 10.96.0.0/12
karmadaAggregatedAPIServer:
imageRepository: docker.io/karmada/karmada-aggregated-apiserver
imageTag: v1.11.0
replicas: 1
karmadaControllerManager:
imageRepository: docker.io/karmada/karmada-controller-manager
imageTag: v1.11.0
replicas: 1
karmadaScheduler:
imageRepository: docker.io/karmada/karmada-scheduler
imageTag: v1.11.0
replicas: 1
karmadaWebhook:
imageRepository: docker.io/karmada/karmada-webhook
imageTag: v1.11.0
replicas: 1
kubeControllerManager:
imageRepository: registry.k8s.io/kube-controller-manager
imageTag: v1.29.6
replicas: 1
karmadaMetricsAdapter:
imageRepository: docker.io/karmada/karmada-metrics-adapter
imageTag: v1.11.0
replicas: 2
# karmadaSearch: # the component `Karmadasearch` is not installed by default, if you need to install it, uncomment it and note the formatting
# imageRepository: docker.io/karmada/karmada-search
# imageTag: v1.8.0
# replicas: 1
karmadaDescheduler: # the component `KarmadaDescheduler` is not installed by default, if you need to install it, uncomment it and note the formatting
imageRepository: docker.io/karmada/karmada-descheduler
imageTag: v1.11.0
replicas: 1
hostCluster:
networking:
dnsDomain: cluster.local
$ kubectl create secret generic my-kubeconfig --from-file=config=$HOME/.kube/karmada.config --namespace karmada-system
$ kubectl apply -f operator/config/deploy/karmada-operator.yaml
$ kubectl apply -f operator/config/crds/
$ kubectl create namespace test
$ kubectl apply -f karmada.yaml
$ kubectl get pods -ntest
NAME READY STATUS RESTARTS AGE
karmada-demo-aggregated-apiserver-7c94956588-kq6m6 1/1 Running 0 49m
karmada-demo-apiserver-9cf845cf4-bkvhk 1/1 Running 0 49m
karmada-demo-controller-manager-55fb77c55-r4mtr 1/1 Running 0 48m
karmada-demo-descheduler-5b6465697d-xbbqh 1/1 Running 0 48m
karmada-demo-etcd-0 1/1 Running 0 49m
karmada-demo-kube-controller-manager-6f8f9b6cf8-zmmsg 1/1 Running 0 48m
karmada-demo-metrics-adapter-686dbd65d4-5tltz 1/1 Running 0 48m
karmada-demo-metrics-adapter-686dbd65d4-v67dd 1/1 Running 0 48m
karmada-demo-scheduler-7d5846d6db-nspv4 1/1 Running 0 48m
karmada-demo-webhook-5bf69bc674-nh7cp 1/1 Running 0 48m
@zhzhuang-zju How to reproduce it? Does #5040 introduce this issue?
Yes, Just follow the steps as described in comments https://github.com/karmada-io/karmada/pull/5519#issuecomment-2347960323 and will reproduce this issue. I think we can add a CI for installation by the karmada-operator, like #4401
I think we can add a CI for installation by the karmada-operator, like https://github.com/karmada-io/karmada/pull/4401
We do need CI, but a simple CI like #4401 cannot cover all cases, we still need E2E cases to cover features like #5242, #5272, #5448.
I will take a look at #4401 first.
I will take a look at https://github.com/karmada-io/karmada/pull/4401 first.
Recently, karmada-operator has added many new features. The installation verification in #4401 might not be sufficient anymore. For example, it does not validate the installation of the component karmada-scheduler-estimator. Collaborating with #5511 could help resolve this issue, so #4401 needs to be updated.
OK, please cc me again #4401 once it's ready.
By the way, I think this PR can help update https://github.com/karmada-io/karmada/blob/master/operator/config/samples/karmada.yaml, so that the bugs can be found more easily.
@chaosi-zju: GitHub didn't allow me to request PR reviews from the following users: PR, now, has, a, conflict, zhzhuang-zju, your.
Note that only karmada-io members and repo collaborators can review this PR, and authors cannot review their own PRs.
In response to this:
/cc @zhzhuang-zju your PR now has a conflict
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
This PR introduces three new scripts:
- hack/deploy-karmada-operator.sh: deploys karmada-operator on the specified cluster
- hack/deploy-karmada-by-operator.sh: deploys karmada instance to a given cluster via karmada-operator.
- hack/local-up-karmada-by-operator.sh: Combining the two scripts above to provide the capability for a one-click installation of a Karmada instance.
Refer to https://github.com/karmada-io/karmada/actions/runs/11270900302/job/31342651001?pr=5519 and local verifiation, using hack/local-up-karmada-by-operator.sh can create a local Karmada environment with basic functionality working properly.
$ karmadactl get cluster
NAME CLUSTER VERSION MODE READY AGE ADOPTION
member1 Karmada v1.31.0 Push True 104m -
member2 Karmada v1.31.0 Push True 104m -
member3 Karmada v1.31.0 Pull True 104m -
$ karmadactl get deployments --operation-scope all
NAME CLUSTER READY UP-TO-DATE AVAILABLE AGE ADOPTION
nginx Karmada 2/2 2 2 68m -
nginx member1 1/1 1 1 67m Y
nginx member2 1/1 1 1 67m Y
However, to fully validate the functionality through e2e tests, it is necessary to wait for the merge of the following PRs:
- https://github.com/karmada-io/karmada/pull/5511 support deploy estimator in the specified namespace
- https://github.com/karmada-io/karmada/pull/5573 avoid installation failures for 'karmada-scheduler-estimator' and 'karmada-interpreter-webhook-example' due to non-uniform secret names.
Therefore, I suggest that the current Operator CI be used to verify the capability of the Karmada Operator to deploy Karmada instances and basic functionalities, such as the successful execution of karmadactl join. E2E tests as well as other tests can be introduced later.
wdyt? @liangyuanpeng @RainbowMango
Since #5586 modified the karmada-operator manifest and added files such as clusterrole, this PR needs to wait for #5586 to be merged first.
/hold
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: RainbowMango
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~.github/workflows/OWNERS~~ [RainbowMango]
- ~~hack/OWNERS~~ [RainbowMango]
- ~~operator/OWNERS~~ [RainbowMango]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment