Questions (issues?) for a deployment on GKE

[loicalbertin]

Please provide an in-depth description of the question you have:

:wave: Hello, I'm in the process of evaluating Karmada. After playing a bit with it on a local setup with Kind, I currently try to deploy it on a GKE cluster which is one of my targets.

I try to use hack/remote-up-karmada.sh to install Karmada on a an existing GKE cluster but I'm facing several issues.

Issue 1: First when deploying the karmada api server I get a insufficient quota to match these scopes: [{PriorityClass In [system-node-critical system-cluster-critical]}] error.

I'm not very familiar with those quota things so I may have miss something or did it wrong but I managed to workaround this by pre-creating the karmada-system namespace and tweaked resource quotas on it:

apiVersion: v1
kind: Namespace
metadata:
  name: karmada-system

---

apiVersion: v1
kind: ResourceQuota
metadata:
  name: karmada-node-resource-quota
  namespace: karmada-system
spec:
  hard:
    pods: "10000"
  scopeSelector:
    matchExpressions:
      - scopeName: PriorityClass
        operator: In
        values:
          - system-node-critical

---

apiVersion: v1
kind: ResourceQuota
metadata:
  name: karmada-cluster-resource-quota
  namespace: karmada-system
spec:
  hard:
    pods: "10000"
  scopeSelector:
    matchExpressions:
      - operator: In
        scopeName: PriorityClass
        values:
          - system-cluster-critical

Is this the correct way to manage this quotas error?

At least with this I'm able to deploy Karmada on GKE.

But then I have 2 other issues with the hack/remote-up-karmada.sh.

I expose Karmada using a GKE loadbalancer without DNS.

Issue 2: First the script is not able to properly retrieve the Karmada Service IP:

https://github.com/karmada-io/karmada/blob/4504030c057340f1f63582e6386c0c99746f38aa/hack/deploy-karmada.sh#L203

In my understanding the first parameter of util::wait_service_external_ip (the context to use to retrieve the LB service IP) should be ${HOST_CLUSTER_NAME} instead of karmada-apiserver.

      if util::wait_service_external_ip "${HOST_CLUSTER_NAME}" "karmada-apiserver" "${KARMADA_SYSTEM_NAMESPACE}"; then

Issue 3: Then I'm facing a final issue within the util::wait_service_external_ip function.

https://github.com/karmada-io/karmada/blob/4504030c057340f1f63582e6386c0c99746f38aa/hack/util.sh#L524-L529

I did not configure a DNS for this LB. But in this case, on GKE, the value of external_host is not empty but equal <no value>. This will cause the script to try to resolve <no value> and that's definitely not good.

I propose to add a check on <no value> like this:

    external_host=$(kubectl --context="$context_name" get service "${service_name}" -n "${namespace}" --template="{{range .status.loadBalancer.ingress}}{{.hostname}} {{end}}" | xargs)
    external_ip=$(kubectl --context="$context_name" get service "${service_name}" -n "${namespace}" --template="{{range .status.loadBalancer.ingress}}{{.ip}} {{end}}" | xargs)
    set -e
    if [[ ! -z "$external_host" ]] && [[ "${external_host}" != "<no value>" ]]; then # Compatibility with hostname, such as AWS
      external_ip=$external_host
    fi

I'm sorry this is a very long report with different issues but as I'm not sure to do everything right I prefer to open it as a single question. Feel free to ask me to split them into different issues.

Finally, kindly note that I will be happy to contribute PRs if issues are confirmed.

Environment:

• Karmada version: 1.3.0
• Kubernetes version: 1.22
• Others: GKE cluster

[RainbowMango]

/assign I'm in the middle of something. Will get back to you later. Glad to see you again @loicalbertin .

[loicalbertin]

I'm in the middle of something. Will get back to you later.

No rush, with above workarounds I'm able to continue my evaluation of Karmada.

Thanks & have a nice day!