console icon indicating copy to clipboard operation
console copied to clipboard

CONSOLE-3241: Use cluster proxy for managed cluster API server requests

Open TheRealJon opened this issue 2 years ago • 38 comments

Update server to remove original multicluster proxying implementation and instead proxy managed cluster requests to the MCE cluster proxy service

TheRealJon avatar Aug 19 '22 18:08 TheRealJon

/retest

TheRealJon avatar Aug 23 '22 15:08 TheRealJon

QE Approver /assign @yapei

Docs Approver: /assign @opayne1

PX Approver: /assign @RickJWagner

TheRealJon avatar Aug 23 '22 15:08 TheRealJon

/label px-approved

RickJWagner avatar Aug 23 '22 18:08 RickJWagner

/retest

CI job timed out

TheRealJon avatar Aug 25 '22 19:08 TheRealJon

/label docs-approved

opayne1 avatar Aug 29 '22 19:08 opayne1

/retest

TheRealJon avatar Aug 31 '22 17:08 TheRealJon

/retest

Panic in openshift operator controller manager pod. Reported in Slack.

TheRealJon avatar Sep 01 '22 14:09 TheRealJon

/retest

Cypress verification failed

TheRealJon avatar Sep 01 '22 20:09 TheRealJon

@TheRealJon wheh navigating through manged cluster pages, it seems managed cluster pages can not be loaded successfully and I can see some errors in console log

I0920 10:01:32.237240 1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/uscluster/apis/project.openshift.io/v1/projects`
I0920 10:01:32.500360 1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/uscluster/apis/apps/v1/namespaces/openshift-console/deployments`
I0920 10:01:32.503337 1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/uscluster/apis/authorization.k8s.io/v1/selfsubjectaccessreviews`
2022/09/20 10:01:34 http: proxy error: context canceled
I0920 10:03:03.027486 1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/uscluster/apis/apps/v1/namespaces/openshift-console/deployments`
2022/09/20 10:03:03 http: proxy error: context canceled
I0920 10:03:03.529456 1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/uscluster/apis/project.openshift.io/v1/projects`
2022/09/20 10:03:03 http: proxy error: context canceled
Screen Shot 2022-09-20 at 18 02 52

yapei avatar Sep 20 '22 10:09 yapei

@yapei This is likely because your managed cluster has not been imported through ACM or the cluster name in ACM is different from the cluster name in the managed cluster config consumed by the console back end. So, in order for multicluster to work when running bridge locally, you'll have to follow these steps:

  • Provision or use existing cluster with ACM installed
  • Provision or use an existing second cluster as the managed cluster
  • Follow the instructions in the contrib/multicluster-environment.sh script to init your local multicluster environment variables
  • Import the managed cluster to the ACM cluster, using the same cluster name found in the BRIDGE_MANAGED_CLUSTERS env var
  • Set BRIDGE_K8S_MODE_OFF_CLUSTER_MANAGED_CLUSTER_PROXY env var to the public cluster proxy route (on ACM cluster oc get route cluster-proxy-addon-user -n multicluster-engine)
  • Build the backend with the changes in this PR and run bridge

You should then be able to visit localhost:9000 authenticate as kubeadmin on the hub cluster, use the cluster dropdown to select the managed cluster, authenticate again with the kubeadmin creds for that cluster, then test to make sure requests are being proxied appropriately. I was able to follow these steps myself and get this working. I did run into the exact issue above before realizing that I had not imported the managed cluster through ACM.

TheRealJon avatar Sep 20 '22 20:09 TheRealJon

tested again today

  • Enable multi cluster on Hub cluster
  • Import one managed cluster with name 'test-cluster-one'
  • Visit 'test-cluster-one' from All Clusters dropdown, check console pod logs and we can see that console backend makes managed cluster API server requests through the cluster proxy that the MCE operator provides
I0921 13:34:38.657877       1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/test-cluster-one/apis/project.openshift.io/v1/projects/yapei-test-one-project`
I0921 13:34:39.231052       1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/test-cluster-one/api/v1/namespaces/yapei-test-one-project/resourcequotas`
I0921 13:34:39.569416       1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/test-cluster-one/apis/quota.openshift.io/v1/namespaces/yapei-test-one-project/appliedclusterresourcequotas`
I0921 13:34:39.573411       1 proxy.go:105] PROXY: `https://cluster-proxy-addon-user.multicluster-engine.svc:9092/test-cluster-one/apis/apps/v1/namespaces/yapei-test-one-project/deployments`

/label qe-approved

yapei avatar Sep 21 '22 13:09 yapei

/retest

TheRealJon avatar Sep 22 '22 18:09 TheRealJon

@TheRealJon looks like the PR needs rebase due to conflicts

jhadvig avatar Sep 27 '22 08:09 jhadvig

/label tide/merge-method-squash

TheRealJon avatar Sep 28 '22 16:09 TheRealJon

/retest-required

Remaining retests: 0 against base HEAD 98523601dd716b8b479e3b0112eaaeda83bf39ee and 2 for PR HEAD f24e592e1b067666f94b5fe2ccdb2f3316fbbceb in total

openshift-ci-robot avatar Sep 29 '22 07:09 openshift-ci-robot

/retest

TheRealJon avatar Oct 03 '22 14:10 TheRealJon

/retest-required

Remaining retests: 0 against base HEAD 0780f4cd7b3c60c052c3764fe15a25b1b884b6e8 and 1 for PR HEAD f24e592e1b067666f94b5fe2ccdb2f3316fbbceb in total

openshift-ci-robot avatar Oct 04 '22 16:10 openshift-ci-robot

/retest-required

Remaining retests: 0 against base HEAD 3ebee34394e1a1498dd9f6e090c29b8af11e8028 and 2 for PR HEAD 4fa255cbf1ab8683dc235d5d4307165421bf42a8 in total

openshift-ci-robot avatar Oct 05 '22 19:10 openshift-ci-robot

/test e2e-gcp-console

jhadvig avatar Oct 06 '22 09:10 jhadvig

/test e2e-gcp-console

jhadvig avatar Oct 06 '22 15:10 jhadvig

/retest-required

Remaining retests: 0 against base HEAD 887b2fccaa5e9ce15be9b6fc6f1ab67586cf4d88 and 1 for PR HEAD 4fa255cbf1ab8683dc235d5d4307165421bf42a8 in total

openshift-ci-robot avatar Oct 06 '22 19:10 openshift-ci-robot

/retest-required

Remaining retests: 0 against base HEAD 747b86657bda31424a6c2fde82ee8a25e569bbbc and 0 for PR HEAD 4fa255cbf1ab8683dc235d5d4307165421bf42a8 in total

openshift-ci-robot avatar Oct 07 '22 09:10 openshift-ci-robot

/hold

Revision 4fa255cbf1ab8683dc235d5d4307165421bf42a8 was retested 3 times: holding

openshift-ci-robot avatar Oct 07 '22 13:10 openshift-ci-robot

/retest

TheRealJon avatar Oct 07 '22 17:10 TheRealJon

/retest

TheRealJon avatar Oct 11 '22 14:10 TheRealJon

/retest

TheRealJon avatar Oct 12 '22 15:10 TheRealJon

/hold cancel

jhadvig avatar Oct 13 '22 10:10 jhadvig

/retest

jhadvig avatar Oct 13 '22 10:10 jhadvig

/retest

TheRealJon avatar Oct 13 '22 14:10 TheRealJon

/retest-required

Remaining retests: 0 against base HEAD 5ba18580676a25e4304df78253aad6a9832d4d56 and 2 for PR HEAD 4fa255cbf1ab8683dc235d5d4307165421bf42a8 in total

openshift-ci-robot avatar Oct 13 '22 15:10 openshift-ci-robot