glooctl install fails inside kubernetes container where KUBERNETES_SERVICE_PORT environment variable is set

[mallikkml]

Gloo Edge Version

1.9.x

Kubernetes Version

1.21.x

Describe the bug

We use kubernetes in our devops pipeline, when we try installing gloo on a kubernetes container it fails with following error:

Warning: Could not determine gloo server versions (is Gloo running outside of kubernetes?): deployments.apps is forbidden: User "XXXXXXXXXXXX" cannot list resource "deployments" in API group "apps" in the namespace "gloo-system"

After lot of investigation found that, this is happening due to a environment variable (KUBERNETES_SERVICE_PORT=443) set inside kubernetes container.

After unset KUBERNETES_SERVICE_PORT environment variable, gloo install works as expected.

PS: This issue happens only inside a kubernetes container, works fine on a standard VM.

Steps to reproduce the bug

Try installing glooctl install gateway inside a kubernetes container.

Even version command shows warning when KUBERNETES_SERVICE_PORT environment variable is set.

# echo $KUBERNETES_SERVICE_PORT 443

# glooctl version Warning: Could not determine gloo server versions (is Gloo running outside of kubernetes?): deployments.apps is forbidden: User "system:serviceaccount:opsplane:default" cannot list resource "deployments" in API group "apps" in the namespace "gloo-system" Client: {"version":"1.9.10"} Server: version undefined, could not find any version of gloo running

# unset KUBERNETES_SERVICE_PORT

# glooctl version Client: {"version":"1.9.10"} Server: {"type":"Gateway","kubernetes":{"containers":[{"Tag":"1.9.10","Name":"discovery","Registry":"quay.io/solo-io"},{"Tag":"1.9.10","Name":"gateway","Registry":"quay.io/solo-io"},{"Tag":"1.9.10","Name":"gloo-envoy-wrapper","Registry":"quay.io/solo-io"},{"Tag":"1.9.10","Name":"sds","Registry":"quay.io/solo-io"},{"Tag":"1.10.6","Name":"proxyv2","Registry":"docker.io/istio"},{"Tag":"1.9.10","Name":"gloo","Registry":"quay.io/solo-io"}],"namespace":"gloo-system"}}

Expected Behavior

glooctl install gateway should work inside a kubernetes container

Additional Context

No response

[mallikkml]

Even glooct check fails with below error:

# glooctl check Warning: Could not determine gloo server versions (is Gloo running outside of kubernetes?): deployments.apps is forbidden: User "XXXXXXXXXXXXX" cannot list resource "deployments" in API group "apps" in the namespace "gloo-system" Error: 1 error occurred: * Could not communicate with kubernetes cluster: namespaces "gloo-system" is forbidden: User "XXXXXXXXXX" cannot get resource "namespaces" in API group "" in the namespace "gloo-system"