kubebox icon indicating copy to clipboard operation
kubebox copied to clipboard

Improved message when not authorised to list namespaces

Open vgibilmanno opened this issue 5 years ago • 63 comments

I can't use kubebox when I only have access to a namespace in a cluster. It would be awesome if I could use kubebox just for the namespace I have access to, ignoring everything outside the namespace.

vgibilmanno avatar Nov 27 '19 08:11 vgibilmanno

You should be able to use Kubebox just for the namespaces you're granted access to. What error do you face?

astefanutti avatar Nov 27 '19 09:11 astefanutti

The login fails with "Authentication failed" and I'm being asked for an username and a password. The cluster runs in AWS and I have the AWS credentials set. Because of that I don't really have a username or password to set. Kubectl can be executed perfectly fine.

vgibilmanno avatar Nov 27 '19 09:11 vgibilmanno

AWS EKS is supposed to be working :(

There have been some issues that may provide some context: #32 #60.

What version of Kubebox do you use? Could you provide the content of your kubeconfig file?

In the meantime, you can run:

$ aws eks get-token --cluster-name [my-cluster] 

To get a token to authenticate with Kubebox.

astefanutti avatar Nov 27 '19 10:11 astefanutti

Of course :) that's the content of the kubeconfig. I hope it's ok, that I replaced personal information with placeholders. But I didn't change the structure of the content.

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://OMITTED.amazonaws.com
  name: SOMENAME
contexts:
- context:
    cluster: SOMENAME
    user: SOMENAME
  name: SOMENAME
current-context: SOMENAME
kind: Config
preferences: {}
users:
- name: SOMENAME
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - token
      - -i
      - SOMEOTHERNAME
      - -r
      - arn:aws:iam::SOMEROLEID:role/SOMEROLE
      command: aws-iam-authenticator
      env: null

I'm using kubebox v0.6.1

How can I replace the token in the login screen? I'm too stupid right now sry

vgibilmanno avatar Nov 27 '19 10:11 vgibilmanno

Thanks for the details. Nothing obvious so we'll have to try reproducing.

There is a token field in the login window, below the username and password fields. You can paste the token retrieve from AWS CLI.

astefanutti avatar Nov 27 '19 13:11 astefanutti

@vgibilmanno perhaps some useful info would be to run the command locally yourself:

aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE and maybe show us the output ? Of course censuring private data. Thanks !

johnpoth avatar Nov 27 '19 15:11 johnpoth

@johnpoth I get the following

{
    "kind": "ExecCredential",
    "apiVersion": "client.authentication.k8s.io/v1alpha1",
    "spec": {},
    "status": {
        "expirationTimestamp": "2019-11-27T15:51:10Z",
        "token": "SOMETOKEN"
    }
}

vgibilmanno avatar Nov 27 '19 15:11 vgibilmanno

@johnpoth could it be that the provideAuth method does not get a first token with refresh_token when there is no token information?

astefanutti avatar Nov 27 '19 15:11 astefanutti

@johnpoth the refresh_token method returns false when there is no expiration date:

console.log((undefined - Date.now()) < 5000)
> false

So we may have to add a condition to check whether a token exists:

https://github.com/astefanutti/kubebox/blob/f8cb4db7688be5316d7cf87ce3b677911f18755b/lib/auth/exec.js#L14

WDYT?

astefanutti avatar Nov 27 '19 15:11 astefanutti

Hi @astefanutti so the expired is always set to now:

ExecAuthProvider ... this.expiry = Date.now();

So the token is always refreshed even if it's still valid on the first run. Therefore I think the problem lies elsewhere but I may be wrong...

@vgibilmanno have you tried pasting the TOKEN you got from running the command into the kubebox login screen ? This should help narrow things down a bit... thanks

johnpoth avatar Nov 27 '19 16:11 johnpoth

@johnpoth I've just realised the expiry is set there:

https://github.com/astefanutti/kubebox/blob/f8cb4db7688be5316d7cf87ce3b677911f18755b/lib/config/user.js#L143

It seems it'd be easier to reason about if it would not be initialised... Could it be the login sequence takes less than 5ms so that the check fails?

astefanutti avatar Nov 27 '19 16:11 astefanutti

@johnpoth I can't paste the token in the login screen. The paste command is not working and I would have to type in the whole token manually. I'm using alpine linux v3.10 in case that's relevant.

vgibilmanno avatar Nov 27 '19 16:11 vgibilmanno

@astefanutti I hadn't thought about that... judging from the feedback #60 I'm thinking it might be something different ? That being said I don't see what could be causing this from the Kubebox Exec plugin perspective...

It may be interesting for @vgibilmanno to use the Token directly and see if Kubebox can connect to the cluster ?

johnpoth avatar Nov 27 '19 17:11 johnpoth

@vgibilmanno could you update your kubeconfig manually and add a token field with the copied token returned by AWS CLI, e.g.:

users:
- name: SOMENAME
  user:
    token: PASTE THERE

Then run Kubebox? The token should be picked-up by Kubebox.

astefanutti avatar Nov 27 '19 17:11 astefanutti

@astefanutti Doesn't work. The token displayed in the login screen is not the same as I entered in the kubeconfig. My config file is located under /root/.kube/config

vgibilmanno avatar Nov 28 '19 08:11 vgibilmanno

@vgibilmanno the token may just be truncated in the UI. Have you changed the KUBECONFIG environment variable or does you /root/.kube/config file contain multiple contexts?

astefanutti avatar Nov 28 '19 08:11 astefanutti

@vgibilmanno Can you also just try a second time connecting, by just pressing enter when the login widget displays with the error message?

astefanutti avatar Nov 28 '19 08:11 astefanutti

@astefanutti I tried, but it doesn't work. I'm getting the "authentication failed" message. The token doesn't seem to be the one I define in the kubeconfig. I looked for some substring (last 5 characters) but it doesn't match

vgibilmanno avatar Nov 28 '19 15:11 vgibilmanno

@vgibilmanno thanks for the feedback. Would you mind checking you have only one context defined in your kubeconfig file?

astefanutti avatar Nov 28 '19 15:11 astefanutti

@astefanutti There is only one context defined. It's exactly like the one I posted before with the addition, that tere is a token value inside the user field.

vgibilmanno avatar Nov 28 '19 15:11 vgibilmanno

@vgibilmanno thanks. May I ask, is there a token filled in the login widget when using the original kubeconfig, before adding the token manually to it? Finally could you double check the cluster URL displayed in the login widget is the correct one?

astefanutti avatar Nov 28 '19 16:11 astefanutti

@astefanutti When I use the original kubeconfig, there is a token filled in the login widget. The cluster URL displayed in the login widget is correct.

vgibilmanno avatar Nov 28 '19 16:11 vgibilmanno

@vgibilmanno thanks again. Just one last test if you don't mind (sorry). Could you re-try the same test of adding the token manually to your kubeconfig file AND removing the whole exec section? Then check the token is the same as the one displayed in the login widget?

astefanutti avatar Nov 28 '19 16:11 astefanutti

@astefanutti Now the correct token is displayed, but the authentication fails. I tried pressing enter/log-in multiple times in series

vgibilmanno avatar Nov 28 '19 16:11 vgibilmanno

OK thanks a lot! So there is a request Kubebox does and that's not authorised for your user. Let me check and come back to you. Thanks again!

astefanutti avatar Nov 28 '19 16:11 astefanutti

@vgibilmanno if you could try executing the following commands and provide the results:

$ curl -k -v -H "Authorization: Bearer <TOKEN>" https://OMITTED.amazonaws.com/

$ curl -k -v -H "Authorization: Bearer <TOKEN>" https://OMITTED.amazonaws.com/api

$ curl -k -v -H "Authorization: Bearer <TOKEN>" https://OMITTED.amazonaws.com/api/v1/namespaces

astefanutti avatar Nov 28 '19 17:11 astefanutti

@astefanutti All 3 requests fail with the following message

*   Trying SOMEIP:443...
* TCP_NODELAY set
* Connected to SOMESERVER.amazonaws.com (SOMEIP) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=kube-apiserver
*  start date: Jan 23 14:39:32 2019 GMT
*  expire date: Nov 17 23:02:45 2020 GMT
*  issuer: CN=kubernetes
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56318da783e0)
> GET / HTTP/2
> Host: SOMEHOST.amazonaws.com
> User-Agent: curl/7.66.0
> Accept: */*
> Authorization: Bearer SOMETOKEN
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 401
< audit-id: 6562ed5f-ef44-4d1b-8210-5c98f0c3cea5
< content-type: application/json
< content-length: 165
< date: Thu, 28 Nov 2019 17:23:36 GMT
<
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "Unauthorized",
  "reason": "Unauthorized",
  "code": 401
* Connection #0 to host SOMEHOST.amazonaws.com left intact

vgibilmanno avatar Nov 28 '19 17:11 vgibilmanno

@vgibilmanno thanks, just to be sure, could you try with a fresh token that you've just retrieve with aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE?

astefanutti avatar Nov 28 '19 18:11 astefanutti

@astefanutti I tried but it still fails. I tried the following url too (fails too) curl -k -v -H "Authorization: Bearer <TOKEN>" https://OMITTED.amazonaws.com/api/v1/namespaces/SOMENAMESPACE/pods

vgibilmanno avatar Nov 28 '19 18:11 vgibilmanno

@vgibilmanno thanks, this is very surprising. Could you try executing the following command and provide the output:

$ kubectl --v=9 get --raw /api/v1/namespaces/SOMENAMESPACE/pods

astefanutti avatar Nov 28 '19 18:11 astefanutti

@astefanutti This command works. I get the following:

I1129 07:56:44.179968     155 loader.go:359] Config loaded from file /root/.kube/config
I1129 07:56:44.180408     155 round_trippers.go:419] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.13.8 (linux/amd64) kubernetes/0c6d31a" 'https://OMITTED.amazonaws.com/api/v1/namespaces/SOMENAMESPACE/pods'
I1129 07:56:45.749509     155 round_trippers.go:438] GET https://OMITTED.amazonaws.com/api/v1/namespaces/SOMENAMESPACE/pods 200 OK in 1569 milliseconds
I1129 07:56:45.749809     155 round_trippers.go:444] Response Headers:
I1129 07:56:45.749821     155 round_trippers.go:447]     Date: Fri, 29 Nov 2019 07:56:45 GMT
I1129 07:56:45.749832     155 round_trippers.go:447]     Audit-Id: f15acf96-5aa7-4f65-a60c-563a94c2bb81
I1129 07:56:45.749840     155 round_trippers.go:447]     Content-Type: application/json
{HUGEJSONCONTENT}

vgibilmanno avatar Nov 29 '19 07:11 vgibilmanno

@vgibilmanno thanks. Did you remove -H "Authorization: Bearer SOMETOKEN" from the curl command in the output?

Could you just run:

$ curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.13.8 (linux/amd64) kubernetes/0c6d31a" 'https://OMITTED.amazonaws.com/api/v1/namespaces/SOMENAMESPACE/pods'

astefanutti avatar Nov 29 '19 08:11 astefanutti

@astefanutti

*   Trying SOMEIP:443...
* TCP_NODELAY set
* Connected to OMITTED.amazonaws.com (SOMEIP) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=kube-apiserver
*  start date: Jan 23 14:39:32 2019 GMT
*  expire date: Nov 17 22:59:51 2020 GMT
*  issuer: CN=kubernetes
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x561242df8320)
> GET /api/v1/namespaces/SOMENAMESPACE/pods HTTP/2
> Host: OMITTED.amazonaws.com
> Accept: application/json, */*
> User-Agent: kubectl/v1.13.8 (linux/amd64) kubernetes/0c6d31a
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 403
< audit-id: e42d6da9-82ef-4882-93df-532e4655384f
< content-type: application/json
< x-content-type-options: nosniff
< content-length: 271
< date: Fri, 29 Nov 2019 08:10:38 GMT
<
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \"system:anonymous\" cannot list resource \"pods\" in API group \"\" in the namespace \"SOMENAMESPACE\"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
* Connection #0 to host OMITTED.amazonaws.com left intact

vgibilmanno avatar Nov 29 '19 08:11 vgibilmanno

@vgibilmanno thanks, it seems logical. I'm surprised there is no -H "Authorization: Bearer SOMETOKEN" output for the curl command when you run kubectl --v=9 get --raw /api/v1/namespaces/SOMENAMESPACE/pods.

I'm running out of idea. Just to be sure, what's the output of:

$ echo $KUBECONFIG

astefanutti avatar Nov 29 '19 08:11 astefanutti

@astefanutti It's strange, that the curl command works inside of kubectl but the same curl command does not work outside of it.

$ echo $KUBECONFIG

returns nothing

vgibilmanno avatar Nov 29 '19 08:11 vgibilmanno

@vgibilmanno yes, there is something else that we are missing there.

astefanutti avatar Nov 29 '19 08:11 astefanutti

It may be that certificate-authority-data: DATA+OMITTED from @vgibilmanno's Kubernetes config isn't taken into account when executing the curl command, if it's not present in /etc/ssl/certs/ca-certificates.crt - so adding it will micmic more closely what kubetcl does. That being said I don't think it is the root cause as kubebox takes the ca into account

ps: it is normal that the Authorization Http header isn't logged, see https://github.com/kubernetes/client-go/blob/8d0e6f1b7b780ef8c8f81800d504509cd1f68e9f/transport/round_trippers.go#L392

johnpoth avatar Nov 29 '19 09:11 johnpoth

@johnpoth I see the Authorization header set when I run the command locally. Not sure about how the CA data could interfere.

astefanutti avatar Nov 29 '19 09:11 astefanutti

401 means that the provided token cannot be authenticated to a known identity. It's like the token used is not correct. Maybe there is an encoding issue when Kubebox reads from stdout and parse the response returned from the executed AWS CLI command.

astefanutti avatar Nov 29 '19 09:11 astefanutti

@vgibilmanno by any chance, would you be able to clone this repository and run Kubebox from source?

astefanutti avatar Nov 29 '19 09:11 astefanutti

@astefanutti Yes I'm able to 👍

vgibilmanno avatar Nov 29 '19 09:11 vgibilmanno

@vgibilmanno awesome! You'll need Node.js installed, so if you can:

  • Clone this repository: git clone https://github.com/astefanutti/kubebox.git && cd kubebox
  • Then install dependencies: npm i
  • Add require('fs').writeFileSync('reponse.json', response); above the following line: https://github.com/astefanutti/kubebox/blob/01b28a2dc33831c635f4577e691c3a2bc977a6a0/lib/auth/exec.js#L52
  • Then run node index.js
  • Finally cat response.json

Thanks again!

astefanutti avatar Nov 29 '19 10:11 astefanutti

@astefanutti I get the following in the response.json

{
    "kind": "ExecCredential",
    "apiVersion": "client.authentication.k8s.io/v1alpha1",
    "spec": {},
    "status": {
        "expirationTimestamp": "2019-11-29T11:07:51Z",
        "token": "SOMETOKEN"
    }
}

vgibilmanno avatar Nov 29 '19 10:11 vgibilmanno

@vgibilmanno thanks. Could you try running:

$ aws-iam-authenticator verify -i <clusterid> -t SOMETOKEN

by taking SOMETOKEN from response.json.

astefanutti avatar Nov 29 '19 11:11 astefanutti

@astefanutti When I enter the cluster name in I get the following:

could not verify token: sts getCallerIdentity failed: error from AWS (expected 200, got 403)

When I enter the cluster address in I get the following:

could not verify token: input token was not properly formatted: non-whitelisted query parameter "X-Amz-SignedHeade"

vgibilmanno avatar Nov 29 '19 11:11 vgibilmanno

@vgibilmanno could you try:

$ aws-iam-authenticator verify -i SOMEOTHERNAME -t SOMETOKEN

and provide the output of:

$ env

I suspect there are some environment variables that come into the play.

astefanutti avatar Nov 29 '19 12:11 astefanutti

@astefanutti So the command again fails with:

could not verify token: sts getCallerIdentity failed: error from AWS (expected 200, got 403)

And have the following entries in env:

HOSTNAME=e39ab07991c2
PYTHON_PIP_VERSION=19.3.1
SHLVL=2
HOME=/root
GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/ffe826207a010164265d9cc807978e3604d18ca0/get-pip.py
TERM=xterm
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LANG=C.UTF-8
PYTHON_VERSION=3.8.0
PWD=/
PYTHON_GET_PIP_SHA256=b86f36cc4345ae87bfd4f10ef6b2dbfa7a872fbff70608a1e43944d283fd0eee

vgibilmanno avatar Nov 29 '19 13:11 vgibilmanno

Damn, would you mind now trying:

$ aws-iam-authenticator verify -i SOMEOTHERNAME -t `aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE | jq -r .status.token`

(if you don't have jq, you can copy the token manually)

astefanutti avatar Nov 29 '19 13:11 astefanutti

@astefanutti &{ARN:arn:aws:sts::SOMEROLEID:assumed-role/SOMEROLE/SOMESESSIONNAME CanonicalARN:arn:aws:iam::SOMEROLEID:role/SOMEROLE AccountID:SOMEACCOUNTID UserID:SOMEUSERID SessionName:SOMESESSIONNAME}

vgibilmanno avatar Nov 29 '19 13:11 vgibilmanno

@vgibilmanno could you check any difference between the token from response.json and the one returned by aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE?

astefanutti avatar Nov 29 '19 13:11 astefanutti

@astefanutti The tokens are different. The first 190 characters are identical. The next some thousand characters are different.

vgibilmanno avatar Nov 29 '19 13:11 vgibilmanno

I don't understand why it does not work when a token generated with AWS CLI, that verifies correctly, is copied into the kubeconfig file and used by Kubebox. And why a token generated within Kubebox does not verify!

I've stumbled upon kubernetes-sigs/aws-iam-authenticator#157. I'm not sure if that applies to that issue to some extent.

astefanutti avatar Nov 29 '19 15:11 astefanutti

@vgibilmanno could you try:

$ curl -k -v -H "Authorization: Bearer `aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE | jq -r .status.token`" https://OMITTED.amazonaws.com/api/v1/namespaces/SOMENAMESPACE/pods

astefanutti avatar Nov 29 '19 15:11 astefanutti

@astefanutti This one worked!

*   Trying SOMEIP:443...
* TCP_NODELAY set
* Connected to OMMITED.amazonaws.com (SOMEIP) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=kube-apiserver
*  start date: Jan 23 14:39:32 2019 GMT
*  expire date: Nov 17 23:02:45 2020 GMT
*  issuer: CN=kubernetes
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5653cfda06e0)
> GET /api/v1/namespaces/SOMENAMESPACE/pods HTTP/2
> Host: OMMITED.amazonaws.com
> User-Agent: curl/7.66.0
> Accept: */*
> Authorization: Bearer SOMETOKEN
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< audit-id: c62fa312-7120-426b-970e-9663a90ec01f
< content-type: application/json
< date: Fri, 29 Nov 2019 15:58:25 GMT
<
{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/namespaces/SOMENAMESPACE/pods",
    "resourceVersion": "82672454"
  },
  "items": [SOMEITEMS]
* Connection #0 to host OMMITED.amazonaws.com left intact

vgibilmanno avatar Nov 29 '19 16:11 vgibilmanno

@vgibilmanno, good, what about:

$ curl -k -v -H "Authorization: Bearer `aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE | jq -r .status.token`" https://OMITTED.amazonaws.com/

$ curl -k -v -H "Authorization: Bearer `aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE | jq -r .status.token`" https://OMITTED.amazonaws.com/api

$ curl -k -v -H "Authorization: Bearer `aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE | jq -r .status.token`" https://OMITTED.amazonaws.com/api/v1/namespaces

astefanutti avatar Nov 29 '19 16:11 astefanutti

@astefanutti Only the 2. command worked. The other fails with statuscode 403 Forbidden. The 2. command has the following response body:

"kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "OMITTED.compute.internal:443"
    }
  ]

I tried the following command and it worked too returning a huge json object:

$ curl -k -v -H "Authorization: Bearer aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE | jq -r .status.token" https://OMITTED.amazonaws.com/api/v1

vgibilmanno avatar Nov 29 '19 16:11 vgibilmanno

@vgibilmanno thanks a lot. Can you confirm the following command returns 403:

$ curl -k -v -H "Authorization: Bearer `aws-iam-authenticator token -i SOMEOTHERNAME -r arn:aws:iam::SOMEROLEID:role/SOMEROLE | jq -r .status.token`" https://OMITTED.amazonaws.com/api/v1/namespaces

astefanutti avatar Nov 29 '19 16:11 astefanutti

@astefanutti Yes

HTTP/2 403
< audit-id: 31bb7393-cefd-414b-8956-898135d0da83
< content-type: application/json
< x-content-type-options: nosniff
< content-length: 347
< date: Fri, 29 Nov 2019 16:56:30 GMT
<
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "namespaces is forbidden: User \"SOMEUSER\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope",
  "reason": "Forbidden",
  "details": {
    "kind": "namespaces"
  },
  "code": 403
* Connection #0 to host OMITTED.amazonaws.com left intact

vgibilmanno avatar Nov 29 '19 16:11 vgibilmanno

OK so I think we've nail down the root cause of this issue. Your user account is not granted permission to list namespaces.

As as work-around, could you update your kubeconfig file and add the namespace you have access to, e.g.:

- context:
    cluster: SOMENAME
    user: SOMENAME
    namespace: NAMESPACE
...

Then run Kubebox.

astefanutti avatar Nov 29 '19 17:11 astefanutti

@astefanutti ok yes now it works. But it seems like I don't have permissions to see the resources usage metrics. I can execute kubectl top pod though. Well... better than nothing :) I hope I didn't waste too much of your time.

vgibilmanno avatar Nov 29 '19 17:11 vgibilmanno

@vgibilmanno great! thanks for the feedback. Resources usage metrics requires extra permissions to proxy nodes.

Thanks a lot for your collaboration on this. I think we can let that issue open so that we improve the error message when the user is granted permission to list the namespaces and no namespace is provided.

astefanutti avatar Nov 29 '19 17:11 astefanutti

I'm 99% sure this issue will cover this but I get an ugly error when I try and list namespaces. I only have access to a single namespace, and it works fine otherwise, but if I hit N to list namespaces this comes up. I understand why it won't let me, but catching the error with a better message would be cool.

Screen Shot 2020-01-16 at 3 26 39 PM

bradam12 avatar Jan 16 '20 20:01 bradam12

@bradamson thanks a lot for the feedback. I agree with your suggestion to catch the error and display a proper message in the namespaces list box, instead of dumping the stack trace.

astefanutti avatar Jan 17 '20 07:01 astefanutti