k8s clustering doesn't work

[rjbaat]

I have followed the documentation and used this setup:

apiVersion: v1
kind: Service
metadata:
  name: emqx
spec:
  ports:
  - port: 32333
    nodePort: 32333
    targetPort:  emqx-dashboard
    protocol: TCP
  selector:
    app: emqx
  type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: emqx
  labels:
        app: emqx
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: emqx
    spec:
      containers:
      - name: emqx
        image: emqx/emqx:latest
        ports:
        - name: emqx-dashboard
          containerPort: 18083
        env:
        - name: EMQX_CLUSTER__DISCOVERY
          value: k8s
        - name: EMQX_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__APISERVER
          value: "https://kubernetes.default:443"
        - name: EMQX_CLUSTER__K8S__NAMESPACE
          value: default
        - name: EMQX_CLUSTER__K8S__SERVICE_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE
          value: ip
        - name: EMQX_CLUSTER__K8S__APP_NAME
          value: emqx
        tty: true

But when checking the logs i see that emqx gives the following message:

2019-04-01 07:56:53.613 [error] Ekka(AutoCluster): Discovery error: {403,
1-4-2019 09:56:53 "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"endpoints \\\"emqx\\\" is forbidden: User \\\"system:serviceaccount:default:default\\\" cannot get resource \\\"endpoints\\\" in API group \\\"\\\" in the namespace \\\"default\\\"\",\"reason\":\"Forbidden\",\"details\":{\"name\":\"emqx\",\"kind\":\"endpoints\"},\"code\":403}\n"}

Is there something i need to do to make the api accessible? I am running k8s via rancher.

[rjbaat]

added this role to the service account that is used and now it works:

kubectl create clusterrolebinding default-admin --clusterrole cluster-admin --serviceaccount=default:default

[Rory-Z]

@rjbaat Sorry to reply you so late I suggest you try to deploy emqx on k8s using helm, you can get our chart from here.

[rjbaat]

@zhanghongtong i tried the helm chart, but it gives me an error: Wait helm template failed. Error: stat /emqx: no such file or directory : exit status 1 What do i do wrong?

[Rory-Z]

@rjbaat Please provide more details

[rjbaat]

Well i added the helm url as global to rancher catalog manager. Then launched the app via search on the catalog. Didn't change anything and launched. So with no answers. All default. Then it tries to start it but i get: Wait helm template failed. Error: stat /emqx: no such file or directory : exit status 1. I came accross this topic: https://github.com/rancher/rancher/issues/18535#issuecomment-491100379 Can this be of some help to you?

[rjbaat]

Now i tried not to add the url: https://repos.emqx.io/charts but instead add the url: https://github.com/emqx/emqx-chart.git as chart repo and i can install the chart. It only doesnt want to get installed in a namespace other that default.

Edit: i see there is an answer in the chart that provides the namespace and this has to be the same as the namespace it is installed to.

[Rory-Z]

@rjbaat Thank you for your feedback, If you want to install emqx in any other namespace, you need to create the namespace first

[rjbaat]

Thnx for your help. Any chance to point me in a direction of adding (LE) certificates to the cluster too?

[Rory-Z]

@rjbaat I didn't try to add certificates to the previous cluster, I suggest you try to create the certificate file as a Secret resource and mount it on the pods The command to create the Secret resource is as follows:

Kubectl create secret generic your-secret-name --from-file=/path/to/your/file

Looking forward to your feedback