consul-k8s icon indicating copy to clipboard operation
consul-k8s copied to clipboard

Use role/rolebinding for sync-catalog when running in a single ns

Open ishustava opened this issue 3 years ago • 1 comments

Changes proposed in this PR:

Sometimes users of sync catalog do not have permissions to create Cluster*-level RBAC resources. In those cases, typically they would need to only use a single namespace to sync services from. This PR proposes to support that use case by allowing a user to provide namespace for the catalog sync controller to watch.

  • When syncCatalog.k8sAllowNamespaces is a single item list that is not [*], we will create Role and RoleBinding instead of the ClusterRole and ClusterRoleBinding
  • Sync catalog controller will only watch that one namespace.

How I've tested this PR:

Step 1

Create namespace where my services will be synced from:

$ kubectl create ns test

Step 2

Create a role and role binding allowing sync catalog's service account in the installation namespace to read services and endpoints in the test namespace

# role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: consul-sync-catalog
  namespace: test
rules:
  - apiGroups: [""]
    resources:
      - services
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups: [""]
    resources:
      - nodes
    verbs:
      - get
# rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: consul-sync-catalog
  namespace: test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: consul-sync-catalog
subjects:
  - kind: ServiceAccount
    name: consul-sync-catalog
    namespace: consul

Step 3

Deploy helm chart with the following config, providing the namespace where the services to be synced will be running via k8sAllowNamespaces (in my case it's called test):

global:
  name: consul
  imageK8S: ishustava/consul-k8s-dev:sync-single-ns
syncCatalog:
  enabled: true
  toK8S: false
  k8sAllowNamespaces: ["test"]
helm install consul -f config.yaml -n consul --create-namespace charts/consul

Step 4

Deploy a service to the test namespace and check that it's synced to consul

# service.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: static-server
spec:
  selector:
    app: static-server
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: static-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: static-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: static-server
  template:
    metadata:
      name: static-server
      labels:
        app: static-server
    spec:
      containers:
        - name: static-server
          env:
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
          image: ishustava/http-echo:latest
          command:
              - "/bin/sh"
              - "-ec"
              - |
                /http-echo -listen=:8080 -text="hello world $POD_IP"
          ports:
            - containerPort: 8080
              name: http
      terminationGracePeriodSeconds: 0
      serviceAccountName: static-server

After ~30sec, check that the service is registered:

$ kubectl exec sts/consul-server -n consul -- consul catalog services
consul
static-server-test

How I expect reviewers to test this PR:

👀

Checklist:

  • [ ] Tests added
  • [ ] CHANGELOG entry added

    HashiCorp engineers only, community PRs should not add a changelog entry. Entries should use present tense (e.g. Add support for...)

ishustava avatar Jan 28 '22 22:01 ishustava

CLA assistant check
All committers have signed the CLA.

hashicorp-cla avatar Mar 12 '22 17:03 hashicorp-cla

This feature really suits my use-case in a rather restricted environment, I'd love for this to be merged.

GoodOldJack12 avatar Dec 19 '22 15:12 GoodOldJack12