External `kubectl` executor

[mszostok]

Overview

Currently, the kubectl executors is built-in. After https://github.com/kubeshop/botkube/issues/650 we need to create an external kubectl plugin.

Lessons learned

Based on the current implementation, we can see such problems:

1. User facing problems:

   • Always need to put all flags at the end of the command
     • For example the default example from kubectl wait -h doesn't work in BotKube
       • wait --for=condition=Ready pod/foo
   • Only one version of kubectl is supported
   • When a valid kubectl verb is specified, we got command not supported. It would be nice to get more verbose information. For example:
     • the server doesn't have a resource type "certs"
     • you are not authorized to work with "pods"
     • unknown command "ges" for "kubectl"

       Did you mean this?
       set
       get
       

   • If all kubectl executors are disabled, you always get command nod supported instead of sth like the kubectl is not configured to work in this channel
   • minor: needs to learn a new syntax for commands and namespaces.

2. Developer facing problems:

   • It's hard to ensure a proper security
     • We need to do the job of API Sever on our end, which is error-prone
       • Recently for -A we returned all resources even if not authorized.
   • We need to reimplement the kubectl parsing command
     • it's problematic and hard to follow the latest releases
     • we had an issue with slash syntax: get po/podname-123
     • we still have an issue when flags are specified before the resource name

Ideas

• [breaking change] Explicitly say the @Botkube kubectl get po instead of @Botkube get po

  • It's more typing, but we can support aliases, e.g. for kubectl -> kc -> k etc.

• Security:

  1. It would be good to utilize a general approach https://github.com/kubeshop/botkube/issues/675
  2. If not possible, then, based on the commands.verbs and commands.resource create a dedicated:
     • Role in all namespaces.include (take care of namespaces.exclude)
     • ClusterRole if namespace.include: ["*"] and empty namespace.exclude
  3. [breaking change] Or simply use the native K8s rbac where the user can attach a given SA to a given kubectl executor. Instead of defining all this verbs and commands.

• Execute kubectl command with a user instead of doing the validation logic in BotKube.