k8s: Use contexts instead of namespaces for flexibility

[adelbertc]

NOTE: This is a breaking change for in-cluster deployments, more info below:

Using the --namespace flag assumes the "current" credentials are valid for that namespace (e.g. kubectl does not try to switch contexts when you explicitly specify a namespace, which makes sense). However it is possible that a Kubernetes deployment expects different credentials per-namespace, which KUBECONFIG supports. However given how we're using --namespace right now, Nelson isn't leveraging that flexibility.

This change instead uses --context to explicitly specify the context (and therefore token + namespace). However since contexts can be named anything (there is a logical name for each context which ties together (cluster, namespace, token)), and because we expect each DC to have its own KUBECONFIG, Nelson will assume the context name is the same as the namespace name.

In addition this change also removes in/out-cluster distinction in the Kubernetes backend.

Previous in-cluster behavior used assumed administrative credentials automatically mounted in the Pod (https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/#accessing-the-api-from-a-pod) to do deployments in-cluster. However with the previous change to use --context instead of --namespace, this no longer works (because there is no KUBECONFIG file, it just uses the token). Therefore even if Nelson is deployed in the same cluster a corresponding kubeconfig must still be mounted + specified. In any case this also makes the semantics perhaps slightly less confusing and/or more consistent.

[codecov-io]

Codecov Report

Merging #228 into 0.11.x will increase coverage by 2.48%. The diff coverage is 28.57%.

@@            Coverage Diff             @@
##           0.11.x     #228      +/-   ##
==========================================
+ Coverage   53.08%   55.56%   +2.48%     
==========================================
  Files         133      134       +1     
  Lines        4591     4373     -218     
  Branches      111      112       +1     
==========================================
- Hits         2437     2430       -7     
+ Misses       2154     1943     -211

Impacted Files	Coverage Δ
core/src/main/scala/Datacenter.scala	83.05% <ø> (+3.05%)	:arrow_up:
...ore/src/main/scala/scheduler/KubernetesShell.scala	5.26% <0%> (ø)	:arrow_up:
core/src/main/scala/Kubectl.scala	0% <0%> (ø)	:arrow_up:
core/src/main/scala/Config.scala	76.44% <100%> (-2.41%)	:arrow_down:
core/src/main/scala/Http4sConsul.scala	70% <0%> (-14.62%)	:arrow_down:
core/src/main/scala/workflows/Magnetar.scala	2.94% <0%> (-0.64%)	:arrow_down:
core/src/main/scala/Nelson.scala	40.1% <0%> (-0.32%)	:arrow_down:
core/src/main/scala/yaml/ManifestV1Parser.scala	77.77% <0%> (-0.08%)	:arrow_down:
core/src/main/scala/ManifestValidator.scala	94.78% <0%> (-0.05%)	:arrow_down:
... and 18 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 47a275f...36d4b65. Read the comment docs.

[timperrett]

@adelbertc did you get a chance to think about what to do here? This will break existing deployments