Field to disallow remote clusters

[m1o1]

Requesting this feature after asking about this in the skaffold slack.

I would like to have an option in the skaffold.yaml which disallows running against any remote clusters (recursively, including any "required" sub-skaffold configs), similar to the restrictions that Tilt puts in place by default. A field that ensures noone on the team can accidentally use skaffold on our remote clusters, since we have a different setup for deploying there, and we only use skaffold for local testing.

A few times, I've run skaffold run against our remote cluster when I meant to run against my local one. Even just being a dev remote cluster, it is still a bit of a hassle to undo once I've done that. If there were some top-level field to restrict skaffold to only run on local clusters, that would be very helpful in ensuring that noone on the team accidentally messes up our remote dev cluster with an accidental skaffold command.

Profiles are helpful, but there's no option for "all non-remote clusters". I had to do a workaround where I:

• moved my skaffold.yaml to skaffold-core.yaml (this skaffold-core.yaml "requires" a few other skaffold configs).
• Made my skaffold.yaml have multiple profiles for docker-desktop, minikube, etc.
• In my skaffold.yaml, I "require" my skaffold-core.yaml and activate its "local" profile if docker-desktop, minikube, etc. are activated.

This would be okay, but #5459 is preventing this workaround from working for us.

[briandealwis]

You could also consider using moving the sensitive clusters to a separate kube-config file that you pull in as required with the KUBECONFIG environment variable.

[m1o1]

Yes that could help, but it's a bit difficult to enforce on the team. The clusters are not super sensitive (just dev clusters) but they are deployed in a different way. So I think most on the team would want to be able to access it with their default contexts.

Also, I've done something kinda similar myself, but there were times I've lost track of which shell I have open has which KUBECONFIG set.

My preferred solution would be to have a kubeContext activation for profiles that activates when using a "local" cluster defined by skaffold's local cluster heuristics. I am not familiar with the codebase but I can try to submit a pr if accepted.

[briandealwis]

@m1o1 the team chatted about this a bit and although we appreciate the problem, there are mechanisms already within Kubernetes (specifically roles and service accounts) to avoid some of these problems. And there's also a significant gap involving pushing to registries.

As you noted, Skaffold's auto-activated profiles could be used to provide protection. You could use something like the following to prevent deployments except for minikube.

deploy:
  kubectl:
    manifests: ['do not deply except to minikube']

profiles:
- activation:
    kubeContext: 'minikube' 
  deploy:
    kubectl:
      manifests: ["k8s-*.yaml"]

[m1o1]

Yeah the profiles with kubeContext activation provides what I need to make this work. It's sort-of just a cleanliness thing to only need one for local instead of a list, but it's not a big deal since people on the team can just add their local context to the list if it's missing. And there probably won't be more than 2 or 3.

[briandealwis]

I'm going to re-open this issue as a newcomer to Skaffold was asking why Skaffold tried to push their image to their repository. They had setup their skaffold.yaml to reference their full image paths:

build:
  artifacts:
  - image: gcr.io/project/app/frontend
  - image: gcr.io/project/app/backend

They weren't using a local cluster, and then tried skaffold dev. As a result, Skaffold tried to build and push the images to the registry, which they found surprising.

One pattern that prevents accidental pushes is to use plain image names, and they use --default-repo or profile-patching to adjust the image names. But --default-repo requires use of a script to launch skaffold, and profile-patching is annoying to set up.

[nkubala]

related to https://github.com/GoogleContainerTools/skaffold/issues/1313. we originally closed that issue since this can technically be solved by configuring RBAC on the cluster, but i think this would be a fine feature to support in skaffold as a safeguard mechanism

[DazEdword]

Hi all, just a general thumbs up to this feature request. I do understand there are mechanisms in place at the cluster level, but I'd welcome a skaffold level safeguard too.