skaffold icon indicating copy to clipboard operation
skaffold copied to clipboard

User defined port forwardings are not k8s context aware

Open mefu opened this issue 8 months ago • 0 comments

It seems there is no way to define which kubeContext a port forwarding should use.

For example, I want to create a development environment with multiple clusters using a config like below

apiVersion: skaffold/v4beta11
kind: Config
metadata:
  name: config-for-cluster-1
profiles:
- name: p1
  build:
    artifacts:
    - image: service1
      # ...
  deploy:
    kubeContext: context-for-cluster-1
    helm:
      releases:
      # helm chart that uses service1 image built above
      - name: chart1
        namespace: ns1
        createNamespace: true
        chartPath: charts/chart1
        # ...
  portForward:
  - resourceType: service
    resourceName: service1
    namespace: ns1
    port: 8000
    localPort: 8000
---
apiVersion: skaffold/v4beta11
kind: Config
metadata:
  name: config-for-cluster-2
profiles:
- name: p2
  build:
    artifacts:
    - image: service2
      # ...
  deploy:
    kubeContext: context-for-cluster-2
    helm:
      releases:
      # helm chart that uses service2 image built above
      - name: chart2
        namespace: ns2
        createNamespace: true
        chartPath: charts/chart2
        # ...
  portForward:
  - resourceType: service
    resourceName: service2
    namespace: ns2
    port: 8000
    localPort: 8001

This way I can actually use skaffold dev -p p1,p2 or skaffold dev -p p1 or skaffold dev -p p2 and it works as expected. Even port forwards work as expected. Only problem is I get constant errors with both profiles enabled. I feel like it tries to apply port forwards of both configs to both clusters because I get following errors like 10 times a second:

time="2025-03-27T20:26:49+03:00" level=warning msg="could not map pods to service ns1/service1/8000: getting service ns1/service1: services \"service1\" not found" subtask=service/service1 task=PortForward
time="2025-03-27T20:26:49+03:00" level=warning msg="could not map pods to service ns2/service2/8000: getting service ns2/service2: services \"service2\" not found" subtask=service/service2 task=PortForward

There are few port <x> is taken, retrying... errors mixed in too. I should say this again, port forwards are actually done. http://127.0.0.1:8000 and http://127.0.0.1:8001 actually goes to where it should.

If I delete portForward from both of configs and use --port-forward from CLI, it also works and there are no erorrs. Seems like skaffold is aware of different clusters when it does port forwarding for services automatically. Problem with this is ports are random and I need static ports to configure some other stuff.

I was expecting to find a kubeContext option for portForward, like the one for deploy but it does not exist. Should that be implemented or can skaffold find correct context by tracking where service is deployed?

I anonymized image/service/chart names etc from both logs and config. If this is not enough and you need something reproducible, I can try to create that.

mefu avatar Mar 27 '25 18:03 mefu