Port forwarding failing to start (using Argo Rollouts)

[mkonecny-atlassian]

Expected behavior

When using skaffold dev --port-forward the ports I have setup in my YAML would automatically port-forward.

Actual behavior

We are using Argo Rollout in our Helm chart (not sure if this matters). When using the above I see the following:

Saving 1 charts
Deleting outdated charts
NAME: dx-ref-java-quarkus-im
LAST DEPLOYED: Mon Jan 13 16:57:25 2025
NAMESPACE: dx-ref-java-quarkus-im
STATUS: deployed
REVISION: 1
TEST SUITE: None
Waiting for deployments to stabilize...
Deployments stabilized in 12.8455ms
port forwarding pod-dx-ref-java-quarkus-im-webserver-dx-ref-java-quarkus-im-7070 got terminated: output: Error from server (NotFound): pods "dx-ref-java-quarkus-im-webserver" not found

port forwarding service-dx-ref-java-quarkus-im-service-canary-dx-ref-java-quarkus-im-80 got terminated: output: error: unable to forward port because pod is not running. Current status=Pending

port forwarding service-dx-ref-java-quarkus-im-service-dx-ref-java-quarkus-im-80 got terminated: output: error: unable to forward port because pod is not running. Current status=Pending

Listing files to watch...
 - docker.atl-paas.net/atlassian/dx-ref-java-quarkus-im
Press Ctrl+C to exit
Watching for changes...
[microservice] INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager -cp "." -jar /deployments/quarkus-run.jar
[microservice] INFO running in /deployments
[microservice] __  ____  __  _____   ___  __ ____  ______
[microservice]  --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
[microservice]  -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
[microservice] --\___\_\____/_/ |_/_/|_/_/|_|\____/___/
[microservice] 2025-01-13 15:57:27,776 INFO  [io.und.websockets] (main) UT026003: Adding annotated server endpoint class com.atlassian.nebulae.micros.service.websocket.WebsocketResource for path /ws/{id}
[microservice] 2025-01-13 15:57:28,501 INFO  [io.quarkus] (main) dx-ref-java-quarkus-im 0.0.238-SNAPSHOT on JVM (powered by Quarkus 3.17.5) started in 1.056s. Listening on: http://0.0.0.0:7070
[microservice] 2025-01-13 15:57:28,501 INFO  [io.quarkus] (main) Profile local activated.
[microservice] 2025-01-13 15:57:28,501 INFO  [io.quarkus] (main) Installed features: [agroal, amazon-dynamodb, amazon-s3, amazon-sns, amazon-sqs, cdi, elasticsearch-rest-client, jdbc-postgresql, narayana-jta, redis-client, resteasy, resteasy-client, resteasy-jackson, smallrye-context-propagation, vertx, websockets, websockets-client]

Information

• Skaffold version: v2.13.2
• Operating system: Darwin 24.1.0
• Installed via: Homebrew
• Contents of skaffold.yaml:

apiVersion: skaffold/v4beta11
kind: Config
metadata:
  name: dx-ref-java-quarkus-im
build:
  artifacts:
    - image: <redacted>
      custom:
        buildCommand: |
          ./mvnw package -Dmaven.test.skip -T 1C -Pnebulae
          docker tag <redacted> $IMAGE
        dependencies:
          paths:
            - pom.xml
            - src/main
            - src/test
deploy:
  helm:
    releases:
      - name: dx-ref-java-quarkus-im
        chartPath: helm
        valuesFiles:
          - helm/values.yaml
        namespace: dx-ref-java-quarkus-im

## User defined port forwards, currently not working
portForward:
  - resourceType: pod
    resourceName:  dx-ref-java-quarkus-im-webserver
    namespace: dx-ref-java-quarkus-im
    port: 7070

Additional logs:

DEBU[0004] found open port: 7070                         subtask=-1 task=DevLoop
DEBU[0004] Running command: [kubectl --context kind-nebulae port-forward --pod-running-timeout 1s --namespace dx-ref-java-quarkus-im pod/dx-ref-java-quarkus-im-webserver 7070:7070]  subtask=pod/dx-ref-java-quarkus-im-webserver task=PortForward
DEBU[0004] port forwarding pod-dx-ref-java-quarkus-im-webserver-dx-ref-java-quarkus-im-7070 got terminated: exit status 1, output: Error from server (NotFound): pods "dx-ref-java-quarkus-im-webserver" not found  subtask=pod/dx-ref-java-quarkus-im-webserver task=PortForward
port forwarding pod-dx-ref-java-quarkus-im-webserver-dx-ref-java-quarkus-im-7070 got terminated: output: Error from server (NotFound): pods "dx-ref-java-quarkus-im-webserver" not found

I am surprised that the deployment is "stabilized" in 12ms before the pod is actually running. Note that the Pod is healthy in ~2s but port forward needs to be manually setup. The name is dx-ref-java-quarkus-im-webserver-6fb9db8544-6xx4s (note the hash) and it's not controlled by Deployment (but by a ReplicaSet which is controlled by ArgoRollout).