Document port_forward usage with charts that deploy multiple pods

[nathanperkins]

I'm deploying prometheus-community/kube-prometheus-stack with multiple pods and it's not clear how I should use port_forwards to forward ports from specific pods.

[rboydns]

@nathanperkins did you ever find a solution to this?

[JustinLex]

I think this is functionality missing from helm_resource().

Right now, the port_forwards argument on helm_resource() passes directly to k8s_resource()'s port_forwards argument underneath. See the PR that added the port forward functionality: https://github.com/tilt-dev/tilt-extensions/pull/356

Because of this, there is no way to specify which pod should be port forwarded. I think k8s_resource() just picks the first pod from the set, from what I've seen happen.

The best workaround for this is to add your own k8s_resource() to forward the port, but use discovery_strategy='selectors-only' and the extra_pod_selectors argument to filter to only the pod you want.

Here's a working example for the Pyroscope operator:

helm_resource('pyroscope',
              chart='grafana/pyroscope',
              release_name='pyroscope',
              namespace="pyroscope",
              flags=[
                  '--create-namespace'
              ])
k8s_resource(workload='pyroscope',
             extra_pod_selectors={'statefulset.kubernetes.io/pod-name': 'pyroscope-0'},
             discovery_strategy='selectors-only',
             port_forwards="4040:4040")

This port-forwards the pyroscope pod, instead of picking the default pyroscope-agent pod.

Note that I also tried using k8s_resource() with the objects argument, but that doesn't work because objects doesn't work with k8s_custom_deploy(). See https://github.com/tilt-dev/tilt/issues/5406. extra_pod_selectors is the only way to do this, because it is aware of k8s_custom_deploy(). See https://github.com/tilt-dev/tilt/pull/5774.

Hope that helps anybody else encountering this problem!