Kubefwd is not properly reconnecting when new pod is created

[imdbere]

Hi, first of all thanks for making this project available :)

One thing that makes kubefwd harder to use is that it doesn't properly reconnect if e.g. a deployment is updated and therefore an old pod is terminated and a new one is created. The error that is created looks like this:

ERRO[17:36:11] Runtime: an error occurred forwarding 80 -> 80: error forwarding port 80 to pod 1b4e23d5467af287d3685ede5f693f04b5a909d475cb31d470381f6c3c956ffe, uid : failed to find sandbox "1b4e23d5467af287d3685ede5f693f04b5a909d475cb31d470381f6c3c956ffe" in store: not found

After that, only restarting the command makes the port-forwarding work again. Is this intended behaviour ? It would be ideal if kubefwd would properly shift traffic to the new pod like it is done by kubernetes services or at least doing some reconnecting.

This is the command used: sudo kubefwd svc -n default -d some-name -x some-kubeconfig

Thanks!

[ndj888]

@imdbere kubefwd checks the pod list every 5 minutes and will fix this automatically. I don't think it's necessary for kubefwd to monitor pod changes and react in real time, it will consume more resources. It is recommended to use kubefwd for local development environment. If stable access to pods inside the cluster is required, I recommend using a private load balancer for service exposure.

[kikonen]

5 min delay in reconnecting effectively means that user must stop kubefwd and restart it, i.e. waiting 5 mins is long time to wait when trying to do some development work locally.

[gustaff-weldon]

@ndj888 this is a tool for local development isn't it?

When service gets redeployed in the cluster you're connected to, you want your local app to pick this up. 5 min delay makes reconnect virtually non-functional. For me, 5 seconds would be more appropriate as I want to focus on app I work on, and let kubefwd do its job :)

Idea: why not just make timeout a CLI option and let users decide what is best resource/delay combo?

[gustaff-weldon]

Btw, I'm experiencing very similar problem. The moment pod gets reshuffled or new deployment happens, the port forwarding dies and does not recover (even after 5 minutes), see below:

❯ sudo kubefwd svc -v --namespace postgresql -f metadata.name=postgresql
Password:
INFO[15:32:08]  _          _           __             _
INFO[15:32:08] | | ___   _| |__   ___ / _|_      ____| |
INFO[15:32:08] | |/ / | | | '_ \ / _ \ |_\ \ /\ / / _  |
INFO[15:32:08] |   <| |_| | |_) |  __/  _|\ V  V / (_| |
INFO[15:32:08] |_|\_\\__,_|_.__/ \___|_|   \_/\_/ \__,_|
INFO[15:32:08]
INFO[15:32:08] Version 1.22.4
INFO[15:32:08] https://github.com/txn2/kubefwd
INFO[15:32:08]
INFO[15:32:08] Press [Ctrl-C] to stop forwarding.
INFO[15:32:08] 'cat /etc/hosts' to see all host entries.
INFO[15:32:08] Loaded hosts file /etc/hosts
INFO[15:32:08] HostFile management: Original hosts backup already exists at /Users/kolo/hosts.original
INFO[15:32:08] Successfully connected context: docker-desktop
DEBU[15:32:08] Registry: Start forwarding service postgresql.postgresql.docker-desktop
DEBU[15:32:08] Resolving: postgresql to 127.1.27.1 (postgresql)
INFO[15:32:08] Port-Forward:       127.1.27.1 postgresql:5432 to pod postgresql-0:5432
ERRO[15:34:39] Runtime: an error occurred forwarding 5432 -> 5432: error forwarding port 5432 to pod c9a9c7e84ba15c880fd6ce333089cf1022519ba2ca890a44e7afd9172f1195da, uid : container not running (c9a9c7e84ba15c880fd6ce333089cf1022519ba2ca890a44e7afd9172f1195da)
ERRO[15:34:39] Runtime: lost connection to pod
WARN[15:34:39] Stopped forwarding pod postgresql-0 for postgresql.postgresql.docker-desktop
INFO[15:40:10] update service postgresql/postgresql.

[dobesv]

I've been having the same problem since I first started using kubefwd years ago, I have to restart it a lot. I even tried putting in a fix that was merged, but the issue resurfaced again later.

e.g. https://github.com/txn2/kubefwd/pull/153