Error with "cannot assign requested address" After Running for Some Time

[m-yosefpor]

Describe the bug

I'm running argocd-image-update to update some images based on semver strategy. Everything works fine in the beginning, but after some time (couple days) I started getting error says "connect: cannot assign requested address" and nothing works. Restarting pod could bring it back to working state, but the issue will return in a couple of days.

It seems the connections are never closed.

$ netstat -anp  | grep ESTABLISHED | wc -l
84697

Version Image-updater v0.12.0 (helm chart v0.8.0) Argocd v2.3.2

Logs

time="2022-04-11T07:41:40Z" level=error msg="Could not get tags from registry: Get "https://image-registry.openshift-image-registry.svc:5000/v2/": dial tcp 172.30.17.78:5000: connect: cannot assign requested address" alias=cron application=snappdoctor-mds image_name=snappdoctor-mds/mds-cron image_tag=v0.1.1 registry="image-registry.openshift-image-registry.svc:5000" time="2022-04-11T07:41:40Z" level=info msg="Processing results: applications=16 images_considered=26 images_skipped=18 images_updated=0 errors=26"

[jannfis]

Hi @m-yosefpor, what write-back strategy do you use?

[m-yosefpor]

@jannfis Thanks for your attention. argocd application override writeback.

[andreasathanasopoulos]

@m-yosefpor did you find any solution? I have exactly the same issue

[m-yosefpor]

@m-yosefpor did you find any solution? I have exactly the same issue

a cronjob to rollout the deployment every few hours

[icecoffee531]

We also have the same issue, and it has also caused a NAT concurrent session exhaustion, making the cluster unable to access the public network. Have any update?