proxy_next_upstream doesn't work for nginx/nginx-ingress:2.2.2

[AndriiPasternak1989]

trafficstars

Is your feature request related to a problem? Please describe. We can not make the Nginx ingress controller not retry idempotent calls. Please see the screenshots below

Describe the solution you'd like A clear and concise description of what you want to happen. We would like to be able to disable the proxy next upstream . https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-next-upstream Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered. There is only one solution which is to start using community Nginx. Additional context Add any other context or screenshots about the feature request here.

[github-actions[bot]]

Hi @AndriiPasternak1989 thanks for reporting!

Be sure to check out the docs while you wait for a human to take a look at this :slightly_smiling_face:

Cheers!

[haywoodsh]

HI @AndriiPasternak1989

Have you tried Transport Server and configure these directives with upstream parameters?

I think you can also use nginx.org/location-snippets.

[AndriiPasternak1989]

Hi @haywoodsh , We haven't tried the transport server yet. Just to give you an idea, we have too many ingresses that's why I guess the transport server will not work for us. The transport server requires us to define all services. We would like to avoid such a big change to our environment. Any chance you know how we can disable proxy next upstream using location snippets?

[brianehlert]

The Ingress resource and the CRDs TransportServer and VirtualServer resources can co-exist. The catch is that any single hostname can only be defined using one or the other. And that gets down to how hostnames and paths are managed using Ingress in your environment.

proxy-next-upstream is an annotation that is specific to the K8s community ingress controller. This is why using that annotation with the root namespace of this project did not automagically work. Annotations are project specific, there is no annotation standard.

The immediate options are: A snippet can be applied using the Ingress resource https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-snippets/

Custom annotations for Ingress resources can be defined https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/custom-annotations/ to templatize settings.

Use VirtualServer or VirtualServerRoute and define a location snippet. (since this is an http block directive). TransportServer would be used if this was a stream block directive.

Admitting without first validating myself.. Have you tried on the respective Ingress:

    nginx.org/location-snippets: |
        proxy_next_upstream off;

The resulting configuration will still be validated by nginx - since this project works with straight nginx.conf under the hood.

[AndriiPasternak1989]

Hi @brianehlert , adding "location-snippets" did a trick. It would be great if you could add specific annotations like "proxy-next-upstream" for the next release. Managing proxy next upstream via snippets is not that obvious compared to specific annotations. Thanks!

[brianehlert]

Hey @AndriiPasternak1989 Since you raised this issue we have been having a bit of an internal debate about your full use case (not just this single setting).

I am guessing that there is something about your service that we are simply not seeing. Something that matters to you but we have not fully understood from your description.

Can you help us out a bit with a description of your full situation? (not simply the bad behavior)

[AndriiPasternak1989]

Hi @brianehlert , Long story short, at the beginning we did not know that there was a 1-minute value for timeout setting in the Nginx controller. Then we noticed that when the call terminates after 1 minute with an error "503 HTTP" there will be another retry with a "404 HTTP code". What happens behind the scene is that the initial call (it executes method DELETE) schedule a job that lasts for 90 seconds. Nginx controller terminates the initial call because it exceeds the timeout settings (60 seconds). Then, it does retry with the same request. The retry(second request) fails with the error "404 HTTP code" because the previous request had scheduled a job. That job completes even after the initial call has been terminated by Nginx with the error 503 HTTP. At that time it was not clear to us who did retries. Now we know that there is the logic that does retry for "idempotent " calls like "DELETE". It would be great if we could explicitly turn off "proxy_next_upstream" settings via annotations not via snippet annotations. I hope it makes any sense.