bug: unexpected result when ingress restarted

[xiangtianyu]

Issue description

if we apply yaml quickly just after the ingress restarted, it will cause some unexpected appearance

It occurs when i apply new yaml just after ingress restart, if i restart and wait for a minute, it will handle this case as usual.

Environment

• your apisix-ingress-controller version (output of apisix-ingress-controller version --long): 1.5.0-rc1
• your Kubernetes cluster version (output of kubectl version): 1.24.3
• if you run apisix-ingress-controller in Bare-metal environment, also show your OS version (uname -a):

Minimal test code / Steps to reproduce

I used this yaml to create apisix route:

apiVersion: apisix.apache.org/v2beta3
kind: ApisixRoute
metadata:
  name: httpserver-route
spec:
  http:
  - name: rule3
    match:
      hosts:
      - local.httpbin.org
      paths:
      - /*
    backends:
       - serviceName: httpbin
         servicePort: 80

and then i restarted apisix ingress and changed this yaml

apiVersion: apisix.apache.org/v2beta3
kind: ApisixRoute
metadata:
  name: httpserver-route
spec:
  http:
  - name: rule1
    match:
      hosts:
      - local.httpbin.org
      paths:
      - /*
    backends:
       - serviceName: httpbin
         servicePort: 80
  - name: rule2
    match:
      hosts:
      - local.httpbin.org
      paths:
      - /*
    backends:
       - serviceName: httpbin
         servicePort: 80

and then apply

Actual result

it returned like this

Error log

no err log

Expected result

it should be like this

[AlinsRan]

No matter how I reproduce it, it is normal.

[xiangtianyu]

No matter how I reproduce it, it is normal.

I retried again and definitely reproduce it. this time i used the execute file built by the newest code

Or you can stop the ingress and apply yaml and start the ingress.

[AlinsRan]

Maybe you applied yaml before apisix-ingress-controller pod initialization.

[xiangtianyu]

Maybe you applied yaml before initialization.

yes, so if the ingress is stopped, and then it start again, it will not sync this data between the gap?

[AlinsRan]

Maybe you applied yaml before initialization.

yes, so if the ingress is stopped, and then it start again, it will not sync this data between the gap?

Yes, when it is started, it only watches the latest data and does not compare versions. Therefore, operations during this period will be lost.

[xiangtianyu]

IMO, this is a bug, ingress controller unavailable is unavoidable online.So it will certainly cause dirty data. This is inacceptable to me.

[xiangtianyu]

So, is there any suggestion to avoid this? Or it will be fixed in code?

[AlinsRan]

So, is there any suggestion to avoid this? Or it will be fixed in code?

My idea is that at startup, the controller deletes all apisix objects. Or more granular deletion. For example, delete resource label is managed-by: apisix-ingress-controller or crd: httpbin-route.

[xiangtianyu]

So, is there any suggestion to avoid this? Or it will be fixed in code?

My idea is that at startup, the controller deletes all apisix objects. Or more granular deletion. For example, delete resource label is managed-by: apisix-ingress-controller or crd: ApisixRoute.

But if do deletions like this, the online applications will be effected. In a short time, some requests cannot trans in correct way

[AlinsRan]

So, is there any suggestion to avoid this? Or it will be fixed in code?

My idea is that at startup, the controller deletes all apisix objects. Or more granular deletion. For example, delete resource label is managed-by: apisix-ingress-controller or crd: ApisixRoute.

But if do deletions like this, the online applications will be effected. In a short time, some requests cannot trans in correct way

By comparing resources, only redundant objects can be deleted.

[xiangtianyu]

Will it be added into the future version of apisix ingress controller?

[tao12345666333]

My idea is that at startup, the controller deletes all apisix objects. Or more granular deletion. For example, delete resource label is managed-by: apisix-ingress-controller or crd: httpbin-route.

No. This is too dangerous. and may lead to accidents

[AlinsRan]

Will it be added into the future version of apisix ingress controller?

It will be. I will fixes it as soon as possible.

[oyiadin]

Sorry but is this PR related? https://github.com/apache/apisix-ingress-controller/pull/680