envoy-swarm-control-plane icon indicating copy to clipboard operation
envoy-swarm-control-plane copied to clipboard

Prevent parallel service discovery runs

Open nstapelbroek opened this issue 3 years ago • 1 comments

When discovery is triggered almost simultaneously, it looks like you are able to break the resulting configuration. Chances of this happening are reduced significantly after https://github.com/nstapelbroek/envoy-swarm-control-plane/commit/c1e5d67635708e4513f9b36a3a065377380e1454 but I would love to see a more stable approach in the long-term as the update triggers are async by design.

nstapelbroek avatar Dec 21 '20 12:12 nstapelbroek

Although no weird behaviour, the logs still hint that we might be having multiple discovery runs in parallel. Would make sense to eliminate this behaviour with some tests:

envoy_control-plane.1.1ibzu8x0sp0o@x    | {"area":"snapshot-manager","level":"info","msg":"Running service discovery","reason":"a swarm service changed","time":"2021-04-21T06:00:00Z"}
envoy_control-plane.1.1ibzu8x0sp0o@x    | {"area":"snapshot-manager","level":"info","msg":"Running service discovery","reason":"a swarm service changed","time":"2021-04-21T06:00:00Z"}
envoy_control-plane.1.1ibzu8x0sp0o@x    | {"area":"snapshot-manager","level":"info","msg":"Running service discovery","reason":"a swarm service changed","time":"2021-04-21T06:00:05Z"}

nstapelbroek avatar Apr 21 '21 07:04 nstapelbroek