cloud-run-button icon indicating copy to clipboard operation
cloud-run-button copied to clipboard

Race condition preventing successful integration tests

Open glasnt opened this issue 3 years ago • 0 comments

There are many tests/ but one of the first ones (hooks-prepostcreate-inline) sees envvars being updated for the service.

The problem is that it appears there can be a time when the old revision is accepting requests, when the new revision should be.

In this example, there is a revision that should return "A" and a revision that should return "AB" (through inline pre- and post-create commands) We should only be testing the service after the deployment is complete, so we should be only ever returning the latest revision ("AB").

But polling the service once a second as it deploys shows there's times when "A" returns to the mix:

Mon Dec  5 04:29:51 UTC 2022
A
Mon Dec  5 04:29:54 UTC 2022
A
Mon Dec  5 04:29:56 UTC 2022
A
Mon Dec  5 04:29:57 UTC 2022
AB
Mon Dec  5 04:29:59 UTC 2022
AB
Mon Dec  5 04:30:00 UTC 2022
A
Mon Dec  5 04:30:01 UTC 2022
AB
Mon Dec  5 04:30:02 UTC 2022
AB

Pending matching against the service logs to establish it was an older revision receiving the request, etc.

glasnt avatar Dec 05 '22 04:12 glasnt