empire icon indicating copy to clipboard operation
empire copied to clipboard

Possible race between setting config and deploying

Open ejholmes opened this issue 8 years ago • 1 comments

We saw an instance today where a config var was set, then a deployment went out around the same time. emp releases shows something like this:

v1234 Set FOO config var
v1235 Deploy remind101/app

However, emp env didn't show FOO. It appears that v1235 used the Config object from v1233 instead of v1234. Creating releases already should be acquiring a lock to prevent a race condition like this.

ejholmes avatar Apr 19 '17 23:04 ejholmes

So, I think this is a legit race between deploying and setting config. When we deploy, we open a transaction and then lock the releases table to ensure that deployments are always sequential. However, this lock doesn't prevent Config objects from being changed. So it's still possible to grab the config from a previous release.

ejholmes avatar Apr 19 '17 23:04 ejholmes