conf icon indicating copy to clipboard operation
conf copied to clipboard

Support using a Conf store from multiple processes

Open timoxley opened this issue 3 years ago • 4 comments

Currently if you have multiple processes using conf with the same storage location they can, and do, clobber each other's data.

e.g.

  1. p1 reads store
  2. p2 reads store
  3. p1 writes store
  4. p2 writes store (this clobbers p1's changes in step 3, even if the change was a totally different key)

Any suggestions on how to work around this?

I feel like this could warrant a big disclaimer in the readme.

timoxley avatar Mar 11 '21 02:03 timoxley

I added a note to the readme: https://github.com/sindresorhus/conf/commit/ce1dcee195e95cbd71493b1dfd6efcf7fe185a51

I simply never considered this use-case. It would be nice to support though, but it would require locking the store file during read/write, and the file locking story in Node.js is not great. We would also need to read the config from disk on write and merge it in before writing.

sindresorhus avatar Mar 19 '21 09:03 sindresorhus

it would require locking the store file during read/write, and the file locking story in Node.js is not great.

I had a crack at working around this using https://www.npmjs.com/package/lockfile but this was a non-starter since conf implements a sync interface and lockfile, "obviously" doesn't support waiting for a resource to become unlocked using synchronous operations.

For my use-case I (was) using a Map-interface wrapper around conf as the persistent cache for mem, so even if conf became async, it wouldn't solve my particular issue.

I wonder if there's some way to do a synchronous wait that doesn't involve assaulting the cpu/disk with a while loop. Perhaps some hackery with exec.execSync.

timoxley avatar Mar 22 '21 20:03 timoxley

I wonder if there's some way to do a synchronous wait that doesn't involve assaulting the cpu/disk with a while loop. Perhaps some hackery with exec.execSync.

There is, by Sindre: https://github.com/sindresorhus/make-synchronous

papb avatar Mar 23 '21 01:03 papb

Actually, https://github.com/sindresorhus/sleep-synchronously would be better for this purpose.

sindresorhus avatar Mar 23 '21 03:03 sindresorhus