storr icon indicating copy to clipboard operation
storr copied to clipboard
verified publisher icon richfitz

Question: simultaneous writes to non-RDS storrs

Open wlandau opened this issue 7 years ago • 2 comments

Just wondering: at some point, do you plan to ensure the thread safety of alternative drivers? It would be nice to have a DBI driver with protections against race conditions.

wlandau avatar Nov 25 '18 06:11 wlandau

I think it depends what you mean about race conditions. From the point of view of corruption

  • redis is totally safe for concurrent writes
  • DBI should be safe (modulo issues with SQLite and filesytstems)
  • thor is safe with a real filesystem, single host
  • environment store is single use only

richfitz avatar Dec 13 '18 13:12 richfitz

I am concerned specifically about situations in which different processes, either on the same host or different remote nodes, try to write different targets to the same cache at the same time. The tips about redis and thor are helpful. (Is thor safe when the processes are on different nodes?) I have had trouble with DBI/SQLite storrs when caching is set to "worker" here.

I think my SQLite installation just is not up to par. I had similar issues with liteq (which claims to be thread safe) which is the reason why I wrote txtq.

wlandau avatar Dec 13 '18 16:12 wlandau