dolt icon indicating copy to clipboard operation
dolt copied to clipboard

Broken concurrent access to mysql database file

Open Hydrocharged opened this issue 2 years ago • 0 comments

When multiple servers access the same mysql.db file (which currently houses all users and privileges data), each server only reads from the file when starting. Afterward, each server will dump its entire internal mysql database to the file. Assuming different users and privileges will be created and assigned on the different servers, the file will only contain the contents of the server who wrote last, and each server will be completely oblivious to changes made on other servers.

This can occur not only with multiple instances of dolt sql-server, but even with one instance of dolt sql-server and another of dolt sql.

Some ideas on fixing:

  • The first server creates a lock file. Other servers error if they find that lock file. May cause issues if the server crashes or the system unexpectedly shuts down and the lock file is not cleaned up.
  • Sync the internal mysql database to the file after every modification, and add a file watcher that reloads the database when it has been updated. Would also need to handle write/read races across processes.
  • Create some sort of interprocess communication so a new server can ask existing servers which files they use, throwing an error if the same file will be referenced multiple times. Does not have the same crash issue that the lock file has, but unknown how well this will work across all supported platforms.
  • Use the interprocess communication to send updates between servers without relying on a file watcher.

Hydrocharged avatar Jun 29 '22 01:06 Hydrocharged