K9db icon indicating copy to clipboard operation
K9db copied to clipboard

Dataflow Locks

Open KinanBab opened this issue 2 years ago • 0 comments

  • [ ] Dataflow parallelism may be superflous
  • [ ] Improve locks: a flow-wide lock for writers only, a reader-writer lock for matview
  • [ ] Fork Rocksdb and provide a commit that keeps locks unreleased.
  • [ ] Use fork to update dataflow while holding DB locks, then release everything together.

Forking Rocksdb:

  1. Remove the Clear() call from Commit() function in rocksdb/utilities/pessimistic_transaction.cc. Consider using a flag to disable/enable calling clear.
  2. Expose Clear() so that we can call it manually from Pelton after committing. Consider using destructor or rollback.
  3. Test via logging / monitoring of locks.
  4. Open issue/PR?

KinanBab avatar Apr 17 '23 03:04 KinanBab