Request for API support for requesting compaction and waiting for it to finish (feature request)

[mdcallag]

db_bench has flush, compact0 and compact1 commands to request flushing the memtable, compacting from L0->L1 and from L1->L2. It also has waitforcompaction to wait for compaction to finish.

MyRocks has similar commands where the SQL interface is to set a global variable:

set global rocksdb_force_flush_memtable_now=1
set global rocksdb_force_flush_memtable_and_lzero_now=1;

There are problems and I prefer that these be implemented within RocksDB:

• MyRocks has a race in rocksdb_force_flush_memtable_and_lzero_now, see here.
• waitforcompaction in db_bench works for leveled but has intermittent hangs for universal
• some of the db_bench commands have had bugs -- I know because I am the author of those bugs

[jay-zhuang]

Would exposing DBImpl::WaitForCompact() as a public API help? (currently it's for internal test only). Or is there any other requirement? https://github.com/facebook/rocksdb/blob/27f3af596609f3b86af5080cc870c869e836e7f2/db/db_impl/db_impl.h#L1032-L1035

[mdcallag]

That would help. Does it also wait for memtable flush?

[jay-zhuang]

Yes, it will wait for flush too.

[ajkr]

Is it fixed by #10524?

[mdcallag]

Just saw your comment. Will revisit this soon.

[ajkr]

I did not realize at the time of writing that comment that the linked PR would not land. Now it looks like #11436 is more likely to provide the solution.

[jaykorean]

@ajkr I'm wondering if we want to close #10524 to avoid any confusion in the future.

[jaykorean]

WaitForCompact() - #11436 was released with 8.4.0.