BBHash
BBHash copied to clipboard
Relative tmp address
Hi @rizkg and others,
First of all, thank you for your awesome MPHF implementation. It's fast, and easy to use.
There is a need for generating some temporary files on disk during the construction. But the address used for temp files is relative to where one is running the code from. This happened to make our run fail a couple of times, while we were running the code from a disk that was full but giving the output address on another disk that had enough space (also, the binary was not in the same disk as the one we were running our program from). That took us a while to wonder what is causing this failure.
If not wrong, I think the part that one can change root address for temp files to where the final output MPHF is stored should be here: https://github.com/rizkg/BBHash/blob/master/BooPHF.h#L1377 We fixed this by allowing a prefix for the temporary directory to be passed to the constructor. But you as authors might have a better way to do it.
Either way, it seems it would be useful to allow, at least optionally, to override the current behavior.
Thanks, Fatemeh
Hi Fatemeh, giving a prefix to the constructor sounds to me like a great way to fix this.
I'll note also that BBHash has a mode where it doesn't use the disk. I believe it's writeEach=false
in the constructor
Thanks @rchikhi , you're right. It has the mode. But I thought this makes it use maximum memory and we would like to limit memory usage by allowing intermediate disk usage.