BBHash icon indicating copy to clipboard operation
BBHash copied to clipboard

Relative tmp address

Open fataltes opened this issue 4 years ago • 3 comments

Hi @rizkg and others,

First of all, thank you for your awesome MPHF implementation. It's fast, and easy to use.

There is a need for generating some temporary files on disk during the construction. But the address used for temp files is relative to where one is running the code from. This happened to make our run fail a couple of times, while we were running the code from a disk that was full but giving the output address on another disk that had enough space (also, the binary was not in the same disk as the one we were running our program from). That took us a while to wonder what is causing this failure.

If not wrong, I think the part that one can change root address for temp files to where the final output MPHF is stored should be here: https://github.com/rizkg/BBHash/blob/master/BooPHF.h#L1377 We fixed this by allowing a prefix for the temporary directory to be passed to the constructor. But you as authors might have a better way to do it.

Either way, it seems it would be useful to allow, at least optionally, to override the current behavior.

Thanks, Fatemeh

fataltes avatar Apr 10 '20 20:04 fataltes

Hi Fatemeh, giving a prefix to the constructor sounds to me like a great way to fix this.

rchikhi avatar Apr 11 '20 10:04 rchikhi

I'll note also that BBHash has a mode where it doesn't use the disk. I believe it's writeEach=false in the constructor

rchikhi avatar Apr 11 '20 10:04 rchikhi

Thanks @rchikhi , you're right. It has the mode. But I thought this makes it use maximum memory and we would like to limit memory usage by allowing intermediate disk usage.

fataltes avatar Apr 17 '20 19:04 fataltes