hpdbscan icon indicating copy to clipboard operation
hpdbscan copied to clipboard

Published 20 hours ago • verified publisher icon Markus-Goetz

Memory issue

Open daodaofr opened this issue 4 years ago • 1 comments

Great work, thanks.

I have an issue about the memory. In the example, the features have small dimension, eg, 3. When I try larger dimension, I will face out of memory issue. e.g., I tried data = np.random.rand(100, 300) it takes more than 100G memory.

Do you have any idea about this issue ? Thanks.

daodaofr avatar Jul 02 '20 14:07 daodaofr

Hi @daodaofr,

I am aware of this issue. It has something to do with the way high numbers of dimensions are handled internally. Consider it a bug for now. I am looking into it. For now, however, it is simply not possible to process this or you have to buy a machine with a lot of RAM :P.

Cheers

Markus-Goetz avatar Jul 10 '20 07:07 Markus-Goetz