dbscan-python icon indicating copy to clipboard operation
dbscan-python copied to clipboard

Can the memory footprint be reduced?

Open chi2liu opened this issue 3 years ago • 1 comments

With a 14178107*8 vector, a 108GB memory machine is quickly used up. Is there a way to reduce the memory footprint?

The train output:

Input: 14178107 points, dimension 8
scheduler = Parlay-HomeGrown
num-threads = 16
num-cell = 12333095
compute-grid = 5.06638

chi2liu avatar Jan 20 '22 06:01 chi2liu

Hi, I couldn't think of a straight forward way to optimize the memory usage at the moment. I think the high memory usage in your case may be related to a relatively small eps value that you are using. Alternatively, you may also want to try machines with larger memory such as AWS EC2.

wangyiqiu avatar Jan 25 '22 00:01 wangyiqiu