NearestNeighbors.jl icon indicating copy to clipboard operation
NearestNeighbors.jl copied to clipboard

Dynamic trees

Open bkalpert opened this issue 8 years ago • 3 comments

This package has excellent performance for queries on a static set of points. For streaming data, with points added and earliest points removed to limit storage size, can similar performance be achieved through an extension of the implementation?

Procopiuc, Agarwal, Arge, and Vitter, "Bkd-tree: A dynamic scalable kd-tree" (2003), seem to observe that this is possible, by avoiding memory I/O problems of earlier methods. Is there interest in testing their ideas with an implementation?

bkalpert avatar Apr 11 '17 16:04 bkalpert

Right now, the staticness of trees is used quite extensively for performance optimizations. I think that dynamic tress would require a whole new approach to the data structures used, so most of it would have to be written from scratch. It would of course be nice to have dynamic trees in the package but it is significant work. I will not be working on this but will of course review PRs that adds this feature in a way that is consistent with the current "API".

KristofferC avatar Apr 12 '17 06:04 KristofferC

It is quite apparent that a great deal of optimization work is present in the static implementation, but also that significant extensions will be needed for the proposed capability. I will experiment, initially with a primitive streaming implementation, to assess the magnitude and feasibility of the task. Thank you for being open to the possibility.

bkalpert avatar Apr 12 '17 10:04 bkalpert

Thanks @KristofferC for your great work on this package. I am wondering if you or anyone else has any recommendations for a Julia package to use for dynamic k-d trees (with the assumption that all data points will be uniform within some bounds [0,1]^k)? I'm not able to find any existing package. Much appreciated!

clu8 avatar Jun 05 '17 22:06 clu8