New user question: checking use-case

[a-jp]

Hi,

I've come across ArborX and it looks exactly like what I would need. Could I describe my use-case and would you be able to confirm that it's an appropriate library? I'd ideally like to do the following:

• [ ] Read in a list of points (x, y, z) and add them to ArborX. I guess the question here is, if run mpi 2-way, do I read 50% per processor myself, or can the library auto-partition the initial list as it sees fit? Or both?
• [ ] Add variables to each point. Ideally scalars and simple vector types (Eigen say), but I'm more than happy with a simple tuple of 3 doubles (thinking velocity here). I'd need to be able to add integer and double types also to each point as field variables. Is all of this possible?
• [ ] Loop over all points: for each point (already in the tree), find the closest N neighbours? So what I do not want is for a given random xyz, find the closest N neighbours already in the tree. What I need instead is the closest N points already in the tree, for a given point, already in the tree. If that makes sense, is the latter possible?
• [ ] If the above can work the way I need it, just confirming that parallel N-neighbour search works via MPI, which is my parallel use-case here (interest in OpenMP too), but mainly MPI?
• [ ] Finally, once the parallel N-neighbour search can be done, for a given existing point in the tree and returning its N-neighbours, can I access the field variables that I have assigned to those N-neighbours points, again, in parallel? So the neighbour may be on other proc and I want the knowledge of that neighbour, which is remote, and to access its field variables also?

Currently, I do the above in serial only with the point-locators and associated data-structures in VTK. I'd like to do the above with an MPI distributed tree, with data held per point, and the associated parallel read and write to the data held at each point along with the parallel N-neighbour search, where the search is conducted for the neighbours of an existing point in the tree.

Many thanks for the help, Andy

[aprokop]

Hi @a-jp, I'll try to answer your questions:

Read in a list of points (x, y, z) and add them to ArborX. I guess the question here is, if run mpi 2-way, do I read 50% per processor myself, or can the library auto-partition the initial list as it sees fit? Or both?

ArborX takes in the points you provide on each rank, and constructs a local tree based on those points. It does not do any data rebalancing. So, you would want to partition the points yourself before calling ArborX.

Add variables to each point. Ideally scalars and simple vector types (Eigen say), but I'm more than happy with a simple tuple of 3 doubles (thinking velocity here). I'd need to be able to add integer and double types also to each point as field variables. Is all of this possible?

Yes with caveats. More specifically, you can provide a function (we call it a callback) that will be given a query and an index of a primitive it intersects in. You can access your own data in that callback, ArborX does not need to know about it. You can also attach data to a query. However we disabled this functionality for nearest neighbor in the distributed setting. In a general case, we cannot guarantee calling a callback only on the exact nearest neighbors, and it may be called on some intermediaries. If there are side effects to those, the behavior would not be correct. See this comment for some discussion. Your case, however, should work even then, but we would need carefully about how to allow just this kind of behavior.

Loop over all points: for each point (already in the tree), find the closest N neighbours? So what I do not want is for a given random xyz, find the closest N neighbours already in the tree. What I need instead is the closest N points already in the tree, for a given point, already in the tree. If that makes sense, is the latter possible?

You can make queries based on the same points as the tree was constructed on, which will do what you want. You would still need to construct queries, however, they will be a simple wrapper around the data you used to construct the tree.

If the above can work the way I need it, just confirming that parallel N-neighbour search works via MPI, which is my parallel use-case here (interest in OpenMP too), but mainly MPI?

Parallel nearest neighbor search works with any MPI+X, and X can be serial or multithreaded or CUDA, etc

Finally, once the parallel N-neighbour search can be done, for a given existing point in the tree and returning its N-neighbours, can I access the field variables that I have assigned to those N-neighbours points, again, in parallel? So the neighbour may be on other proc and I want the knowledge of that neighbour, which is remote, and to access its field variables also?

See the comment about the callbacks above. It is currently disabled for a general case, but there is an opportunity for us to come back to this.

We have not revisited this code in a while due to lack of interest from users, but it could be a good time to get back it. We could start with working together to produce an example for the functionality you want.

[a-jp]

@aprokop Thank you for your comments. I'll take some time to try to understand the data on variables caveats. I'd be really keen to progress my understanding to see the level of impact for my use-case. Any examples you can provide to help with my questions/understanding would be enormously useful. I have a reasonable bit of experience with Kokkos. Likewise any additional clarity I can provide just ask.

[aprokop]

@a-jp We can probably do a quick hack to allow callbacks for nearest to see the effect for you problem. Could you please provide more details on the problem you are trying to solve? For example, its size, number of ranks, number of neighbors, number of field variables you are trying to get. I'm also interested in hearing what you do with the field variables once you acquire them, as I assume that these are only intermediate values that are then used to produce some quantity of interest.

[aprokop]

Closing for inactivity. We also now have MLS implemented, though the distributed part has not been merged yet.