cuspatial
cuspatial copied to clipboard
[skip-ci] [Do Not Merge] Experiment on `iterator_collections` structure
Description
This PR demonstrates the idea of a iterator_collections structure, in an attempt to simplify the header only API interface.
In arrow LIST type structure, there might be nested iterables that utilizes a random accessible offset array to index into child arrays. Currently, cuspatial exposes the APIs by accepting a "flattened" view/iterator list of the nested columns.
This can easily make the input cumbersome. To show case the complexity, a multipolygon-multipolygon distance API may look like:
auto polygon_distance(
OffsetIteratorA polygonA_geometry_offsets_begin,
OffsetIteratorA polygonA_geometry_offsets_end,
OffsetIteratorB polygonA_parts_offsets_begin,
OffsetIteratorB polygonA_parts_offsets_end,
OffsetIteratorC polygonA_ring_offsets_begin,
OffsetIteratorC polygonA_ring_offsets_end,
VecIterator polygonA_coordinates_begin,
VecIterator polygonA_coordinates_end,
OffsetIteratorD polygonB_geometry_offsets_begin,
OffsetIteratorE polygonB_parts_offsets_begin,
OffsetIteratorE polygonB_parts_offsets_end,
OffsetIteratorF polygonB_ring_offsets_begin,
OffsetIteratorF polygonB_ring_offsets_end,
VecIterator polygonB_coordinates_begin,
VecIterator polygonB_coordinates_end,
OutputIterator distance_begin,
rmm::cuda_stream_view stream
)
An iterator_collection packages up the inputs of different levels of input in a structure and provides different level of iteration to thrust algorithms. Ideally this may simplify the API to be like:
auto polygon_distance(
iterator_collections::multipolygon_array<...> multipolygon1
iterator_collections::multipolygon_array<...> multipolygon2
OutputIterator distance_begin,
rmm::cuda_stream_view stream
)
An iterator_collection can also provide different levels of iterators to allow developers to write kernel at their desired paralleled level. For example:
The below computes the multipoint distance with pair-wise parallel model.
thrust::transform(
multipoint1.geometry_begin(), multipoint1.geometry_end(), multipoint2.geometry_begin(), distances_first, multipoint_wise_distance_op{});
and the below computes the distance with point-wise parallel model with a custom kernel.
point_wise_distance_kernel<<<...>>>(
multipoint1.geometry_offset.begin(),
multipoint1.geometry_offset.end(),
multipoint1.point_offset.begin(),
multipoint1.point_offset.end(),
multipoint2.geometry_offset.begin(),
multipoint2.geometry_offset.end(),
multipoint2.point_offset.begin(),
multipoint2.point_offset.end(),
)
Update 09/23
After pairing with @thomcom, we introduced multilinestring_array iterator collections and adopted this towards point_linestring_distance kernel. This iterator collections supports traversing upwards in the nested hierarchy. Point-linestring distance kernel is launched upon each point in the multilinestring, thus the newly introduced structure should provide ways to retrieve the part/geometry index of the point that current thread operates on, this is essential to finding the corresponding multipoint in the pair. What this has demonstrated is that:
- Several explicit dereference in the original kernel can be abstracted nicely in the
multilinestring_arraystructure. - Some restriction in thrust makes dereferencing a bit cumbersome in kernel, but when wrapped with the new structure, the return type of the method can be made explicit and thus can make use of the implicit cast from
thrust::device_reference<T> -> T&, further simplifying the kernel code path. - Range based for loop is more succinct than nested
for_each.
Checklist
- [ ] I am familiar with the Contributing Guidelines.
- [ ] New or existing tests cover these changes.
- [ ] The documentation is up to date with these changes.