cugraph
cugraph copied to clipboard
[FEA]: Support Cugraph dask algorithms without creating ucx endpoints.
Is this a new feature, an improvement, or a change to existing functionality?
Improvement
How would you describe the priority of this feature request
Medium
Please provide a clear description of problem this feature solves
Cugraph dask algorithms like wcc are used in downstream libraries like NeMo-Curator and require intializing comms with something like cugraph.dask.comms.comms.initialize(p2p=True).
Setting p2p=False is currently unsupported and p2p=True creates UCX endpoints to establish connections to workers even when working with a TCP based dask cluster.
Many systems & clusters have varying network topologies and require different UCX specific configurations to be set up correctly for the setup to work as expected & therefor p2p=True fails in many cluster unless the right UCX configurations are set.
It would improve usability if there was an option to run mnmg algorithms on a TCP cluster without any UCX config or endpoint creation.
Describe your ideal solution
cugraph.dask.comms.comms.initialize(p2p=False) # False works, no ucx created
# Proceed with mnmg algorithms.
Describe any alternatives you have considered
No response
Additional context
No response
Code of Conduct
- [X] I agree to follow cuGraph's Code of Conduct
- [X] I have searched the open feature requests and have found no duplicates for this feature request