Benjamin Zaitlen

Serialization is definitely something RAPIDS cares about. `scikeras` looks interesting -- @adriangb do you know if it forces a host to device transfer ? Does it support the `__cuda_array_interface__` ?...

It's possible the workers are trying to communicate with the scheduler using a public IP address rather than an internal IP addr. I am not as familiar with Azure, but...

We have had similar discussions on the GCP: https://github.com/dask/dask-cloudprovider/issues/229 https://github.com/dask/dask-cloudprovider/issues/215 . And we track internal ips from GCP: https://github.com/dask/dask-cloudprovider/blob/dffd81d3e4ab6feb5378ffa0d9952f3c7da207f7/dask_cloudprovider/gcp/instances.py#L302-L305 My guess is that worker should be connecting to the scheduler...

A number of skimage.transform APIs have already been converted: https://docs.rapids.ai/api/cucim/nightly/api.html#module-cucim.skimage.transform Is there something more specific you are looking for ?

I think for this to work we need to change: https://github.com/dask/dask-cloudprovider/blob/d5dfd99e0d29b1c6b03cb291633484fdd367ed4d/dask_cloudprovider/gcp/instances.py#L318-L330 To handle a spec which looks like the following: ```python workers_options = { 'worker-1': {"cls": "dask.distributed.Nanny", "opts": {"nthreads": 1}},...

You mean, have dask-cloudprovider inspect the system and assign worker/threads accordingly ?

+1 to making `dask.distributed.Nanny` the default.

https://github.com/NVIDIA/nvidia-gcp-samples/blob/master/bigquery-samples/dask-bigquery-connector/bigquery_dataproc_dask_xgboost.ipynb

@natemcintosh can you give a small reproducible example ? This would be extremely beneficial for those trying to help debug ? For help I would consult the following blog: https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports

@joshmoore do you know the minimum version of numpy used in the tests ? I believe [NEP-35](https://numpy.org/neps/nep-0035-array-creation-dispatch-with-array-function.html) was included in NumPy 1.20