dask-gateway
dask-gateway copied to clipboard
dask
A multi-tenant server for securely deploying and managing Dask clusters.
Hi, I have dask-gateway set up on our hadoop cluster with yarn. Without establishing a connection to the gateway, a simple csv read from hdfs works perfectly. ``` df =...
There is mention of the `dask_gateway_config.py` file in various parts of the documentation, but I don't think there is any actual mention of how to create this file, or where...
## Issue summary I ended up with active but unused worker pods, unusable by the scheduler, blocking creation of new pods if a DaskCluster needs to scale up, but unable...
Over in https://github.com/pangeo-data/pangeo-binder/issues/143, we've noticed that some dask-gateway pods hang around. The root cause is likely something like https://github.com/ipython/ipykernel/issues/462. But even if that's fixed, there might be cases like a...
We have some simple dev docs (https://gateway.dask.org/develop.html). We should expand these to include more detail on developing the kubernetes backend. Including: - Setting up a local dev environment - Using...
If a cluster is created, and the scheduler crashes for some reason, for example because it contained an old version of dask-gateway, that is an error very hard to get...
I'm curious about the get_client method. If possible I would like to unify APIs with upstream, either by removing this method, or by moving it to the Cluster superclass? Any...
This is more of a question than an issue (my apologies). I would like to better understand the motivations behind the adaptive logic in dask_cli.py . Some of this seems...
I've encountered this situation a few times now where I realize right after I start a computation that my worker size is too small and I know the task graph...
In #54 we added an API that allows users to configure admin-defined fields when starting a cluster. This API provides an endpoint for discovering what options exist (name, type, description)....