pangeo-example-notebooks icon indicating copy to clipboard operation
pangeo-example-notebooks copied to clipboard

pangeo test example

Open Boes-man opened this issue 6 years ago • 1 comments

Hello, I have upgraded from 0.1.1-86665a6 to 19.03.05 in doing so I notice the example notebooks are gone. Specifically the dask-array.ipynb was very useful, since it allows for a easy, lightweight test of pangeo. Since the upgrade running this notebook errors as per below. Are you able to point me in the direction of how to fix it? Thanks

from dask_kubernetes import KubeCluster
cluster = KubeCluster(n_workers=10)
cluster

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-00757e3889c1> in <module>
      1 from dask_kubernetes import KubeCluster
----> 2 cluster = KubeCluster(n_workers=10)
      3 cluster

/srv/conda/lib/python3.6/site-packages/dask_kubernetes/core.py in __init__(self, pod_template, name, namespace, n_workers, host, port, env, **kwargs)
    178             msg = ("Worker pod specification not provided. See KubeCluster "
    179                    "docstring for ways to specify workers")
--> 180             raise ValueError(msg)
    181 
    182         self.cluster = LocalCluster(ip=host or socket.gethostname(),

ValueError: Worker pod specification not provided. See KubeCluster docstring for ways to specify workers

Boes-man avatar May 22 '19 01:05 Boes-man

We've decoupled the example notebooks (https://github.com/pangeo-data/pangeo-example-notebooks) from the helm chart and the pangeo-stacks images. This is really to help with long term maintenance and extensibility.

Your error seems to be saying you don't have a dask-kubernetes specification set. Can you report what print(dask.config.config['kubernetes']) reports? If you don't have a pod spec, perhaps you are missing a dask config in your setup (https://kubernetes.dask.org/en/latest/#quickstart)?

jhamman avatar May 22 '19 04:05 jhamman