h2o4gpu
h2o4gpu copied to clipboard
Issue getting a working python notebook in Jupyter docker images
Following the instructions in section Running Jupyter Notebooks with Docker of the README file results in the below kernel error when trying to launch a Python 3 notebook from the Jupyter interface.
Traceback (most recent call last):
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/traitlets/traitlets.py", line 528, in get
value = obj._trait_values[self.name]
KeyError: 'loop'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/web.py", line 1592, in _execute
result = yield result
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
yielded = self.gen.throw(*exc_info)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/notebook/services/sessions/handlers.py", line 73, in post
type=mtype))
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
yielded = self.gen.throw(*exc_info)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 79, in create_session
kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
yielded = self.gen.throw(*exc_info)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 92, in start_kernel_for_session
self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/tornado/gen.py", line 326, in wrapper
yielded = next(result)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/notebook/services/kernels/kernelmanager.py", line 160, in start_kernel
super(MappingKernelManager, self).start_kernel(**kwargs)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel
km.start_kernel(**kwargs)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/jupyter_client/manager.py", line 258, in start_kernel
self.start_restarter()
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/jupyter_client/ioloop/manager.py", line 49, in start_restarter
kernel_manager=self, loop=self.loop,
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/traitlets/traitlets.py", line 556, in __get__
return self.get(obj, cls)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/traitlets/traitlets.py", line 535, in get
value = self._validate(obj, dynamic_default())
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/traitlets/traitlets.py", line 591, in _validate
value = self.validate(obj, value)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/traitlets/traitlets.py", line 1677, in validate
self.error(obj, value)
File "/opt/h2oai/h2o4gpu/python/lib/python3.6/site-packages/traitlets/traitlets.py", line 1524, in error
raise TraitError(e)
traitlets.traitlets.TraitError: The 'loop' trait of an IOLoopKernelManager instance must be a ZMQIOLoop, but a value of class 'tornado.platform.asyncio.AsyncIOMainLoop' (i.e. <tornado.platform.asyncio.AsyncIOMainLoop object at 0x7ff535b32f98>) was specified.
Related to customer support issue https://support.h2o.ai/helpdesk/tickets/92731
Relates to jupyterhub/jupyterlab-hub#62 and ipython/ipyparallel#318
To unblock the customer, I created a custom docker image using h2o4gpu conda packages, available on the internal conda server, to create the notebook environment with h2o4gpu support. Once the conda env was created i downgraded tornado from 5.1 to 4.5.3 in that environment and everything worked as expected.
Trying that downgrade approach in the env created by pip (requirements.txt) in the docker runtime created other errors.
Possible next step is that once we have h2o4gpu release completed to anaconda cloud, we can update the docker runtime script to create a conda based environment vs the pip based one. I has better dependency management.
@hemenkapadia did this work for the other person's case?
I am waiting to hear back from the customer. She mentioned she was going to try this today, but that was a custom conda build. I will need to check the one with the new ipykernel.