ipykernel icon indicating copy to clipboard operation
ipykernel copied to clipboard

Various tests fail due to starting kernel on wrong Python version

Open mgorny opened this issue 4 years ago • 6 comments

With ipykernel-6.0.1:

$ PYTHONPATH=. python3.8 -m pytest -x ipykernel/tests/test_async.py 
========================================================= test session starts =========================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /tmp/ipykernel-6.0.1
plugins: timeout-1.4.2, case-1.5.3, xprocess-0.17.1, mock-3.6.1, localserver-0.5.0, expect-1.1.0, httpx-0.12.0, asyncio-0.15.1, snapshottest-0.5.1, forked-1.3.0, freezegun-0.4.2, datadir-1.3.1, metadata-1.8.0, requests-mock-1.9.3, subtests-0.5.0, betamax-0.8.1, regressions-2.2.0, pyfakefs-4.5.0, httpbin-1.0.0, django-4.4.0, shutil-1.7.0, virtualenv-1.7.0, subtesthack-0.1.2, pkgcore-0.12.1, xdist-2.3.0, pylama-7.7.1, cov-2.12.1, xdoctest-0.15.4, anyio-3.2.1, hypothesis-6.14.1, rerunfailures-10.1, flaky-3.7.0, services-2.2.1, aiohttp-0.3.0, Faker-8.10.0, lazy-fixture-0.6.3, xvfb-2.0.0, trio-0.7.0
collected 4 items                                                                                                                     

ipykernel/tests/test_async.py E

=============================================================== ERRORS ================================================================
_________________________________________________ ERROR at setup of test_async_await __________________________________________________

    def setup_function():
        """start the global kernel (if it isn't running) and return its client"""
        global KM, KC
>       KM, KC = start_new_kernel()

ipykernel/tests/test_async.py:20: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ipykernel/tests/utils.py:37: in start_new_kernel
    return manager.start_new_kernel(startup_timeout=STARTUP_TIMEOUT, **kwargs)
/usr/lib/python3.8/site-packages/jupyter_client/manager.py:919: in start_new_kernel
    kc.wait_for_ready(timeout=startup_timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <jupyter_client.blocking.client.BlockingKernelClient object at 0x7fc81942b430>, timeout = 60

    def wait_for_ready(self, timeout=None):
        """Waits for a response when a client is blocked
    
        - Sets future time for timeout
        - Blocks on shell channel until a message is received
        - Exit if the kernel has died
        - If client times out before receiving a message from the kernel, send RuntimeError
        - Flush the IOPub channel
        """
        if timeout is None:
            abs_timeout = float('inf')
        else:
            abs_timeout = time.time() + timeout
    
        from ..manager import KernelManager
        if not isinstance(self.parent, KernelManager):
            # This Client was not created by a KernelManager,
            # so wait for kernel to become responsive to heartbeats
            # before checking for kernel_info reply
            while not self.is_alive():
                if time.time() > abs_timeout:
                    raise RuntimeError("Kernel didn't respond to heartbeats in %d seconds and timed out" % timeout)
                time.sleep(0.2)
    
        # Wait for kernel info reply on shell channel
        while True:
            self.kernel_info()
            try:
                msg = self.shell_channel.get_msg(block=True, timeout=1)
            except Empty:
                pass
            else:
                if msg['msg_type'] == 'kernel_info_reply':
                    # Checking that IOPub is connected. If it is not connected, start over.
                    try:
                        self.iopub_channel.get_msg(block=True, timeout=0.2)
                    except Empty:
                        pass
                    else:
                        self._handle_kernel_info_reply(msg)
                        break
    
            if not self.is_alive():
>               raise RuntimeError('Kernel died before replying to kernel_info')
E               RuntimeError: Kernel died before replying to kernel_info

/usr/lib/python3.8/site-packages/jupyter_client/blocking/client.py:111: RuntimeError
-------------------------------------------------------- Captured stdout setup --------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/tmp/ipykernel-6.0.1/ipykernel_launcher.py", line 15, in <module>
    from ipykernel import kernelapp as app
  File "/tmp/ipykernel-6.0.1/ipykernel/kernelapp.py", line 42, in <module>
    from .ipkernel import IPythonKernel
  File "/tmp/ipykernel-6.0.1/ipykernel/ipkernel.py", line 20, in <module>
    from .debugger import Debugger
  File "/tmp/ipykernel-6.0.1/ipykernel/debugger.py", line 14, in <module>
    import debugpy
ModuleNotFoundError: No module named 'debugpy'
======================================================= short test summary info =======================================================
ERROR ipykernel/tests/test_async.py::test_async_await - RuntimeError: Kernel died before replying to kernel_info
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================== 1 error in 2.83s ===========================================================

If add some debug, it turns out that the underlying cause is that the kernel is started using python3.10 rather than python3.8 (and debugpy doesn't support py3.10). This might be a problem in jupyter_client (version 6.1.12 here) but figuring out where it forces python3.10 is above my pay grade.

mgorny avatar Jul 11 '21 12:07 mgorny

This looks like a Python configuration issue in your environment.

dsblank avatar Jul 11 '21 15:07 dsblank

This looks like a Python configuration issue in your environment.

Could you be more specific, please?

mgorny avatar Jul 11 '21 15:07 mgorny

$ python3.8
Python 3.8.11 (default, Jul  1 2021, 10:04:25) 
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/usr/bin/python3.8'
>>> sys.version_info
sys.version_info(major=3, minor=8, micro=11, releaselevel='final', serial=0)

mgorny avatar Jul 11 '21 15:07 mgorny

I only note that the last error traceback starts with "runpy.py" in the Python 3.10 libraries. It looks like something is set on your system to use Python 3.10. Can you remove your Python 3.10 and try again?

dsblank avatar Jul 11 '21 15:07 dsblank

Telling people 'do not install Python 3.10 or ipykernel will randomly be broken' is not really a solution.

mgorny avatar Jul 11 '21 15:07 mgorny

I'm not telling you to not install Python 3.10. I was trying to help you debug your environment. I'm just a volunteer that thought I'd lend a helpful insight om a weekend. Good luck!

dsblank avatar Jul 11 '21 16:07 dsblank