test_jupyter_path_prefer_env fails

[mgorny]

I can reproduce this test failure both with 4.7.0 and git master (which is practically the same thing ;-)):

__________________________________________________________ test_jupyter_path_prefer_env ___________________________________________________________

    def test_jupyter_path_prefer_env():
        with patch.dict('os.environ', {'JUPYTER_PREFER_ENV_PATH': 'true'}):
            path = jupyter_path()
>       assert path[0] == paths.ENV_JUPYTER_PATH[0]
E       AssertionError: assert '/home/mgorny...share/jupyter' == '/usr/share/jupyter'
E         - /usr/share/jupyter
E         + /home/mgorny/.local/share/jupyter

jupyter_core/tests/test_paths.py:186: AssertionError

[kevin-bates]

Hi @mgorny. Do you happen to run with JUPYTER_PATH defined? This will side-effect that particular test. FWIW, I can only reproduce the issue if I set JUPYTER_PATH.

@jasongrout - would it make sense to have some remove/restore logic for JUPYTER_PATH in that test? Perhaps it could issue a warning if it finds it set, prior to its temporary removal?

[mgorny]

Hi @mgorny. Do you happen to run with JUPYTER_PATH defined? This will side-effect that particular test.

No, I don't have it set in the environment. Also, the mismatched path changes if I change HOME, so it seems to be established relative to the home directory.

[kevin-bates]

Sorry about this Michał. I'm hoping Jason can take a look at this. In the meantime, could you please post the results of:

jupyter --paths --debug

[kevin-bates]

Hmm - I suspect your ENV and SYSTEM paths are the same in that your python is installed is /usr/share/jupyter - which will also side-effect the test.

[mgorny]

$ jupyter --paths --debug
JUPYTER_PREFER_ENV_PATH is not set, making the user-level path preferred over the environment-level path for data and config
JUPYTER_NO_CONFIG is not set, so we use the full path list for config
JUPYTER_CONFIG_PATH is not set, so we do not prepend anything to the config paths
JUPYTER_CONFIG_DIR is not set, so we use the default user-level config directory
JUPYTER_PATH is not set, so we do not prepend anything to the data paths
JUPYTER_DATA_DIR is not set, so we use the default user-level data directory
JUPYTER_RUNTIME_DIR is not set, so we use the default runtime directory

config:
    /home/mgorny/.jupyter
    /usr/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/mgorny/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/mgorny/.local/share/jupyter/runtime

[mgorny]

I am using a regular system IPython install (i.e. /usr/bin/python3.x, /usr/lib/python3.x etc.), and I have Jupyter installed to system site-packages with standard paths. I'll try uninstalling jupyter_core.

[mgorny]

Uninstalling it doesn't help but also makes two more tests fail.

[kevin-bates]

What does this python snippet produce?

import sys
sys.prefix

If either /usr/local/share/jupyter or /usr/share/jupyter, then that's the issue.

[jasongrout]

Hmm - I suspect your ENV and SYSTEM paths are the same in that your python is installed is /usr/share/jupyter - which will also side-effect the test.

Those lines were duplicating the logic from before:

https://github.com/jupyter/jupyter_core/blob/4c072dcf51cede70ca41721b811ad9b3581bdd84/jupyter_core/paths.py#L152-L154

I think it is the test that is wrong. What do we expect to happen if we set to prefer env paths, but really there are no env paths because they are system paths? I expect that the user will be the first, but the test is trying to insist on sysprefix (even though it is a system path).

So I think the test should also check to see if there are env paths that are not system paths, and if there are not any, it should see if the user path is first. Also, I agree with you that the test should temporarily reset the JUPYTER_PATH variable to cut out that side effect.

[mgorny]

$ python
Python 3.9.0 (default, Oct  6 2020, 09:42:04) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/usr/lib/python-exec/python3.9/../../..'

[jasongrout]

If either /usr/local/share/jupyter or /usr/share/jupyter, then that's the issue.

This answers it as well:

data:
    /home/mgorny/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter

There are no non-system environment paths...

[mgorny]

Also:

>>> sys.path
['', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/mgorny/.local/lib/python3.9/site-packages', '/usr/lib/python3.9/site-packages']