jupyter-locate-python: No `python' found in search paths

[fleimgruber]

Running on Windows, Emacs gets these paths from jupyter --paths (the jupyter for that is from C:\Users\user\myenv\.venv\Scripts\jupyter.exe):

config:
    C:\Users\user\.jupyter
    C:\Users\user\myenv\.venv\etc\jupyter
    C:\ProgramData\jupyter
data:
    C:\Users\user\AppData\Roaming\jupyter
    C:\Users\user\myenv\.venv\share\jupyter
    C:\ProgramData\jupyter
runtime:
    C:\Users\user\AppData\Roaming\jupyter\runtime

in which there are none of '("bin/python" "bin/python3" "python3.exe" "python.exe"). I am wondering why it's implemented like that as I would not expect a Python executable present in any of these paths? I would assume that C:\Users\user\myenv\.venv\Scripts\python.exe would be found. I updated from an older version of emacs-jupyter and I am guessing that this could be due to the big changes in 1fb77da2d394b905f9dfa325a9d6ef9e2cf52541?

[nnicandro]

There have not been any changes to jupyter-locate-python since 9dd8e8d9ec6765ace45753c138bc29b5aeec3337 which was before the recent major changes by a while.

I'm not sure why I believed that the Python should be located in one of the data directories of the jupyter --paths command. From the documentation, the data directories are for Jupyter specific data files like kernelspecs. From the test of jupyter-locate-python, I may have settled on that solution since it worked for environments (conda or otherwise) on Unix-like systems. On Windows, I don't think I ever tested it with a conda based Jupyter.

Is it possible that conda on Windows has changed to having python.exe located in that Scripts directory? If so, I'd welcome a pull request which handles that case.

[fleimgruber]

Thanks for the heads-up.

I don't use conda as default on Windows (I only use it occasionally for compatibility with some projects via it's own "Anaconda Prompt"). I install Python via https://scoop.sh/. So there is a global Python on $PATH at C:\Users\user\scoop\apps\python\current\python. I installed jupyter with pip into C:\Users\user\scoop\apps\python\current\Scripts\jupyter. I am using Poetry to manage venvs for projects, e.g. the mentioned C:\Users\user\myenv\.venv\Scripts\python.exe.

I install Python kernels for jupyter via poetry run python -m ipykernel install --user --name kernel_name.

I did not consciously change $PATH in recent time, but maybe some scoop command did? My assumption was that emacs-jupyter only uses jupyter itself which is on $PATH?

[fleimgruber]

Due to problems between scoop and pyenv-win, I now installed pyenv-win myself hoping that this issue would go away. But I see the same behavior. I saw a code comment about Windows, but it is so strange that this part of the code is 4 years old already but always worked for me?

@jkitchin since I remember you also use this on Windows, any thoughts?

[timlod]

FWIW, I also started having issues with that function around the time you opened this issue, although on Linux. I don't remember the details now, but I had to remove "/bin/python3" from the list given (error would be jupyter-session-with-random-ports: ‘jupyter kernel‘ failed to show connection file path.).

Probably unrelated, but given the timing I reckon something somewhere else might have changed that influenced how jupyter-locate-python works?

[fleimgruber]

@timlod Thanks for tuning in. I tried your suggestion, it does not work for me. But it would be great if you could test on Linux what jupyter --paths returns for you, and if there is a python executable in any of these directories? Thanks!

Probably unrelated, but given the timing I reckon something somewhere else might have changed

That's what I tought as well and why I suspected 1fb77da2d394b905f9dfa325a9d6ef9e2cf52541.

[fleimgruber]

As a workaround I just went in and changed jupyter-locate-python to just evaluate (setq path (file-local-name (executable-find "python"))) if on Windows.

[timlod]

@fleimgruber Sorry - I totally forgot to respond here, and only see it now because I'd been having issues with this again... On Linux jupyter --paths also don't return anything with a python binary.

Sometimes the function (jupyter-locate-python) works though, and other times it doesn't really? My original fix to just remove the python3 reference doesn't work on another machine of mine, so I'm also just using your workaround there. Seems to be the most sane as well tbh, just use the python in PATH and be done with it.

[fleimgruber]

@timlod never mind anyway! Happy to hear it works for you as well.