Lecture names cannot be the same as a Package name (when imported in the code)

[mmcky]

Describe the bug

When building this project I am getting some strange results when trying to execute notebooks in _build/jupyter_execute. The cell magic in one lecture is trying to import from a notebook file of the same name matplotlib (due to presence of matplotlib.py).

The error message in python_by_example.ipynb when run in that context is:

Traceback (most recent call last):

  File "anaconda3/envs/tomyst/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-1-9e3324102725>", line 1, in <module>
    get_ipython().run_line_magic('matplotlib', 'inline')

  File "anaconda3/envs/tomyst/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 2326, in run_line_magic
    result = fn(*args, **kwargs)

  File "<decorator-gen-108>", line 2, in matplotlib

  File "anaconda3/envs/tomyst/lib/python3.8/site-packages/IPython/core/magic.py", line 187, in <lambda>
    call = lambda f, *a, **k: f(*a, **k)

  File "anaconda3/envs/tomyst/lib/python3.8/site-packages/IPython/core/magics/pylab.py", line 99, in matplotlib
    gui, backend = self.shell.enable_matplotlib(args.gui.lower() if isinstance(args.gui, str) else args.gui)

  File "anaconda3/envs/tomyst/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3419, in enable_matplotlib
    gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)

  File "anaconda3/envs/tomyst/lib/python3.8/site-packages/IPython/core/pylabtools.py", line 280, in find_gui_and_backend
    import matplotlib

  File "repos-collab/lecture-python-programming.myst/_build/jupyter_execute/matplotlib.py", line 1
    ```{raw} html
    ^
SyntaxError: invalid syntax

If you run the python_by_example.ipynb lecture in a separate folder this doesn't happen.

The jupyter_execute folder contains a lot of py files

▶ ls *.py
404.py                               index.py                             matplotlib.py                        search.py
about_lectures.py                    index_advanced_python_programming.py numba.py                             status.py
about_py.py                          index_learning_python.py             numpy.py                             troubleshooting.py
functions.py                         index_python_scientific_libraries.py parallelization.py                   writing_good_code.py
getting_started.py                   index_toc.py                         scipy.py

causing the issue. The contents of the py files are actually md so this extension is incorrect.

@chrisjsewell is this a config issue on my end.

To Reproduce

1. git clone https://github.com/QuantEcon/lecture-python-programming.myst.git
2. git checkout add-execution
3. make clean && make html
4. cd _build/jupyter_execute
5. run python_by_example.ipynb

Environment

• Python Version [e.g. 3.7.1]:
• Package versions or output of jupyter-book --version: myst_nb = 0.6, jupyter-cache=0.4.1
• Operating System: OS X

[chrisjsewell]

This is a myst-nb issue really

The .py files are output as script files by jupyter-sphinx, but are currently erroneous and I’ve been waiting for a while for thee fix to be merged. The notebooks in that folder are not necessarily intended to be run from there, but I can see why you might want to. It’s a bit of a fluke that your notebook happens to be named the same as one of your imports; obviously you can just change the name of your notebook, to stop this issue, but we could think about placing the script file in a separate folder to fully mitigate

[mmcky]

thanks @chrisjsewell -- I see. I am still confused about what jupyter-sphinx does -- handles {code-cell} directive?. The py files are actually the source md file really.

The notebooks in that folder are not necessarily intended to be run from there

Completely agree - just trying to figure out why execution failed for a bunch of notebooks -- that I know run successfully.

but we could think about placing the script file in a separate folder to fully mitigate

If we need any py files generated I certainly think they should be elsewhere to prevent erroneous import issues

[mmcky]

It won't be the last lecture named matplotlib.md :-)

[mmcky]

This is actually a broader issue for quantecon as we have a lot of lectures written on libraries. For lecture-python-programming we have 5 such as numpy, matplotlib etc. and there isn't much of a reason to change the name of the lecture.

@chrisjsewell is it always going to be true that jupyter-cache writes multiple files / source file. If this is the case perhaps we should have different folders for each file: jupyter-cache/<filename>/<ipynb,py,txt>

[choldgraf]

FWIW I agree that we should not constrain users to not calling source files the names of libraries - I think it is common for folks to name source files in this way if the topic of the page covers a single library.

[chrisjsewell]

is it always going to be true that jupyter-cache writes multiple files / source file

Just to re-emphasise, this is not a jupyter-cache issue, jupyter-cache does not write these files, jupyter-sphinx does. jupyter-cache saves all "its" notebooks in seperate, hashed, folders.

[choldgraf]

ah - in that case we should move this issue over there, though I'm not sure what the exact bug is, perhaps @mmcky knows?

[chrisjsewell]

The "fix" is to save the script files in a seperate folder to the notebook files

[mmcky]

@chrisjsewell so we need to move this to jupyter-sphinx or myst-nb? I agree with the "fix" just not sure where it needs to be implemented.

[chrisjsewell]

yep jupyter-sphinx

[bsipocz]

This is/was an upstream bug, so I'm closing this issue here. Please reopen in jupyter-sphinx if you still experience it.