Subdirectories with json and ipynb files are scanned by `lite:patch`

[goekce]

Problem

I am new to the project and could not get jupyterlite-sphinx running on top of my existing Sphinx site in less than three hours. One of the reasons was:

Extension error (jupyterlite_sphinx.jupyterlite_sphinx):
Handler <function jupyterlite_build at ...> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', '/home/u/projects/.../_contents', '--output-dir', '/home/u/projects/.../_build/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/u/projects/...]' returned non-zero exit status 2.)
make: *** [makefile:25: html] Error 2

Which does not say much but can be expanded using https://jupyterlite-sphinx.readthedocs.io/en/latest/configuration.html#suppressing-jupyterlite-logging. I saw then the following:

TaskError - taskid:build:lite:patch:.gitlab-ci-local/artifacts/build-webpages/_build/html/lite/jupyter-lite.json
PythonAction Error
Traceback (most recent call last):
  File "/home/u/projects/.../venv/lib/python3.12/site-packages/doit/action.py", line 461, in execute
    returned_value = self.py_callable(*self.args, **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/u/projects/aydos.de/venv/lib/python3.12/site-packages/jupyterlite_core/addons/base.py", line 234, in merge_one_jupyterlite
    out_path.write_text(json.dumps(config, **JSON_FMT), **UTF8)
  File "/usr/lib/python3.12/pathlib.py", line 1047, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pathlib.py", line 1013, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/u/projects/.../_build/html/lite/.gitlab-ci-local/artifacts/build-webpages/_build/html/lite/jupyter-lite.json'

You notice the path .gitlab-ci-local/.... It turns out that jupyterlite-sphinx (lite:patch) also ipynb and json files in directories unrelated to the project, which leads to an error. I could not find any option to not read .gitlab-ci-local. exclude_patterns in conf.py did not work.

Proposed Solution

Subdirectories with . prefix are not read or there is an option to suppress specific directories.

[agriyakhetarpal]

Thanks for reporting, @goekce! I guess JupyterLite is trying to pick up its JSON configuration file from an unrelated location, which it shouldn't. Could you provide a small project or a script so that we have a reproducer?

[goekce]

Is it legit if I demonstrate that using jupyterlite-sphinx? 🙂

git clone https://github.com/jupyterlite/jupyterlite-sphinx && cd jupyterlite-sphinx/docs
virtualenv venv
. ./venv/bin/activate
pip install voici pydata-sphinx-theme myst-parser jupyterlite-sphinx
sphinx-build -M html . _build  # Builds successfully
mkdir .test
cp -r . .test
sphinx-build -M html . _build  # Fails 💀

Output:

Extension error (jupyterlite_sphinx.jupyterlite_sphinx):
Handler <function jupyterlite_build at 0x746c9c755260> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', './custom_contents', '--contents', '/home/u/projects/jupyterlite-sphinx/docs/_contents', '--output-dir', '/home/u/projects/jupyterlite-sphinx/docs/_build/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--apps', 'voici', '--lite-dir', '/home/u/projects/jupyterlite-sphinx/docs']' returned non-zero exit status 2.)

I use gitlab-ci-local which creates a cache under .gitlab-ci-local which leads to the same error.

[jtpio]

It looks related to these issues in JupyterLite:

• https://github.com/jupyterlite/jupyterlite/issues/1037
• https://github.com/jupyterlite/jupyterlite/issues/1095

It would be great to fix it in jupyterlite-core, as they seem to be happening quite often.

If anyone would like to look into it, that would be great, thanks!