Compatibilty with Sphinx 8?

[WarrenWeckesser]

The CI doc builds for SciPy recently started failing: https://github.com/scipy/scipy/issues/21323

The error message suggests a problem with jupyterlite_sphinx. It looks like an exception is being generated when used with the recently released Sphinx 8. Is this a known issue?

[agriyakhetarpal]

If no one is taking a look as I type this, I shall do so in a moment.

[steppi]

Thanks @agriyakhetarpal. I’m off this week and have limited availability to look at this. I think it’s just a deprecation warning that gets turned to an error when building the docs with `-werror‘. Something about using strings instead of pathlib path objects. It should be straightforward.

[agriyakhetarpal]

I'm unable to reproduce the error locally, however, and it seems like Sphinx is returning a deprecation warning for me instead of any trace of jupyterlite-sphinx:

Tap to view logs

Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/cmd/build.py", line 337, in build_main
    app.build(args.force_all, args.filenames)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/application.py", line 378, in build
    self.builder.build_update()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 296, in build_update
    self.build(to_build,
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 317, in build
    updated_docnames = set(self.read())
                           ^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 424, in read
    self._read_serial(docnames)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 476, in _read_serial
    self.read_doc(docname)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 535, in read_doc
    publisher.publish()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/docutils/core.py", line 234, in publish
    self.document = self.reader.read(self.source, self.parser,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/io.py", line 106, in read
    self.parse()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/docutils/readers/__init__.py", line 76, in parse
    self.parser.parse(self.input, document)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/myst_nb/sphinx_.py", line 85, in parse
    nb_reader = create_nb_reader(document_path, md_config, nb_config, inputstring)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/myst_nb/core/read.py", line 71, in create_nb_reader
    if path.endswith(suffix):
       ^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/util/_pathlib.py", line 96, in __getattr__
    warnings.warn(_MSG, RemovedInSphinx90Warning, stacklevel=2)
sphinx.deprecation.RemovedInSphinx90Warning: Sphinx 9 will drop support for representing paths as strings. Use "pathlib.Path" or "os.fspath" instead.

Exception occurred:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/scipy-dev/lib/python3.12/site-packages/sphinx/util/_pathlib.py", line 96, in __getattr__
    warnings.warn(_MSG, RemovedInSphinx90Warning, stacklevel=2)
sphinx.deprecation.RemovedInSphinx90Warning: Sphinx 9 will drop support for representing paths as strings. Use "pathlib.Path" or "os.fspath" instead.
The full traceback has been saved in /var/folders/b3/2bq1m1_50bs4c7305j8vxcqr0000gn/T/sphinx-err-pfvg2gf8.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [html-build] Error 2

I'm going to fix the bug assuming there's a problem with my system, but with this I'm unsure how to test if the bug fix works, so it would be good if you could try it out, @WarrenWeckesser!

[steppi]

@agriyakhetarpal, that looks like the exact same deprecation warning. I think you did reproduce it. Have you confirmed the warning goes away with the fix you made in jupyterlite-sphinx?

[agriyakhetarpal]

No, the issue is that while the deprecation warning is the same, jupyterlite_sphinx.py doesn't seem to be involved in those logs anywhere – it seems to be coming from somewhere inside myst-nb's code. I don't see any warnings in our own logs with sphinx>=8, so I requested assistance with reproduction (maybe I'm mixing pip-installed packages in a mamba-environment or something). I built the SciPy docs with an editable installation of jupyterlite-sphinx.