Migrate `doc/bootstrap` to Python

[tobiasdiez]

Migrate the shell script doc/bootstrap to Python. This improves performance and is preparation for building the docs simply by running "sphinx", see https://github.com/sagemath/sage/pull/41156.

The output should be more or less equivalent, expect for minor formatting changes and better escaping of the generated shell commands.

:memo: Checklist

• [ ] The title is concise and informative.
• [ ] The description explains in detail what this PR is about.
• [ ] I have linked a relevant issue or discussion.
• [ ] I have created tests covering the changes.
• [ ] I have updated the documentation and checked the documentation preview.

:hourglass: Dependencies

[github-actions[bot]]

Documentation preview for this PR (built with commit dc56c2ab155f598f379550ffc03b6d9b686c7d18; changes) is ready! :tada: This preview will update shortly after each push to this PR.

[mantepse]

I have no idea how I could help in the reviewing process here. Any hints welcome! (I don't understand shell scripts)

The changes in package.py are purely cosmetic, with the exception of adding read_system_packages, right? (I admit that I almost dislike all of them - with the exception of replacing ' with ", which I don't care about. I find breaking the lines in this way makes the code less readable)

[antonio-rojas]

Doesn't build here

FAILED: [code=1] src/doc/htmlspkg 
/usr/bin/python /build/sagemath-doc-git/src/sage/src/build-docs.py --no-prune-empty-dirs --no-pdf-links reference/spkg html -o src/doc --source /build/sagemath-doc-git/src/build/src/doc
[spkg     ] inventory <https://ipywidgets.readthedocs.io/en/stable/> contains multiple definitions for std:label:examples/Widget Layout.ipynb#display
[spkg     ] building [html]: targets for 467 source files that are out of date
[spkg     ] updating environment: 0 added, 31 changed, 0 removed
[spkg     ] /build/sagemath-doc-git/src/build/src/doc/en/reference/spkg/sagemath_objects.rst:64: WARNING: undefined label: 'spkg_force' [ref.ref]
[spkg     ] The HTML pages are in src/doc/html/en/reference/spkg.
Error building the documentation.
Traceback (most recent call last):
  File "/build/sagemath-doc-git/src/sage/src/build-docs.py", line 11, in <module>
    main()
    ~~~~^^
  File "/build/sagemath-doc-git/src/sage/src/sage_docbuild/__main__.py", line 548, in main
    build()
    ~~~~~^^
  File "/build/sagemath-doc-git/src/sage/src/sage_docbuild/builders.py", line 669, in _wrapper
    getattr(DocBuilder, build_type)(self, *args, **kwds)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/build/sagemath-doc-git/src/sage/src/sage_docbuild/builders.py", line 142, in f
    runsphinx()
    ~~~~~~~~~^^
  File "/build/sagemath-doc-git/src/sage/src/sage_docbuild/sphinxbuild.py", line 325, in runsphinx
    sys.stderr.raise_errors()
    ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/build/sagemath-doc-git/src/sage/src/sage_docbuild/sphinxbuild.py", line 256, in raise_errors
    raise OSError(self._error)
OSError: /build/sagemath-doc-git/src/build/src/doc/en/reference/spkg/sagemath_objects.rst:64: WARNING: undefined label: 'spkg_force' [ref.ref]

[tobiasdiez]

I thought that was fixed already with https://github.com/sagemath/sage/pull/41162/commits/f71e2e63b939eee02de187940e60f8da7fbc9e8a. Strange... I don't have time atm, but will look at it end of next week.

[antonio-rojas]

Looks like the bootstrap.py→bootstrap-docs.py rename somehow confuses github, and it generates an invalid patch that doesn't apply correctly. I built directly from your branch and it works fine.

[tobiasdiez]

@antonio-rojas Happy to hear. So this is good to go from your side?

@mantepse

I have no idea how I could help in the reviewing process here. Any hints welcome! (I don't understand shell scripts)

The added script under tools is the major change. So I would say try to execute that on your system (directly with python) and review the code in there.

The changes in package.py are purely cosmetic, with the exception of adding read_system_packages, right?

Right

(I admit that I almost dislike all of them - with the exception of replacing ' with ", which I don't care about. I find breaking the lines in this way makes the code less readable)

That's how ruff thinks the code should be formatted. I stopped long time ago to argue with linters/code formatters... 🦊

[tobiasdiez]

Thanks!