Migrate CI build-doc to Meson

[tobiasdiez]

trafficstars

Make it possible to build the documentation with meson via

meson compile -C builddir doc-html

For this a couple of changes was necessary in the docbuilder:

• Make sage_docbuild independent of sage so that meson can use sage_docbuild during config time to construct all the docbuild targets (otherwise one needs to first install sage, and then could configure the docbuild)
• Properly handle input and output dirs for the docbuild, without relying on magic sage env variables

:memo: Checklist

• [ ] The title is concise and informative.
• [ ] The description explains in detail what this PR is about.
• [ ] I have linked a relevant issue or discussion.
• [ ] I have created tests covering the changes.
• [ ] I have updated the documentation and checked the documentation preview.

:hourglass: Dependencies

[user202729]

Looks like some file permission goes wrong? (symbolic link get converted to normal file?)

[github-actions[bot]]

Documentation preview for this PR (built with commit 3408887be301371ec911ea2029085f385dee6046; changes) is ready! :tada: This preview will update shortly after each push to this PR.

[tobiasdiez]

Doc build is working now, and the changes relative to the last develop build seem to be minimal and mostly due to using a newer sphinx version now.

Marking it as blocker since currently the docbuild CI for other PRs is completely failing.

[dimpase]

https://github.com/sagemath/sage/actions/runs/15081532937/job/42398821050#step:9:1

fails while building docs the old way - is it normal?

[tobiasdiez]

It fails with

ImportError: libSingular-4.4.0.so: cannot open shared object file: No such file or directory

https://github.com/sagemath/sage/actions/runs/15081532937/job/42398821050#step:9:3982 so this is the same problem as for the other PRs.

[dimpase]

no, my question was - why does that CI run build docs the old way?

[tobiasdiez]

no, my question was - why does that CI run build docs the old way?

Because this CI workflow still runs on top of sage-the-distro which is not using meson. So it still uses the old make infrastructure.

(it's also a CI bug that this particular workflow actually tries to build the docs. It should only test sagelib, not any docs)

[user202729]

• What's with the test failure in test-new?
• Are we dropping support for building documentation the old way? If not, do test both.
• Is the method of building documentation as listed in https://doc.sagemath.org/html/en/developer/sage_manuals.html#editing-the-documentation still work?

[tobiasdiez]

• What's with the test failure in test-new?

That's a CI bug. Meson doesn't install sage_docbuild and it's normally also not tested. But because it's now marked as changed, the doctest gets explicitly invoked on it.

* Are we dropping support for building documentation the old way? If not, do test both.
* Is the method of building documentation as listed in https://doc.sagemath.org/html/en/developer/sage_manuals.html#editing-the-documentation still work?

The old method should still work. I don't have the means tho to test it. Did you experience any issues? The sage manual will be migrated to the meson build in a follow-up.

[user202729]

But because it's now marked as changed, the doctest gets explicitly invoked on it.

Hm, the doctest framework has a mechanism to mark something as not tested. (See https://github.com/sagemath/sage/pull/39102/files#diff-5c3fa597431eda03ac3339ae6bf7f05e1a50d6fc7333679ec38e21b337cb6721R123 ). Maybe it is also applicable here?

[user202729]

• Are we dropping support for building documentation the old way? If not, do test both.

The old method should still work. I don't have the means tho to test it. Did you experience any issues? The sage manual will be migrated to the meson build in a follow-up.

I think it's the same issue as https://github.com/sagemath/sage/pull/39467 again.

While I don't use the mechanism myself, I think it makes sense to

• either it is supported, then the code to test it should remain in the code base (if it's temporarily broken then just add a if: false or something),
• or it is not supported, then we should update the documentation.

Since you intend to remove the documentation on how to build documentation the old way, I suppose you intend to un-support that?

[tobiasdiez]

But because it's now marked as changed, the doctest gets explicitly invoked on it.

Hm, the doctest framework has a mechanism to mark something as not tested. (See https://github.com/sagemath/sage/pull/39102/files#diff-5c3fa597431eda03ac3339ae6bf7f05e1a50d6fc7333679ec38e21b337cb6721R123 ). Maybe it is also applicable here?

Thanks, I was not aware of this mechanism!

I think it's the same issue as https://github.com/sagemath/sage/pull/39467 again. either it is supported, then the code to test it should remain in the code base

I think the situation here is a bit different. The doc build ci workflow is mostly there to provide devs a preview of the doc changes; it's purpose is less to test the docbuilding itself (of course it's testing this as well). Also the meson and make scripts just invoke the sage docbuilder, which is doing most of the work - so I don't think it's worth to test both ways.

But yeah, it's a complicated question on how to best migrate completely to meson, also with respect to the required doc changes. On one hand, it's a lot of additional work to maintain the two systems in parallel (including docs and ci for both) while on the other hand you would want to make the transition as smooth as possible. I'm hoping that we can soft-deprecate the make interface in the next release cycle, which would be then a natural point to revise and migrate more of the documentation to meson. I'm open and happy about other opinions on how this migration should look like.

[user202729]

Thanks, I was not aware of this mechanism!

do you plan to fix this later or just ignore it for this pull request?

[tobiasdiez]

Thanks, I was not aware of this mechanism!

do you plan to fix this later or just ignore it for this pull request?

Just wanted to add it to this PR, only to realize that it's not going to be that easy: the new workflow is just using ordinary git commands to stage all changed files (relative to develop), and then calls sage test --new. Probably can be fixed by committing changes to certain directories again manually; but that's definitely something for another PR.

[dimpase]

@user202729 - are you happy enough here now to revert this to positive review?

[user202729]

Actually:

• maybe change/append the PR title to "Migrate CI build-doc to meson"?
• currently build-doc-pdf is not tested at all? What's the plan for that?

[user202729]

Also the meson and make scripts just invoke the sage docbuilder, which is doing most of the work - so I don't think it's worth to test both ways.

Actually, the test is exactly doing its jobs, no?

In other words: is https://github.com/sagemath/sage/issues/40120 just an issue with the CI, or does an actual user using sage-the-distro (since that isn't completely deprecated yet we ought to assume some user exists…) trying to build doc that way would encounter the same error? The latter appears more likely? (I'm not using sage-the-distro myself)

In that case if the CI infrastructure is removed how do we know if a similar breakage won't happen again with some other package upgrade?

[tobiasdiez]

Actually:

* maybe change/append the PR title to "Migrate CI build-doc to meson"?

Done.

* currently build-doc-pdf is not tested at all? What's the plan for that?

I don't have a plan for this atm. To be honest, I don't even see the point of building a pdf of the docs in 2025. Having a good online documentation should suffice.

In other words: is https://github.com/sagemath/sage/issues/40120 just an issue with the CI, or does an actual user using sage-the-distro (since that isn't completely deprecated yet we ought to assume some user exists…) trying to build doc that way would encounter the same error?

The error is actually not related to the docs build. What happened is that the cython module was compiled with Singular x. Then Singular was updated to version y. The problem is that the CI/sage-the-distro doesn't recognize that the version of Singular changed and the cython module needs to be compiled again to link against Singular y and not x. So the underlying issue is with the caching of the CI/sage-the-distro.

[user202729]

So the underlying issue is with the caching of the CI/sage-the-distro.

Indeed, I also guessed that. But isn't the CI caching the exact same mechanism as the normal caching anyway (e.g. an user of sage-the-distro originally use 4.4.0, then git pull to upgrade it to 4.4.1, will face the same error)

Anyway… can we somehow delete the CI cache to fix test-long for now?

I don't even see the point of building a pdf of the docs in 2025

In some rare cases, this catches some LaTeX errors… as a fallback if the author forget to check the documentation diff.

(Also, some HTML pages are extremely large that it lags my browser, it might be beneficial for someone to look at the PDF, because it's paginated it should render at reasonable speed)

[user202729]

@vbraun What's the issue?

[tobiasdiez]

@vbraun is it because of http://build.sagemath.org/#/builders/47/builds/124? This indeed fails for the docs, but the full error points more to the m4ri update as the source.

[vbraun]

Does pdf docbuild work? I don't remember what the error was, but pretty sure the merge script failed because of that.

And yes, the point of running TeX is to verify that the equation are valid markup.

[dimpase]

@vbraun there are no changes to anything TeX-typesettable here.

[user202729]

Does pdf docbuild work? I don't remember what the error was, but pretty sure the merge script failed because of that.

And yes, the point of running TeX is to verify that the equation are valid markup.

Not on the CI at the moment, but that was broken by a combination of previous pull requests. My understanding (best guess?) of the situation is explained in https://github.com/sagemath/sage/pull/40157 .

I'm not sure if build-doc-pdf depends on build-doc in some way however. A quick reading shows no dependency.

[vbraun]

----------------------------------------------------------------------
sage -t --long --warn-long 30.0 --random-seed=123 src/sage_docbuild/builders.py  # 16 doctests failed
----------------------------------------------------------------------

[dimpase]

I am trying to run meson compile -C builddir doc-html (after doing meson install) and I am getting errors such as the following

[2/189] Generating src/doc/doc-inventory-reference-references with a custom command
[reference] Configuration error!
[reference] Versions
[reference] ========
[reference] * Platform:         linux; (Linux-6.13.4-gentoo-x86_64-x86_64-Intel-R-_Core-TM-_Ultra_7_165U-with-glibc2.41)
[reference] * Python version:   3.13.3 (CPython)
[reference] * Sphinx version:   8.2.3
[reference] * Docutils version: 0.21.2
[reference] * Jinja2 version:   3.1.6
[reference] * Pygments version: 2.19.1
[reference] Last Messages
[reference] =============
[reference] None.
[reference] Loaded Extensions
[reference] =================
[reference] None.
[reference] Traceback
[reference] =========
[reference]       File "/usr/lib/python3.13/site-packages/sphinx/config.py", line 616, in eval_config_file
[reference]         raise ConfigError(msg % traceback.format_exc()) from exc
[reference]     sphinx.errors.ConfigError: There is a programmable error in your configuration file:
[reference]     Traceback (most recent call last):
[reference]       File "/usr/lib/python3.13/site-packages/sphinx/config.py", line 601, in eval_config_file
[reference]         exec(code, namespace)  # NoQA: S102
[reference]         ~~~~^^^^^^^^^^^^^^^^^
[reference]       File "/home/dima/software/sage-src/src/doc/en/reference/references/conf.py", line 19, in <module>
[reference]         for tag in feature_tags():
[reference]                    ^^^^^^^^^^^^
[reference]     NameError: name 'feature_tags' is not defined
[reference] The full traceback has been saved in:

which seem to indicate that no sphinx extensions are loaded. Why?

[dimpase]

@tobiasdiez : what is feature_tags It's something not defined in Sage sources

[tobiasdiez]

Tests are passing now.

[tobiasdiez]

@vbraun @dimpase Can we get this in now?

[dimpase]

please update uv.lock to have primecountpy 0.1.1

https://files.pythonhosted.org/packages/ec/99/a705fe480cfdccd697682d126972f401ec43922da0ee7a06aafd29a12d98/primecountpy-0.1.1.tar.gz

(I'm not sure how to get all the needed data there, probably just one uv call)