Meson: test void linux on CI and add devcontainer for it

[tobiasdiez]

trafficstars

Add a CI test for Void Linux building sage with meson (similar to #40038 for Fedora). A few minor bugs in the system package installation are fixed: a missing --yes for the package index update, and adding a few missing void packages.

:memo: Checklist

• [ ] The title is concise and informative.
• [ ] The description explains in detail what this PR is about.
• [ ] I have linked a relevant issue or discussion.
• [ ] I have created tests covering the changes.
• [ ] I have updated the documentation and checked the documentation preview.

:hourglass: Dependencies

[tornaria]

I've no idea what this is supposed to be testing. This is definitely not the way a user would be installing sagemath on void linux, or is it?

eval $(build/bin/sage-print-system-package-command $SYSTEM update)
eval $(build/bin/sage-print-system-package-command $SYSTEM --yes --ignore-missing install $(build/bin/sage-get-system-packages $SYSTEM $(uv run --no-project build/bin/sage-package list :standard:)))

:point_up: my head is about to explode.

BTW, once uv is involved for deps, this is no longer void linux. All the python packages that appear here are available with up to date versions, if you want to test integration with void linux it's better IMO if you install the system packages.

I'm not opposing to anything here, just stating an opinion. I thought the point of the meson build was to get rid of this nonsense. Installing sagemath "by hand" should be simpler than that.

PS: I'm ready to switch to meson build for our package once it's ready. Is there a follow up to #39015?

[dimpase]

I've no idea what this is supposed to be testing. This is definitely not the way a user would be installing sagemath on void linux, or is it?

As far as I know, you don't have a void develop branch that follows the Sage develop branch, and, anyway, how do you test our develop/PRs ? I'm sure you don't re-write all the packages and what not involved in void install scripts (not sure how they are called), right?

I assume this creates a container in which we have

1. installed all the known to void Sage dependencies

2. taken them into account, and resolved the rest of the dependencies in the usual meson/pip way. (this means that perhaps some void dependencies are not used - it would be good if these are shown in some way)

eval $(build/bin/sage-print-system-package-command $SYSTEM update)
eval $(build/bin/sage-print-system-package-command $SYSTEM --yes --ignore-missing install $(build/bin/sage-get-system-packages $SYSTEM $(uv run --no-project build/bin/sage-package list :standard:)))

☝️ my head is about to explode.

I hear you -- now I am getting back to rational curves and schemes over $\mathbb{R}$ 👅

BTW, once uv is involved for deps, this is no longer void linux. All the python packages that appear here are available with up to date versions, if you want to test integration with void linux it's better IMO if you install the system packages.

OK, perhaps @tobiasdiez got carried away - as in e.g. Fedora and Debian many packages are not present, so one has to deal with it. Tobias, do I make sense here?

I'm not opposing to anything here, just stating an opinion. I thought the point of the meson build was to get rid of this nonsense. Installing sagemath "by hand" should be simpler than that.

Arch, Gentoo, and Void are outliers, as they try to be up to date, as opposed to "big tech Linux" Fedora/Debian/Suse.

PS: I'm ready to switch to meson build for our package once it's ready. Is there a follow up to #39015?

[tobiasdiez]

I've no idea what this is supposed to be testing. This is definitely not the way a user would be installing sagemath on void linux, or is it?

It's supposed to be the new way of developing sagemath: install all system packages and run uv sync. This gives you a venv with an editable install of sage. The CI test is mostly there to see if recent changes in sage broke the meson build and to see if all system packages are recognized correctly. For example, as you can see in the test output https://github.com/sagemath/sage/actions/runs/14974466210/job/42063137564?pr=40089 the elliptic curves database is not yet found correctly, so this needs further work.

In general, I would agree in that these tests for void have limited value as you usual follow pretty closely the current develop branch of sage and provide up-to-date system dependencies. Things are harder for other linux distros.

☝️ my head is about to explode.

Mine too. It's the typical syntactic sugar of sage-the-distro ;-)

BTW, once uv is involved for deps, this is no longer void linux. All the python packages that appear here are available with up to date versions, if you want to test integration with void linux it's better IMO if you install the system packages.

Sure, you can also run uv with use-system-packages. For void this probably makes sense, but as Dima said, for many other linux distros you need to install most python packages from pip. So for consistency, I used uv everywhere.

I thought the point of the meson build was to get rid of this nonsense. Installing sagemath "by hand" should be simpler than that.

What do you think should/can be improved?

PS: I'm ready to switch to meson build for our package once it's ready. Is there a follow up to #39015?

Awesome! Yes, @antonio-rojas still made a few nice improvements to the CLI https://github.com/sagemath/sage/issues?q=is%3Apr%20author%3Aantonio-rojas

taken them into account, and resolved the rest of the dependencies in the usual meson/pip way. (this means that perhaps some void dependencies are not used - it would be good if these are shown in some way)

Void provides recent-enough versions for all system packages and meson uses all of them. sage-the-distro actually rejects a few of these dependencies and rebuilt them. So that's another win for meson ;-)

[tornaria]

I've no idea what this is supposed to be testing. This is definitely not the way a user would be installing sagemath on void linux, or is it?

As far as I know, you don't have a void develop branch that follows the Sage develop branch, and, anyway, how do you test our develop/PRs ?

I can build sagemath-standard "in place" from source (git checkout) using standard python tooling which works assuming all dependencies are available. In fact, I have a script to run bootstrap and fix everything (sagemath-standard), this allows me to work on the tree.

Also, at some point in the release cycle I create a draft PR for void-packages in which I track the beta releases using the build template we will later use to create our packages, so usually I'm able to release the void linux package very quickly after a release of sagemath.

I'm sure you don't re-write all the packages and what not involved in void install scripts (not sure how they are called), right?

All the dependencies are already available from other packages.

I assume this creates a container in which we have

1. installed all the known to void Sage dependencies

2. taken them into account, and resolved the rest of the dependencies in the usual meson/pip way.
   (this means that perhaps some void dependencies are not used - it would be good if these are shown in some way)

Every necessary dependency can (and should) be installed from void. Then python -m build -nw should "just work", or whatever other way. There should be some "generic" instructions on what I should do. Then it is "my" responsibility to provide all dependencies, and it's "your" responsibility to tell me what generic steps work to build. E.g. something like:

$ meson setup bdir
...
$ ninja -C bdir
...

should work? Everything builds without a glitch, but how do I run it now???

The only thing I was able to do is

$ ninja -C bdir test
...

which fails miserably with 226 errors during collection...

Where are the build instructions for the meson build?

Arch, Gentoo, and Void are outliers, as they try to be up to date, as opposed to "big tech Linux" Fedora/Debian/Suse.

:shrug: I'll shut up and retreat to my outlier corner. You are welcome to join https://docs.voidlinux.org/installation/index.html

[tobiasdiez]

You can just pip install . as with every other python project. Or use use meson install. See https://doc.sagemath.org/html/en/installation/meson.html#background-information for build instructions.

(btw, did you saw my previous message above?)

[tornaria]

I've no idea what this is supposed to be testing. This is definitely not the way a user would be installing sagemath on void linux, or is it?

It's supposed to be the new way of developing sagemath: install all system packages and run uv sync.

$ uv sync
error: Failed to generate package metadata for `sagemath @ editable+.`
  Caused by: The build backend returned an error
  Caused by: Call to `mesonpy.prepare_metadata_for_build_editable` failed (exit status: 1)

[stderr]
Traceback (most recent call last):
  File "<string>", line 8, in <module>
    import mesonpy as backend
ModuleNotFoundError: No module named 'mesonpy'

hint: This usually indicates a problem with the package or the build environment.

I do have mesonpy:

$ python -c 'import mesonpy; print(mesonpy)'
<module 'mesonpy' from '/usr/lib/python3.13/site-packages/mesonpy/__init__.py'>

In general, I would agree in that these tests for void have limited value as you usual follow pretty closely the current develop branch of sage and provide up-to-date system dependencies.

On the contrary, given that we have all dependencies, it's very useful that there's a simple instruction on how to build and run sagemath from source on void linux using all system dependencies, and the integration test should be checking that it really works.

Mine too. It's the typical syntactic sugar of sage-the-distro ;-)

So, is this testing sage-the-distro or is this testing sagelib?

Sure, you can also run uv with use-system-packages. For void this probably makes sense, but as Dima said, for many other linux distros you need to install most python packages from pip. So for consistency, I used uv everywhere.

I understand, my point is that then you are not testing if the meson build works with void linux packages, but you are only testing that the meson build works with the packages you install with uv. But those should be more or less "the same" regardless of distro, no?

I thought the point of the meson build was to get rid of this nonsense. Installing sagemath "by hand" should be simpler than that.

What do you think should/can be improved?

Right now the sagemath-standard sdist is completely independent of sage-the-distro. It doesn't ship anything from build/* and it works perfectly fine. I'd expect the sagemath sdist to also be independent, for me this means not using anything from build/*.

Ideally, IMO, the test script should be a (maybe not user friendly but still useful) set of instructions on what is the recommended way to build sagemath on each given distro. I still don't know what is the recommended way to build sagemath on a reasonable distro that has all dependencies available (there are at least three of those, afaik).

I think it's very important to have an "official" way to build & install sagemath that is simple, uses standard tooling, and is acceptable for distros. Then distros will follow.

Awesome! Yes, @antonio-rojas still made a few nice improvements to the CLI https://github.com/sagemath/sage/issues?q=is%3Apr%20author%3Aantonio-rojas

Nice.

taken them into account, and resolved the rest of the dependencies in the usual meson/pip way. (this means that perhaps some void dependencies are not used - it would be good if these are shown in some way)

Void provides recent-enough versions for all system packages and meson uses all of them. sage-the-distro actually rejects a few of these dependencies and rebuilt them. So that's another win for meson ;-)

We don't need a replacement for sage-the-distro, we need a replacement for sagemath-standard (aka sagelib, i.e. the python distribution that ships the sage.* namespace). I don't think this is your goal anyway.

[tornaria]

You can just pip install . as with every other python project. Or use use meson install. See https://doc.sagemath.org/html/en/installation/meson.html#background-information for build instructions.

pip is not distro-friendly (I'd say it's not friendly at all but YMMV).

(btw, did you saw my previous message above?)

yes, thanks

Here's my log, following the instructions(starting always from a clean git checkout of 10.7.beta3):

$ meson setup builddir --prefix=$PWD/build-install
...
$ meson compile -C builddir
...
$ meson install -C builddir
ninja: Entering directory `/home/tornaria/src/sage/sage-git-clean/builddir'
ninja: no work to do.
Installing subdir /home/tornaria/src/sage/sage-git-clean/src/sage/cli to /usr/lib/python3.13/site-packages/sage/cli
Installation failed due to insufficient permissions.
Attempt to use /usr/bin/sudo to gain elevated privileges? [y/n] n
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/mesonbuild/mesonmain.py", line 193, in run
    return options.run_func(options)
           ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/mesonbuild/minstall.py", line 870, in run
    installer.do_install(datafilename)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/mesonbuild/minstall.py", line 549, in do_install
    self.install_subdirs(d, dm, destdir, fullprefix) # Must be first, because it needs to delete the old subtree.
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/mesonbuild/minstall.py", line 617, in install_subdirs
    dm.makedirs(full_dst_dir, exist_ok=True)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/mesonbuild/minstall.py", line 109, in makedirs
    self.makedirs_impl(path, exist_ok=exist_ok)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/mesonbuild/minstall.py", line 316, in makedirs
    os.makedirs(*args, **kwargs)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "<frozen os>", line 227, in makedirs
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.13/site-packages/sage/cli'

ERROR: Unhandled python OSError. This is probably not a Meson bug, but an issue with your build environment.

It seems --prefix is doing nothing here.

[dimpase]

@tornaria - the docs in the current beta say that you have to run

 meson setup builddir --prefix=/desired/install/path -Dpython.install_env=prefix

but you haven't used -Dpython.install_env

That's from a recent change:

commit 60105962be08dc0a5ecf8f7cbe201b141965d36a
Author: Tobias Diez <[email protected]>
Date:   Thu Apr 17 12:29:45 2025 +0800

    Meson: automatically install into venv if activated

[antonio-rojas]

E.g. something like:

$ meson setup bdir
...
$ ninja -C bdir
...

should work? Everything builds without a glitch, but how do I run it now???

Packaging should work like in any other Python package - see https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD.meson?h=sagemath-git for the Arch script.

[tobiasdiez]

I've no idea what this is supposed to be testing. This is definitely not the way a user would be installing sagemath on void linux, or is it?

It's supposed to be the new way of developing sagemath: install all system packages and run uv sync.

$ uv sync
error: Failed to generate package metadata for `sagemath @ editable+.`
  Caused by: The build backend returned an error
  Caused by: Call to `mesonpy.prepare_metadata_for_build_editable` failed (exit status: 1)

[stderr]
Traceback (most recent call last):
  File "<string>", line 8, in <module>
    import mesonpy as backend
ModuleNotFoundError: No module named 'mesonpy'

hint: This usually indicates a problem with the package or the build environment.

I do have mesonpy:

Sorry, it should have been:

uv venv
uv pip install \
    meson-python \
    "cypari2 >=2.2.1" \
    "cython >=3.0, != 3.0.3, != 3.1.0" \
    "cython >=3.0, != 3.0.3" \
    "gmpy2 ~=2.1.b999" \
    memory_allocator \
    "numpy >=1.25" \
    jinja2 \
    setuptool
uv sync --frozen --inexact --no-build-isolation

In general, I would agree in that these tests for void have limited value as you usual follow pretty closely the current develop branch of sage and provide up-to-date system dependencies.

On the contrary, given that we have all dependencies, it's very useful that there's a simple instruction on how to build and run sagemath from source on void linux using all system dependencies, and the integration test should be checking that it really works.

These instructions have been added in #40124, and the CI added here is exactly testing these installations steps.

Mine too. It's the typical syntactic sugar of sage-the-distro ;-)

So, is this testing sage-the-distro or is this testing sagelib?

This is testing the meson build of sagelib. But in order to install the system packages, it queries the information part of sage-the-distro. This translates to xbps-install <a long list>. In the long term, it would be nice to extract this information out of sage-the-distro (perhaps to https://github.com/jaimergp/external-metadata-mappings / https://github.com/rgommers/external-deps-build)

Sure, you can also run uv with use-system-packages. For void this probably makes sense, but as Dima said, for many other linux distros you need to install most python packages from pip. So for consistency, I used uv everywhere.

I understand, my point is that then you are not testing if the meson build works with void linux packages, but you are only testing that the meson build works with the packages you install with uv. But those should be more or less "the same" regardless of distro, no?

Yes, we only test here the non-python packages of void. We could also install the python packages, but that information is mostly missing in the build directory (someone would need to create a lot of void.txt files there specifying the mapping of spkg -> void pkg).

I thought the point of the meson build was to get rid of this nonsense. Installing sagemath "by hand" should be simpler than that.

What do you think should/can be improved?

Right now the sagemath-standard sdist is completely independent of sage-the-distro. It doesn't ship anything from build/* and it works perfectly fine. I'd expect the sagemath sdist to also be independent, for me this means not using anything from build/*.

The meson sdist is also completely independent of sage-the-distro and the build directory.

Ideally, IMO, the test script should be a (maybe not user friendly but still useful) set of instructions on what is the recommended way to build sagemath on each given distro. I still don't know what is the recommended way to build sagemath on a reasonable distro that has all dependencies available (there are at least three of those, afaik).

I think it's very important to have an "official" way to build & install sagemath that is simple, uses standard tooling, and is acceptable for distros. Then distros will follow.

Awesome! Yes, @antonio-rojas still made a few nice improvements to the CLI https://github.com/sagemath/sage/issues?q=is%3Apr%20author%3Aantonio-rojas

Nice.

taken them into account, and resolved the rest of the dependencies in the usual meson/pip way. (this means that perhaps some void dependencies are not used - it would be good if these are shown in some way)

Void provides recent-enough versions for all system packages and meson uses all of them. sage-the-distro actually rejects a few of these dependencies and rebuilt them. So that's another win for meson ;-)

We don't need a replacement for sage-the-distro, we need a replacement for sagemath-standard (aka sagelib, i.e. the python distribution that ships the sage.* namespace). I don't think this is your goal anyway.

[dimpase]

@tobiasdiez in your long comment, the cython line in uv pip instructions is repeated twice.

[tobiasdiez]

@tobiasdiez in your long comment, the cython line in uv pip instructions is repeated twice.

Where exactly do you find this duplication? I made a small mistake while merging develop into this branch, but fixed it later. I cannot find it now in the final diff.

[dimpase]

@tobiasdiez in your long comment, the cython line in uv pip instructions is repeated twice.

Where exactly do you find this duplication? I made a small mistake while merging develop into this branch, but fixed it later. I cannot find it now in the final diff.

I meant that part of your comment above:

uv venv
uv pip install \
    meson-python \
    "cypari2 >=2.2.1" \
    "cython >=3.0, != 3.0.3, != 3.1.0" \
    "cython >=3.0, != 3.0.3" \
    "gmpy2 ~=2.1.b999" \
    memory_allocator \
    "numpy >=1.25" \
    jinja2 \
    setuptool
uv sync --frozen --inexact --no-build-isolation

you have constraints on cython twice.

[tobiasdiez]

Right, thanks. In this comment it's indeed still wrong - but it's correct in the branch.

[tobiasdiez]

@tornaria In https://github.com/sagemath/sage/pull/40142, I've now added the flag to use system site packages for the venv. Is there anything else I can do to make these CI runs more useful for you?

[dimpase]

a meaningful use of CI here would be to test if one can develop Sage on void: that is, install the dependencies from the system, and build/test sagelib as if for developement.

[tobiasdiez]

a meaningful use of CI here would be to test if one can develop Sage on void: that is, install the dependencies from the system, and build/test sagelib as if for developement.

That's exactly what's tested here.

[github-actions[bot]]

Documentation preview for this PR (built with commit 6eb4fc7af433b5ab2084cd05ad6172d6b966efe9; changes) is ready! :tada: This preview will update shortly after each push to this PR.

[tobiasdiez]

Reverted back to the glibc version, since github actions run into issues on musl (https://github.com/actions/runner/issues/3852).

[tobiasdiez]

okay done at https://github.com/sagemath/sage/issues/40538.

[dimpase]

https://github.com/sagemath/sage/actions/runs/16703101385/job/47277223668#step:11:1

shows that gmp (!) does not get built. Why does it even try?

[tobiasdiez]

https://github.com/sagemath/sage/actions/runs/16703101385/job/47277223668#step:11:1

shows that gmp (!) does not get built. Why does it even try?

This is a known issue, appeared some time ago.

[dimpase]

we probably should rather try setting up CI for the Alpine Linux (the musl-based distro which appears to be supported by GH Actions)

[tobiasdiez]

Added testing for Alpine in https://github.com/sagemath/sage/pull/40561.

For me the most important thing to test (and that's why I've created this PR) is the detection of all dependencies via meson. For example, the current run (https://github.com/sagemath/sage/actions/runs/16703101367/job/47277216520?pr=40089) shows that some of the db like database_cremona_mini_ellcurve is not correctly recognized. Fixing this will be done after the migration of sagedistro to meson.

[dimpase]

Added testing for Alpine in #40561.

For me the most important thing to test (and that's why I've created this PR) is the detection of all dependencies via meson. For example, the current run (https://github.com/sagemath/sage/actions/runs/16703101367/job/47277216520?pr=40089) shows that some of the db like database_cremona_mini_ellcurve is not correctly recognized. Fixing this will be done after the migration of sagedistro to meson.

I am not sure what "this" in the last comment is.

I must say I don't like the sagedistro features creep into the meson build as it happens now (I think @orlitzky is of a similar opinion). I would rather have the sagelib meson build being taught to use the stuff it needs in SAGE_LOCAL and other non-standard locations. As these are all non-python things we need to worry about (right?), I would like to understand the obstacles to this. It seems that having pkg-config files helps?

This will be more than enough to satisfy the needs of builds in all sorts of scenarios, and then we can think of sagedistro.

[tobiasdiez]

I must say I don't like the sagedistro features creep into the meson build as it happens now (I think @orlitzky is of a similar opinion).

Which features exactly do you mean here?

I would rather have the sagelib meson build being taught to use the stuff it needs in SAGE_LOCAL and other non-standard locations. As these are all non-python things we need to worry about (right?), I would like to understand the obstacles to this. It seems that having pkg-config files helps?

The only real issues are executables like maxima and databases. Libraries are also found in SAGE_LOCAL by setting the correct compiler flags. pkg-config files are nice as they are more robust, but not essential.

[dimpase]

I must say I don't like the sagedistro features creep into the meson build as it happens now (I think @orlitzky is of a similar opinion).

Which features exactly do you mean here?

I meant things like replacing spkg-install scripts with the corresponding meson calls. However, I might be conflating CI-specific fixes where meson is used, or maybe some of your branches which didn't make it into the mainline, sorry.

[tobiasdiez]

Yeah, I'm also not the biggest fan of these subprojects. However, it's the only path forward that I see how to remove sage-the-distro. Otherwise, even on recentish systems, you would need to tell people to build say flint from source as it's not available from their system package manager.

[dimpase]

would it be more reasonable to have an optional meson sage-distro, with sagelib a subproject (and the latter can be installable without sage-distro) ?

[dimpase]

Libraries are also found in SAGE_LOCAL by setting the correct compiler flags. pkg-config files are nice as they are more robust, but not essential.

actually, having a pkg-config file for, say, foobar almost magically allows to merely say something like foobar = dependency('foobar') in meson.build, done; different libraries can then come from rather different places. E.g. I tried meson setup cp312 in Conda; pari wasn't found, but

$ cat /opt/conda/lib/pkgconfig/pari.pc 
prefix=/opt/conda
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${exec_prefix}/lib

Name: Pari
Description: Pari Library for Number Theory
Version: 2.17.2 
URL: https://whatever
Requires: gmp >= 6.2.1
Cflags: -I${includedir}
Libs: -L${libdir} -lpari

allowed the Pari test to pass. The only non-generic part in the above pari.pc is prefix= in the 1st line.