msys2
Add passagemath - general purpose mathematical system (SageMath fork)
Package name
passagemath
Brief description of package
Modularized fork of the general purpose mathematical system SageMath.
It consists of ~70 pip-installable packages, most with some non-Python library dependencies (see https://github.com/passagemath#passagemath-in-the-mathematical-software-landscape for an overview)
Porting effort to Windows (mingw64 for the libraries, native for the Python packages themselves): https://github.com/passagemath/passagemath/issues/1044
URL for package's homepage
https://github.com/passagemath/passagemath
Provide a basic test case to validate the package's functionality.
No response
MINGW environments where you need the package
- [ ] MINGW64
- [ ] MINGW32
- [x] UCRT64
- [ ] CLANG64
- [x] CLANGARM64
Are you willing to submit a PR?
No, but I would be interested in collaborating on mingw64 porting
Just for the record, I don't know passagemath or SageMath. So the following may be based on my (possibly incorrect or incomplete) understanding of what passagemath is. Feel free to correct me, if you spot any errors.
As far as I can see from the links provided above, there are a lot of passagemath Python packages. Having all of it in MSYS2 would mean that all of the "~70 pip-installable packages" had to be packaged for MSYS2. And that is just the passagemath part. Of course, all the dependencies would have to be available to. So as a test I did pip3 install passagemath-plot in a venv. (Package was chosen randomly, could have been any other passagemath package, too.) That required several other dependencies:
Installing collected packages: mpmath, jupyter-threejs-sage, typing_extensions, six, pyparsing, platformdirs, pillow, packaging, numpy, kiwisolver, gmpy2, fonttools, Cython, cysignals, cycler, scipy, python-dateutil, passagemath-objects, passagemath-environment, memory_allocator, contourpy, passagemath-categories, matplotlib, passagemath-modules, passagemath-plot
Successfully installed Cython-3.1.3 contourpy-1.3.3 cycler-0.12.1 cysignals-1.12.4 fonttools-4.59.2 gmpy2-2.3.0a1 jupyter-threejs-sage-130 kiwisolver-1.4.9 matplotlib-3.10.6 memory_allocator-0.1.4 mpmath-1.3.0 numpy-2.3.2 packaging-25.0 passagemath-categories-10.6.16 passagemath-environment-10.6.16 passagemath-modules-10.6.16 passagemath-objects-10.6.16 passagemath-plot-10.6.16 pillow-11.3.0 platformdirs-4.4.0 pyparsing-3.2.3 python-dateutil-2.9.0.post0 scipy-1.16.1 six-1.17.0 typing_extensions-4.15.0
So these are 25 Python packages, and 20 without the passagemath packages. Some of those packages, for example matplotlib or scipy are already available in MSYS2. Others like cysignals and memory_allocator are not. For passagemath to work those would have to be made available / packaged in MSYS2, too.
To get an idea how many packages would be required, let's take a look at Arch Linux. MSYS2 is not Arch Linux, but MSYS2 uses the same package manager as Arch Linux, pacman. And what's more, Arch Linux already has a SageMath package, of which passagemath is a fork, right? Only important information there is the part where it says "Dependencies (123)", meaning SageMath on Arch Linux depends on 123 other packages. Some of them are optional, but that's still a lot. Assuming that SageMath and passagemath still have roughly the same dependency graph I guess we are looking at somewhere between 100 and 200 new packages to package all of passagemath for MSYS2. Maybe closer to 100 than 200, if most of the optional stuff is skipped.
What that means is: It will take quite a lot of time to get all of passagemath into MSYS2, if someone were to undertake that task. Luckily, passagemath is modularized (from what I understand), so unlike SageMath it does not need to be done all in one go.
Still, that's quite a task. Maybe somebody could provide a list of passagemath packages prioritized by most used or most relevant packages. The basic idea would be to provide only a small but already useful subset of passagemath which could then be expanded with more packages later on.
So if you had to pick - let's say - at most ten of the circa 70 passagemath packages, which ones would that be?
@striezel Thanks for looking into this. Your analysis is correct.
Luckily, passagemath is modularized (from what I understand), so unlike SageMath it does not need to be done all in one go.
Exactly.
if you had to pick - let's say - at most ten of the circa 70 passagemath packages, which ones would that be?
I would suggest to focus on the packages for which the porting to native Windows is already done; look for "(Available on Windows.) in https://github.com/passagemath/passagemath?tab=readme-ov-file#modularized-distributions:~:text=Available%20on%20Windows
In particular, passagemath-combinat, passagemath-graphs, passagemath-modules and their short list of dependencies. Together they provide a very useful subset of the functionality of the whole system.
@striezel Thanks for looking into this. Your analysis is correct.
Good. 👍
if you had to pick - let's say - at most ten of the circa 70 passagemath packages, which ones would that be?
I would suggest to focus on the packages for which the porting to native Windows is already done; look for "(Available on Windows.) in https://github.com/passagemath/passagemath?tab=readme-ov-file#modularized-distributions:~:text=Available%20on%20Windows
In particular, passagemath-combinat, passagemath-graphs, passagemath-modules and their short list of dependencies. Together they provide a very useful subset of the functionality of the whole system.
Just to see what we're getting into and how much work would be needed for those, I did the same Python virtual environment approach I did earlier on Linux for those packages.
passagemath-combinat:
Installing collected packages: typing_extensions, platformdirs, packaging, gmpy2, Cython, cysignals, passagemath-objects, passagemath-environment, memory_allocator, passagemath-categories, passagemath-combinat
Successfully installed Cython-3.2.1 cysignals-1.12.6 gmpy2-2.3.0a1 memory_allocator-0.1.4 packaging-25.0 passagemath-categories-10.6.36 passagemath-combinat-10.6.36 passagemath-environment-10.6.36 passagemath-objects-10.6.36 platformdirs-4.5.0 typing_extensions-4.15.0
Packages in MSYS2/MINGW64:
- ✅ cython: https://packages.msys2.org/base/mingw-w64-cython
- ❌ cysignals
- ✅ gmpy2: https://packages.msys2.org/base/mingw-w64-python-gmpy2
- ❌ memory_allocator
- ✅ packaging: https://packages.msys2.org/base/mingw-w64-python-packaging
- ❌ passagemath-categories
- ❌ passagemath-combinat
- ❌ passagemath-environment
- ❌ passagemath-objects
- ✅ platformdirs: https://packages.msys2.org/base/mingw-w64-python-platformdirs
- ✅ typing_extensions: https://packages.msys2.org/base/mingw-w64-python-typing_extensions
5/11 available, 6 more to package
passagemath-graphs:
Installing collected packages: typing_extensions, platformdirs, packaging, gmpy2, Cython, cysignals, passagemath-objects, passagemath-environment, memory_allocator, passagemath-categories, passagemath-graphs
Successfully installed Cython-3.2.1 cysignals-1.12.6 gmpy2-2.3.0a1 memory_allocator-0.1.4 packaging-25.0 passagemath-categories-10.6.36 passagemath-environment-10.6.36 passagemath-graphs-10.6.36 passagemath-objects-10.6.36 platformdirs-4.5.0 typing_extensions-4.15.0
Packages in MSYS2/MINGW64:
- ✅ cython: https://packages.msys2.org/base/mingw-w64-cython
- ❌ cysignals
- ✅ gmpy2: https://packages.msys2.org/base/mingw-w64-python-gmpy2
- ❌ memory_allocator
- ✅ packaging: https://packages.msys2.org/base/mingw-w64-python-packaging
- ❌ passagemath-categories
- ❌ passagemath-environment
- ❌ passagemath-graphs
- ❌ passagemath-objects
- ✅ platformdirs: https://packages.msys2.org/base/mingw-w64-python-platformdirs
- ✅ typing_extensions: https://packages.msys2.org/base/mingw-w64-python-typing_extensions
5/11 available, 6 more to package (not counting overlap with / duplicates from passagemath-combinat)
passagemath-modules:
Installing collected packages: mpmath, typing_extensions, platformdirs, packaging, gmpy2, Cython, cysignals, passagemath-objects, passagemath-environment, memory_allocator, passagemath-categories, passagemath-modules
Successfully installed Cython-3.2.1 cysignals-1.12.6 gmpy2-2.3.0a1 memory_allocator-0.1.4 mpmath-1.3.0 packaging-25.0 passagemath-categories-10.6.36 passagemath-environment-10.6.36 passagemath-modules-10.6.36 passagemath-objects-10.6.36 platformdirs-4.5.0 typing_extensions-4.15.0
Packages in MSYS2/MINGW64:
- ✅ cython: https://packages.msys2.org/base/mingw-w64-cython
- ❌ cysignals
- ✅ gmpy2: https://packages.msys2.org/base/mingw-w64-python-gmpy2
- ❌ memory_allocator
- ✅ mpmath: https://packages.msys2.org/base/mingw-w64-python-mpmath
- ✅ packaging: https://packages.msys2.org/base/mingw-w64-python-packaging
- ❌ passagemath-categories
- ❌ passagemath-environment
- ❌ passagemath-modules
- ❌ passagemath-objects
- ✅ platformdirs: https://packages.msys2.org/base/mingw-w64-python-platformdirs
- ✅ typing_extensions: https://packages.msys2.org/base/mingw-w64-python-typing_extensions
6/12 available, 6 more to package (not counting overlap with / duplicates from passagemath-combinat or passagemath-graphs list)
So if someone were to package one of those dependencies per day, the first package could be available in approx. one week, and the other two packages in another week (needs less work due to overlap, so it should be faster).
@mkoeppe Meanwhile I managed to package some parts of passagemath for MSYS2.
However, during the attempt to package passagemath-modules (#26508) I encountered an error:
error: failed to commit transaction (conflicting files)
/ucrt64/lib/python3.12/site-packages/sage/ext/interpreters/__init__.py exists in both 'mingw-w64-ucrt-x86_64-python-passagemath-categories' and 'mingw-w64-ucrt-x86_64-python-passagemath-modules'
/ucrt64/lib/python3.12/site-packages/sage/ext/interpreters/__pycache__/__init__.cpython-312.opt-1.pyc exists in both 'mingw-w64-ucrt-x86_64-python-passagemath-categories' and 'mingw-w64-ucrt-x86_64-python-passagemath-modules'
/ucrt64/lib/python3.12/site-packages/sage/ext/interpreters/__pycache__/__init__.cpython-312.pyc exists in both 'mingw-w64-ucrt-x86_64-python-passagemath-categories' and 'mingw-w64-ucrt-x86_64-python-passagemath-modules'
I've checked the contents of the categories package (already available in the repository) and the modules package from the failed CI manually, and they both contain the file /ucrt64/lib/python3.12/site-packages/sage/ext/interpreters/__init__.py. It has the same content in both cases:
# Automatically generated by D:/M/msys64/ucrt64/lib/python3.12/site-packages/sage_setup/autogen/interpreters/internal/generator.py. Do not edit!
It's basically empty, apart from the line that tells us it was auto-generated. Is it intended that both packages place that same file in the same location? If yes, why? If not, what is the intended workaround? Or am I missing something during the package creation process? I could just remove the conflicting __init__.py from one of the packages (which one?) during the package process, but that would feel like only curing the symptoms of a deeper problem.
That's a bug. There should be no __init__.py file in this package directory, for neither of the packages. It is meant to be set up as implicit namespace packages (PEP 420).
For now, you can remove it from both packages, meanwhile I fix what may need to be fixed on my side.
It's fixed in https://github.com/passagemath/passagemath/releases/tag/passagemath-10.6.38, but deploying this release to PyPI is currently blocked by GH Actions bug https://github.com/actions/runner-images/issues/13341
I've patched this bug in passagemath-setuptools for MSYS2 in https://github.com/msys2/MINGW-packages/pull/26524, so this is currently not a problem anymore. I'll continue to package version 10.6.37 until the GH Actions bug is fixed and/or releases for 10.6.38 are available on PyPI.
Now that some of the passagemath packages are available in MSYS2, I would like people who are familiar with passagemath / SageMath to give them a try and see whether this works for them. Available packages in MSYS2 can be found via the package search of MSYS2 and searching for "passagemath".
The basic command to install passagemath packages from within MSYS2 should look like this in the case of passagemath-combinat:
# for Windows on 64-bit Intel/AMD processors:
$ pacman -S mingw-w64-ucrt-x86_64-python-passagemath-combinat
# for Windows on ARM64:
$ pacman -S mingw-w64-clang-aarch64-python-passagemath-combinat
Adjust commands as needed for other passagemath packages, basically only the last part of the package name needs to be changed (for example from mingw-w64-ucrt-x86_64-python-passagemath-combinat to mingw-w64-ucrt-x86_64-python-passagemath-modules).
Thanks a lot! I think I'll set up a workflow on GH Actions to test these downstream packages.
In https://github.com/passagemath/passagemath/actions/runs/19692334962/job/56410652561?pr=1832#step:4:217, I am running into a problem with the sage-runtests script (from passagemath-repl):
/usr/bin/bash: line 1: /ucrt64/bin/sage-runtests: cannot execute: required file not found
Interesting.
Let's take a look at the contents of /ucrt64/bin/sage-runtests from the mingw-w64-ucrt-x86_64-python-passagemath-repl package (currently version 10.6.37-1) then:
#!<launcher_dir>\python.exe
import sys
from sage.doctest.__main__ import main
if __name__ == "__main__":
sys.exit(main())
It looks like the shebang line got butchered by some automatism. Of course, <launcher_dir>\python.exe does not exist. The proper path for the Python 3.x executable on UCRT64 is /ucrt64/bin/python.exe or /ucrt64/bin/python3.exe. So the first line should contain that instead. When I look at the sage-runtests file in the passagemath repository for version 10.6.37, I see #!/usr/bin/env sage-python as the first line. So there may be some mechanism that tries to convert the path and fails.
Some other scripts in the passagemath-repl package also contain that <launcher_dir> stuff, namely:
sage-cleaner
sage-coverage
sage-eval
sage-fixdoctests
sage-ipython
sage-notebook
sage-preparse
sage-run
sage-run-cython
sage-runtests
sage-startuptime.py
The sage-runtests file from the .tar.gz in PyPI (that's where our build process fetches the source) also contains #!/usr/bin/env sage-python as shebang. So whatever conversion happens during the build process, it seems to take a wrong turn somewhere.
Unfortunately, I don't know where or why that conversion happens, but maybe you can give some hints.
It seems to come from here: https://github.com/msys2/MINGW-packages/blob/d62144100f378f5f0926822d91f28432e8088c1b/mingw-w64-python-installer/001-launcher-relative.patch#L8
@lazka
I'm no Python expert, but this looks wrong to me. Just adding that line without any way to actually replace the <launcher_dir> part by a proper path is bound to cause problems. I'd like to remove that patch, but maybe @lazka has some more info on what that patch is supposed to do.
By the way, the passagemath packages in MSYS2 have been updated to version 10.6.38 (https://github.com/msys2/MINGW-packages/pull/26571), now that the source packages for 10.6.38 are available on PyPI, too.
I'll set up a workflow on GH Actions to test these downstream packages.
The first green checkmarks (for tests that are not affected by the above) can be seen at https://github.com/passagemath/passagemath/actions/runs/19717082734/job/56491678500?pr=1832
With mingw-w64-python-passagemath-environment 10.6.38-2 and mingw-w64-python-passagemath-repl 10.6.38-2 now available, I am re-running tests at https://github.com/passagemath/passagemath/actions/runs/19771134223
The next issue is a bug on our side (https://github.com/passagemath/passagemath/issues/1824), which is fixed in https://github.com/passagemath/passagemath/releases/tag/passagemath-10.6.39, expected to be on PyPI by tomorrow.
<launcher_dir> is supposed to be interpreted by the python launcher: https://github.com/pypa/distlib/blob/6286442857de9f734686d08f0e59ca8048ee357a/docs/tutorial.rst#specifying-a-custom-executable-for-shebangs
That shebang should never be used by bash etc in theory.
Also "installer" is supposed to append them to the launcher .exe, so they should never be visible.
From what I see setuptools rewrites shebangs of python scripts using to "#!python", and "installer" rewrites those to the new interpreter, even in the non-launcher case. I'll try to improve the installer patch.
Confirming that the issue with "launcher_dir" is resolved by #26593.
Thank you for the confirmation. I did not expect a fix for python-installer to be available so soon, so I added a workaround.
The next issue is a bug on our side (passagemath/passagemath#1824), which is fixed in https://github.com/passagemath/passagemath/releases/tag/passagemath-10.6.39, expected to be on PyPI by tomorrow.
If @lazka's <launcher_dir> patch (#26601) has been merged by then, we will get passagemath 10.6.39 packages in MSYS2 with the proper path. I'll leave the fix from #26593 in for the moment, because it won't do anything once the <launcher_dir> part disappears from the scripts.
10.6.39 is up now
Pull request for the update is on the way (https://github.com/msys2/MINGW-packages/pull/26624) and - if it passes - will be merged soon. [Edit] It's merged. [/Edit]
Due to the python-installer change I diffed the generated packages for passagemath-environment and passagemath-repl. (These are the ones where the <launcher_dir> stuff occurred.) You can see the result here: https://github.com/msys2/MINGW-packages/pull/26624#issuecomment-3591759340
Paths in *-environment package changed from #!/usr/bin/env python3 to #!/usr/bin/env python (sage-num-threads.py) and from #!/ucrt64/bin/python3 to #!/usr/bin/env python (sage-venv-config). This should be harmless, because they basically end up with the same python(3).exe executable.
Paths in *-repl package changed from #!/usr/bin/env sage-python to #!/usr/bin/env python (sage-cleaner, sage-eval, sage-fixdoctests, sage-ipython, sage-notebook, sage-preparse, sage-run, sage-run-cython, sage-runtests, sage-startuptime.py). That may or may not be as intended, I am not sure.
Thanks a lot @striezel and @lazka.
These changes to the shebang lines are fine. In particular, replacing sage-python by python matches what pip does when installing the package.
@striezel Now with 10.6.39, real tests are being run in my downstream tests on GH Actions, for example:
- homfly (all tests pass) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291839?pr=1842#step:4:231
- combinat (test failures similar to testing with my Windows wheels) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291797?pr=1842
- nauty (test failures that might indicate problems of the MSYS2 nauty package; I don't have https://github.com/passagemath/passagemath/pull/1089 ready so I cannot compare it) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291821?pr=1842#step:4:219
- homfly (all tests pass) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291839?pr=1842#step:4:231
Yes! 🚀
- combinat (test failures similar to testing with my Windows wheels) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291797?pr=1842
The relevant output there ends with
File "sage/rings/real_mpfr.pyx", line 3699, in sage.rings.real_mpfr.RealNumber.simplest_rational
from sage.rings.real_mpfi import RealIntervalField
ModuleNotFoundError: No module named 'sage.rings.real_mpfi'
Maybe this is due to some part of passagemath not being available in MSYS2 yet? What passagemath package does the Python module 'sage.rings.real_mpfi' belong to?
- nauty (test failures that might indicate problems of the MSYS2 nauty package; I don't have Build Windows wheels for passagemath-nauty passagemath/passagemath#1089 ready so I cannot compare it) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291821?pr=1842#step:4:219
I've just added dynamic libraries to the nauty package (https://github.com/msys2/MINGW-packages/pull/26639). Maybe that will help, I will have to investigate. There might indeed be another problem.
- combinat (test failures similar to testing with my Windows wheels) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291797?pr=1842
The relevant output there ends with
File "sage/rings/real_mpfr.pyx", line 3699, in sage.rings.real_mpfr.RealNumber.simplest_rational from sage.rings.real_mpfi import RealIntervalField ModuleNotFoundError: No module named 'sage.rings.real_mpfi'Maybe this is due to some part of passagemath not being available in MSYS2 yet?
No, this one happens to be an expected failure. A few lines above, https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291797?pr=1842#step:4:40399, you see
sage -t --warn-long 5.0 --random-seed=157068334641477321995495109500174177372 --environment=sage.all__sagemath_combinat sage/categories/bialgebras_with_basis.py # [failed in baseline]
The text [failed in baseline] indicates that there are known failures in the tests of this file. The tests are still run and failures displayed, but they do not show up in the error status.
This is one of several mechanisms that I use in passagemath to make the testsuite, written for the monolithic SageMath, suitable for modularized testing. (It can be refined over time by marking individual doctests as dependent on the availability of certain features; in this case, one would use the tag # needs sage.rings.real_interval_field.)
What passagemath package does the Python module 'sage.rings.real_mpfi' belong to?
It's in passagemath-flint.
- nauty (test failures that might indicate problems of the MSYS2 nauty package; I don't have Build Windows wheels for passagemath-nauty passagemath/passagemath#1089 ready so I cannot compare it) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291821?pr=1842#step:4:219
I've just added dynamic libraries to the nauty package (#26639). Maybe that will help
It's relatively unlikely, but we'll see. These tests use nauty's executables, not a library interface.
- combinat (test failures similar to testing with my Windows wheels) https://github.com/passagemath/passagemath/actions/runs/19788086989/job/56698291797?pr=1842
The relevant output there ends
That it ends there is a real bug, but also one that I see in tests with my Windows wheels. There are supposed to be many more tests. Somehow the terminal is lost at this point, I think. I am trying to address this in https://github.com/passagemath/passagemath/pull/1843
What passagemath package does the Python module 'sage.rings.real_mpfi' belong to?
It's in passagemath-flint.
OK. That is not available in MSYS2, yet, because it needs passagemath-ntl which is also not available yet. I guess I should make a list or I might lose track of these things.
I've just added dynamic libraries to the nauty package (#26639). Maybe that will help
It's relatively unlikely, but we'll see. These tests use
nauty's executables, not a library interface.
I see.