cschwan
ImportError: failed to map segment from shared object
I've never seen this before. It's present with vanilla and also on Prefix. The failure on Prefix is basically the same but with a different library that can't be imported. The following is on Gentoo.
$ sage -t --long /usr/lib/python3.8/site-packages/sage/repl/all.py
Running doctests with ID 2020-11-21-19-01-03-837ef810.
Using --optional=dochtml,memlimit,sage
Doctesting 1 file.
*** Warning: not enough memory, new PARI stack 432537600
Traceback (most recent call last):
File "/usr/lib/python-exec/python3.8/sage-runtests", line 182, in <module>
err = DC.run()
File "/usr/lib/python3.8/site-packages/sage/doctest/control.py", line 1232, in run
self.run_doctests()
File "/usr/lib/python3.8/site-packages/sage/doctest/control.py", line 928, in run_doctests
self.dispatcher = DocTestDispatcher(self)
File "/usr/lib/python3.8/site-packages/sage/doctest/forker.py", line 1670, in __init__
init_sage(controller)
File "/usr/lib/python3.8/site-packages/sage/doctest/forker.py", line 206, in init_sage
controller.load_environment()
File "/usr/lib/python3.8/site-packages/sage/doctest/control.py", line 523, in load_environment
return import_module(self.options.environment)
File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/lib/python3.8/site-packages/sage/repl/ipython_kernel/all_jupyter.py", line 5, in <module>
from sage.all_cmdline import *
File "/usr/lib/python3.8/site-packages/sage/all_cmdline.py", line 26, in <module>
from sage.all import *
File "/usr/lib/python3.8/site-packages/sage/all.py", line 138, in <module>
from sage.rings.all import *
File "/usr/lib/python3.8/site-packages/sage/rings/all.py", line 62, in <module>
from .padics.all import *
File "/usr/lib/python3.8/site-packages/sage/rings/padics/all.py", line 3, in <module>
from .factory import Zp, Zq, Zp as pAdicRing, ZpCR, ZpCA, ZpFM, ZpFP, ZpLC, ZpLF, ZqCR, ZqCA, ZqFM, ZqFP #, ZpL, ZqL
File "/usr/lib/python3.8/site-packages/sage/rings/padics/factory.py", line 57, in <module>
from .padic_extension_leaves import (EisensteinExtensionFieldCappedRelative,
File "/usr/lib/python3.8/site-packages/sage/rings/padics/padic_extension_leaves.py", line 56, in <module>
from .qadic_flint_FM import qAdicFixedModElement
ImportError: /usr/lib/python3.8/site-packages/sage/rings/padics/qadic_flint_FM.cpython-38-x86_64-linux-gnu.so: failed to map segment from shared object
But the following works:
$ sage -t --long --memlimit=0 /usr/lib/python3.8/site-packages/sage/repl/all.py
Running doctests with ID 2020-11-21-19-01-53-f1c63596.
Using --optional=dochtml,sage
Doctesting 1 file.
sage -t --long --warn-long 223.3 --random-seed=0 /usr/lib/python3.8/site-packages/sage/repl/all.py
[0 tests, 0.00 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 0.0 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds
The failure on Prefix does not exhibit the PARI issue, but adding the --memlimit=0 does allow the doctest to pass.
[I] sci-mathematics/pari
Available versions: 2.11.4(0/6)^t ~2.13.0(0/7)^t {X data doc fltk gmp test threads}
Installed versions: 2.11.4(0/6)^t(08:37:52 PM 10/03/2020)(X data doc gmp threads -fltk -test)
Homepage: https://pari.math.u-bordeaux.fr/
Description: Computer-aided number theory C library and tools
[I] sci-mathematics/flint
Available versions: 2.5.2-r1(0/13) ~2.6.3(0/14)^t {doc gc ntl static-libs test}
Installed versions: 2.5.2-r1(0/13)(09:31:29 PM 10/20/2020)(ntl -doc -gc -static-libs)
Homepage: http://www.flintlib.org/
Description: Fast Library for Number Theory
[U] dev-python/cypari2 [1]
Available versions: (~)2.1.1 (~)2.1.2 {PYTHON_TARGETS="python3_6 python3_7 python3_8"}
Installed versions: 2.1.1(09:24:29 PM 10/04/2020)(PYTHON_TARGETS="python3_7 python3_8 -python3_6")
Homepage: https://github.com/sagemath/cypari2
Description: A Python interface to the number theory library libpari
[U] sci-mathematics/singular
Available versions: 4.0.2 ~4.0.3 (~)4.1.1_p2-r2[1] (~)4.1.1_p2-r3[1] ~4.1.3_p2[1] ~4.1.3_p2-r1[1] {boost doc emacs examples python +readline static-libs PYTHON_SINGLE_TARGET="+python2_7"}
Installed versions: 4.1.1_p2-r2[1](01:11:30 PM 05/11/2020)(readline -emacs -examples -python -static-libs PYTHON_SINGLE_TARGET="-python2_7")
Homepage: https://www.singular.uni-kl.de/ https://github.com/Singular/Sources
Description: Computer algebra system for polynomial computations
[1] "sage-on-gentoo" /var/lib/layman/sage-on-gentoo
This could have been present for some time since I had been doctesting with `--memlimit=0` for other memory limit failures.
That's a very weird message too. What does readelf -d /usr/lib/python3.8/site-packages/sage/rings/padics/qadic_flint_FM.cpython-38-x86_64-linux-gnu.so says?
$ readelf -d /usr/lib/python3.8/site-packages/sage/rings/padics/qadic_flint_FM.cpython-38-x86_64-linux-gnu.so
Dynamic section at offset 0x60bf0 contains 26 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libflint.so.13]
0x0000000000000001 (NEEDED) Shared library: [libgmp.so.10]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000c (INIT) 0x97f0
0x000000000000000d (FINI) 0x5469c
0x0000000000000019 (INIT_ARRAY) 0x260be0
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x260be8
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x190
0x0000000000000005 (STRTAB) 0x19e0
0x0000000000000006 (SYMTAB) 0x360
0x000000000000000a (STRSZ) 6153 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x261000
0x0000000000000002 (PLTRELSZ) 3912 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x88a8
0x0000000000000007 (RELA) 0x3400
0x0000000000000008 (RELASZ) 21672 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x33d0
0x000000006fffffff (VERNEEDNUM) 1
0x000000006ffffff0 (VERSYM) 0x31ea
0x000000006ffffff9 (RELACOUNT) 833
0x0000000000000000 (NULL) 0x0
On Prefix the failure is with a different lib
File "/storage/strogdon/gentoo-rap/usr/lib/python3.8/site-packages/sage/repl/ipython_kernel/all_jupyter.py", line 5, in <module>
from sage.all_cmdline import *
File "/storage/strogdon/gentoo-rap/usr/lib/python3.8/site-packages/sage/all_cmdline.py", line 26, in <module>
from sage.all import *
File "/storage/strogdon/gentoo-rap/usr/lib/python3.8/site-packages/sage/all.py", line 133, in <module>
from sage.libs.all import *
File "/storage/strogdon/gentoo-rap/usr/lib/python3.8/site-packages/sage/libs/all.py", line 7, in <module>
import sage.libs.symmetrica.all as symmetrica
File "/storage/strogdon/gentoo-rap/usr/lib/python3.8/site-packages/sage/libs/symmetrica/all.py", line 4, in <module>
from .symmetrica import start, end
ImportError: libsymmetrica-2.0.so: failed to map segment from shared object
Upgrading to symmetrica-3.0.1 to see if things change on Prefix.
Wondering if this is something like this https://stackoverflow.com/questions/58975839/dopen-failed-to-map-segment-from-shared-object-when-not-running-as-root in which case changing ulimit may do something. The settings for that stuff sits in /etc/security/limits.conf and there is a limits.d/ folder to read more granular settings.
Interesting point that I have seen sometimes when ssh-ing in a box, some extra limits may be applied.
What do you have for
$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 30854
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 99
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 30854
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
I changed core file size to unlimited with no change.
I have, when ssh-ing
$ ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 127294
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 127294
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
core file size puts limits on the size of coredumps. Leaving it at 0 means no coredump.
On prefix I have
max user processes (-u) 257289
I'm unable to change uname -u on Gentoo (no sshing involved)
ulimit -u 127294
bash: ulimit: max user processes: cannot modify limit: Operation not permitted
You may have to be root to change it. Otherwise, you can change things in /etc/security/limits.conf, I don't know if you need a reboot or a new login for it to be effective.
Same failure on Prefix as above comment after upgrading to symmetrica-3.0.1. user processes on Prefix is twice what you have. There must be some component that's doing this.
I am seeing this too on vanilla: https://groups.google.com/g/sage-devel/c/49cYaiuRzWU/m/FR0fQy0RBwAJ
it's Python 3.7 for me, though. (And building Sage 9.3.beta1 from source, using as many Gentoo system packages as are currently recognised by spkg-configure macros)
For me --memlimit=0 makes a difference in tests too. E.g.
./sage -tp --memlimit=0 src/doc/de/thematische_anleitungen/sage_gymnasium.rst
works, but omitting --memlimit=0 leads to
./sage -tp src/doc/de/thematische_anleitungen/sage_gymnasium.rst >/tmp/x
*** Warning: not enough memory, new PARI stack 432537600
Process DocTestWorker-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/mnt/opt/Sage/sage-dev/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2191, in run
task(self.options, self.outtmpfile, msgpipe, self.result_queue)
File "/mnt/opt/Sage/sage-dev/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2541, in __call__
result_queue.put(result, False)
File "/usr/lib/python3.7/multiprocessing/queues.py", line 87, in put
self._start_thread()
File "/usr/lib/python3.7/multiprocessing/queues.py", line 170, in _start_thread
self._thread.start()
File "/usr/lib/python3.7/threading.py", line 852, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
and in the output I see this ImportError:
File "src/doc/de/thematische_anleitungen/sage_gymnasium.rst", line 615, in doc.de.thematische_anleitungen.sage_gymnasium
Failed example:
plot(f)
Exception raised:
Traceback (most recent call last):
File "/mnt/opt/Sage/sage-dev/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 720, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/mnt/opt/Sage/sage-dev/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
exec(compiled, globs)
File "<doctest doc.de.thematische_anleitungen.sage_gymnasium[1]>", line 1, in <module>
plot(f)
File "/mnt/opt/Sage/sage-dev/local/lib/python3.7/site-packages/sage/misc/decorators.py", line 492, in wrapper
return func(*args, **options)
File "/mnt/opt/Sage/sage-dev/local/lib/python3.7/site-packages/sage/plot/plot.py", line 1961, in plot
G = funcs.plot(*args, **original_opts)
File "sage/symbolic/expression.pyx", line 12164, in sage.symbolic.expression.Expression.plot (build/cythonized/sage/symbolic/expression.cpp:65514)
f = self._plot_fast_callable(param)
File "sage/symbolic/expression.pyx", line 12222, in sage.symbolic.expression.Expression._plot_fast_callable (build/cythonized/sage/symbolic/expression.cpp:66065)
return fast_callable(self, vars=vars, expect_one_var=True)
File "sage/ext/fast_callable.pyx", line 487, in sage.ext.fast_callable.fast_callable (build/cythonized/sage/ext/fast_callable.c:5186)
import sage.ext.interpreters.wrapper_py
ImportError: /mnt/opt/Sage/sage-dev/local/lib/python3.7/site-packages/sage/ext/interpreters/wrapper_py.cpython-37m-x86_64-linux-gnu.so: failed to map segment from shared object
...
I certainly recall being able to build and run Sage 9.2 on the box just fine. I'm rebuilding 9.2 now to see if it exhibits the same oddity.
I see that Gentoo now provides Pari 2.13. I wonder if it makes a difference.
Sage 9.2 built from source: deja vu, same ImportError. That is, one needs to bisect Gentoo updates rather than Sage updates to get to the root cause, it seems to me.
pari 2.13 does not get installed due to a weird texlive error. No idea why it is trying to do something with pdftex.fmt
make -j4 docpdf
cd doc && make docpdf
make[1]: Entering directory '/var/tmp/portage/sci-mathematics/pari-2.13.0/work/pari-2.13.0/doc'
rm -f develop.std
rm -f libpari.std
rm -f parallel.std
pdftex develop
pdftex libpari
pdftex parallel
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020 Gentoo Linux) (preloaded format=pdftex)
restricted \write18 enabled.
kpathsea: Running mktexfmt pdftex.fmt
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020 Gentoo Linux) (preloaded format=pdftex)
restricted \write18 enabled.
kpathsea: Running mktexfmt pdftex.fmt
rm -f tutorial.std
pdftex tutorial
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020 Gentoo Linux) (preloaded format=pdftex)
restricted \write18 enabled.
kpathsea: Running mktexfmt pdftex.fmt
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020 Gentoo Linux) (preloaded format=pdftex)
restricted \write18 enabled.
kpathsea: Running mktexfmt pdftex.fmt
mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order):
mktexfmt: /etc/texmf/web2c/fmtutil.cnf
mktexfmt: /usr/share/texmf-dist/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes:
mktexfmt: /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-config/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order):
mktexfmt: /etc/texmf/web2c/fmtutil.cnf
mktexfmt: /usr/share/texmf-dist/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes:
mktexfmt: /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-config/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order):
mktexfmt: /etc/texmf/web2c/fmtutil.cnf
mktexfmt: /usr/share/texmf-dist/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes:
mktexfmt: /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-config/web2c/fmtutil.cnf
mktexfmt [INFO]: mktexfmt [INFO]: writing formats under /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-var/web2c
writing formats under /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-var/web2c
mktexfmt [INFO]: writing formats under /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-var/web2c
mktexfmt [INFO]: did not find entry for byfmt=pdftex, skipped
mktexfmt [INFO]: did not find entry for byfmt=pdftex, skipped
mktexfmt [INFO]: not selected formats: 33
mktexfmt [INFO]: total formats: 33
mktexfmt [INFO]: did not find entry for byfmt=pdftex, skipped
mktexfmt [INFO]: not selected formats: 33
mktexfmt [INFO]: total formats: 33
mktexfmt [INFO]: exiting with status 0
mktexfmt [INFO]: exiting with status 0
mktexfmt [INFO]: not selected formats: 33
mktexfmt [INFO]: total formats: 33
mktexfmt [INFO]: exiting with status 0
I can't find the format file `pdftex.fmt'!
I can't find the format file `pdftex.fmt'!
make[1]: *** [Makefile:62: parallel.pdf] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile:40: libpari.pdf] Error 1
I can't find the format file `pdftex.fmt'!
make[1]: *** [Makefile:51: develop.pdf] Error 1
mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order):
mktexfmt: /etc/texmf/web2c/fmtutil.cnf
mktexfmt: /usr/share/texmf-dist/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes:
mktexfmt: /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-config/web2c/fmtutil.cnf
mktexfmt [INFO]: writing formats under /var/tmp/portage/sci-mathematics/pari-2.13.0/homedir/.texlive/texmf-var/web2c
mktexfmt [INFO]: did not find entry for byfmt=pdftex, skipped
mktexfmt [INFO]: not selected formats: 33
mktexfmt [INFO]: total formats: 33
mktexfmt [INFO]: exiting with status 0
I can't find the format file `pdftex.fmt'!
make[1]: *** [Makefile:73: tutorial.pdf] Error 1
make[1]: Leaving directory '/var/tmp/portage/sci-mathematics/pari-2.13.0/work/pari-2.13.0/doc'
make: *** [Makefile:54: docpdf] Error 2
* ERROR: sci-mathematics/pari-2.13.0::gentoo failed (compile phase):
* emake failed
pari 2.13 does not get installed due to a weird texlive error. No idea why it is trying to do something with
pdftex.fmt
Someone else reported a problem with the documentation build in pari a while ago, but neither Francois nor I could reproduce it. If you ever figure out what's going wrong with make docpdf, please let us know so we can fix the ebuild.
pdftex is just broken on my system for no good reason
$ pdftex develop.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020 Gentoo Linux) (preloaded format=pdftex)
restricted \write18 enabled.
kpathsea: Running mktexfmt pdftex.fmt
mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order):
mktexfmt: /etc/texmf/web2c/fmtutil.cnf
mktexfmt: /usr/share/texmf-dist/web2c/fmtutil.cnf
mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes:
mktexfmt: /home/dima/.texlive/texmf-config/web2c/fmtutil.cnf
mktexfmt [INFO]: writing formats under /home/dima/.texlive/texmf-var/web2c
mktexfmt [INFO]: did not find entry for byfmt=pdftex, skipped
mktexfmt [INFO]: not selected formats: 33
mktexfmt [INFO]: total formats: 33
mktexfmt [INFO]: exiting with status 0
I can't find the format file `pdftex.fmt'!
basically you can patch the doc/Makefile in pari to use xe(la)tex and be done with.
https://wiki.gentoo.org/wiki/TeX_Live#eselect_pdftex
makes me wonder whether special steps are needed to enable/disable pdftex
https://wiki.gentoo.org/wiki/TeX_Live#eselect_pdftex
makes me wonder whether special steps are needed to enable/disable pdftex
No, that page is just outdated... eselect-pdftex was removed back in August 2020, and was obsolete before then. There are various hacks floating around to fix fmt issues, but I don't really understand this stuff. Regardless.. if you haven't done anything to break your latex installation, pdftex should just work. (I don't think pari's documentation requires xetex, but I could be wrong.) Maybe @zlogene can suggest something?
I enabled xetex as it has better utf-8 support, I don't recall how; perhaps they don't function together?
OK, I reinstalled dev-texlive/texlive-basic (the package that contains pdftex.fmt) and now pari 2.13 builds.
Doctesting vanilla 9.3.beta1 using all available Gentoo components as
sage -tp 5 --long --all --memlimit=0
gives the failures
sage -t --long --warn-long 223.1 --random-seed=0 src/sage/doctest/test.py # 29 doctests failed
sage -t --long --warn-long 223.1 --random-seed=0 src/sage/tests/cmdline.py # 7 doctests failed
The failures in src/sage/doctest/test.py are all memory-related failures involving subprocess.call. The failures in src/sage/tests/cmdline.py are perhaps formatting issues?
in the 1st coredump in src/sage/doctest/test.py I see openblas, so it could be the Gentoo openblas upgrade to 3.12 (Sage is still on 3.9) that is responsible for this mess.
On gentoo prefix, where I have symbols compiled in, I have
Cython backtrace
----------------
#0 0x00007f98d795e4e0 in __GI___wait4 () at /storage/strogdon/gentoo-rap/var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/posix/../sysdeps/unix/sysv/linux/wait4.c:30
#1 0x00007f98d2417ab1 in print_enhanced_backtrace () at /storage/strogdon/gentoo-rap/var/tmp/portage/dev-python/cysignals-1.10.2/work/cysignals-1.10.2/build/src/cysignals/implementation.c:563
#2 0x00007f98d2417c4d in sigdie () at /storage/strogdon/gentoo-rap/var/tmp/portage/dev-python/cysignals-1.10.2/work/cysignals-1.10.2/build/src/cysignals/implementation.c:589
#3 0x00007f98d2416fcb in sigdie_for_sig () at /storage/strogdon/gentoo-rap/var/tmp/portage/dev-python/cysignals-1.10.2/work/cysignals-1.10.2/build/src/cysignals/implementation.c:164
#4 0x00007f98d2417172 in cysigs_signal_handler () at /storage/strogdon/gentoo-rap/var/tmp/portage/dev-python/cysignals-1.10.2/work/cysignals-1.10.2/build/src/cysignals/implementation.c:262
#5 0x00007f98d821f5b0 in __restore_rt ()
#6 0x0000000000000000 in ?? ()
#7 0x00007f98c67a4b32 in blas_memory_alloc () at /storage/strogdon/gentoo-rap/var/tmp/portage/sci-libs/openblas-0.3.12-r1/work/OpenBLAS-0.3.12/driver/others/memory.c:2793
#8 0x00007f98c67a5678 in blas_thread_init () at /storage/strogdon/gentoo-rap/var/tmp/portage/sci-libs/openblas-0.3.12-r1/work/OpenBLAS-0.3.12/driver/others/blas_server_omp.c:130
#9 0x00007f98c67a4f22 in gotoblas_init () at /storage/strogdon/gentoo-rap/var/tmp/portage/sci-libs/openblas-0.3.12-r1/work/OpenBLAS-0.3.12/driver/others/memory.c:3123
#10 0x00007f98d8212cc0 in call_init () at /storage/strogdon/gentoo-rap/var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/elf/dl-init.c:74
#11 0x00007f98d8212e13 in call_init () at /storage/strogdon/gentoo-rap/var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/elf/dl-init.c:37
#12 0x00007f98d8212db0 in _dl_init () at /storage/strogdon/gentoo-rap/var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/elf/dl-init.c:121
#13 0x00007f98d79c5f20 in __GI__dl_catch_exception () at /storage/strogdon/gentoo-rap/var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/elf/dl-error-skeleton.c:182
#14 0x00007f98d8216d10 in dl_open_worker () at /storage/strogdon/gentoo-rap/var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/elf/dl-open.c:783
#15 0x00007f98d79c5f20 in __GI__dl_catch_exception () at /storage/strogdon/gentoo-rap/var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/elf/dl-error-skeleton.c:208
So maybe openblas?
In Gentoo prefix this
sage -t --long ~/usr/lib/python3.8/site-packages/sage/doctest/test.py
now passes with openblas-0.3.9-r1 where I has 29 failures with openblas-0.3.12-1
I can confirm that going back to openblas-0.3.9-r1 in my "vanilla" case fixes the problem.
OK, so openblas has historically been playing with cgroups for a while to deal with resource issues. May be something has been switched on. Do both of you, have openmp enabled for openblas?
The documentation issue with pari seems to be a migration problem between versions of texlive. Some rebuilding may need to be enforced.
My openblas
[U] sci-libs/openblas
Available versions: 0.3.9-r1^t 0.3.10^t 0.3.12-r1^t {dynamic eselect-ldso index-64bit openmp pthread +relapack test}
Installed versions: 0.3.9-r1^t(12:36:42 PM 11/23/2020)(eselect-ldso openmp -dynamic -index-64bit -pthread -test)
Homepage: http://xianyi.github.com/OpenBLAS/
Description: Optimized BLAS library based on GotoBLAS2
OK, that may cause some rebuild. Can I ask you guys to try pthread instead of opemp and whether that behaves better.
this seems to be platform-dependent; on Homebrew one can use its OpenBLAS 3.12 with openmp in Sage just fine.
All tests passed for
sage -t --long ~/usr/lib/python3.8/site-packages/sage/doctest/test.py
in prefix with sci-libs/openblas -openmp pthread. The doctest failed miserably with the useflags switched. This was with 0.3.12
There may be something subtle at play because of ldso switching. Or something in Gentoo patch set. That would explain that homebrew does work. Although I guess homebrew could have a critical patch :)
https://github.com/Homebrew/homebrew-core/blob/master/Formula/openblas.rb
no patches as far as I can see
openblas-0.3.12-r1 with USE=-openmp pthread on "vanilla"
sage -t --long --warn-long 222.9 --random-seed=0 src/sage/doctest/test.py
**********************************************************************
File "src/sage/doctest/test.py", line 521, in sage.doctest.test
Failed example:
if system() == "Linux":
P = subprocess.Popen(["sage", "-t", "--warn-long", "0", "--random-seed=0", "--memlimit=2000", "memlimit.rst"], stdout=subprocess.PIPE, **kwds)
out, err = P.communicate()
ok = ("MemoryError: failed to allocate" in bytes_to_str(out))
Expected nothing
Got:
Process DocTestWorker-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/local/sage-git/sage/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2191, in run
task(self.options, self.outtmpfile, msgpipe, self.result_queue)
File "/local/sage-git/sage/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2541, in __call__
result_queue.put(result, False)
File "/usr/lib/python3.8/multiprocessing/queues.py", line 88, in put
self._start_thread()
File "/usr/lib/python3.8/multiprocessing/queues.py", line 173, in _start_thread
self._thread.start()
File "/usr/lib/python3.8/threading.py", line 852, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
**********************************************************************
1 item had failures:
1 of 58 in sage.doctest.test
[57 tests, 1 failure, 175.42 s]
So, maybe there is a hardware/platform component. I believe if the memlimit were increased in the doctest things would be fine.
On Mon, 23 Nov 2020 12:46:35 -0800 Dima Pasechnik [email protected] wrote:
https://github.com/Homebrew/homebrew-core/blob/master/Formula/openblas.rb
no patches as far as I can see
Do you have libgomp linked into your build of openblas on homebrew?
Do you have
libgomplinked into your build ofopenblason homebrew?
yes. It comes from gcc 10, by the way.
I wonder whether your Gentoo tries come with gcc 10, or a "normal" gcc 8 from the system? (it's gcc 8 in my case).
I am all on gcc 10 personally. I usually upgrade early to figure problems out.