FFI example error on Ubuntu 22.04

[mahrud]

One example seems to be failing when bottling M2 from the main branch for Ubuntu 22.04 when using brew's libffi:

 -- making example results for "ForeignFunctionPointerType Function"        
 ulimit -c unlimited; ulimit -t 700; ulimit -m 850000; ulimit -s 8192; ulimit -n 512;  cd /tmp/M2-106561-0/25-rundir/; GC_MAXIMUM_HEAP_SIZE=400M "/tmp/macaulay2-20221114-21055-dzcob8/M2-master/M2/BUILD/build-brew/usr-dist/x86_64-Linux-Ubuntu-22.04/bin/M2-binary" -q --no-randomize --no-readline --silent --stop --print-width 77 -e 'needsPackage("ForeignFunctions",Reload=>true,FileName=>"/tmp/macaulay2-20221114-21055-dzcob8/M2-master/M2/Macaulay2/packages/ForeignFunctions.m2")' <"/tmp/M2-106561-0/0___Foreign__Function__Pointer__Type_sp__Function.m2" >>"/tmp/macaulay2-20221114-21055-dzcob8/M2-master/M2/BUILD/build-brew/usr-dist/common/share/doc/Macaulay2/ForeignFunctions/example-output/___Foreign__Function__Pointer__Type_sp__Function.errors" 2>&1
/tmp/macaulay2-20221114-21055-dzcob8/M2-master/M2/BUILD/build-brew/usr-dist/common/share/doc/Macaulay2/ForeignFunctions/example-output/___Foreign__Function__Pointer__Type_sp__Function.errors:0:1: (output file) error: Macaulay2 exited with status code 1
/tmp/M2-106561-0/0___Foreign__Function__Pointer__Type_sp__Function.m2:0:1: (input file)
M2: *** Error 1
 -- 0.908139 seconds elapsed

I can also reproduce this locally on Fedora using systems libffi.

i3 : capture examples "ForeignFunctionPointerType Function"
-- SIGSEGV
-* stack trace, pid: 1050644
 0# stack_trace(std::ostream&, bool) at /home/mahrud/Projects/M2/M2/M2/Macaulay2/bin/main.cpp:127
 1# segv_handler at /home/mahrud/Projects/M2/M2/M2/Macaulay2/bin/main.cpp:244
 2# 0x00007F5367E3EA30 in /lib64/libc.so.6
 3# 0x00007F536B105010
-- end stack trace *-

Why is the trace so short? I'd expect it to at least show something in the ffi code in the interpreter.

cc: @d-torrance

[d-torrance]

I'll see if I can reproduce this -- I haven't tried using brew's libffi.

What version is the Fedora libffi? I noticed brew has the newest version 3.4.4, which I haven't actually tested yet since the development release of Ubuntu still has 3.4.2.

[mahrud]

Fedora also has 3.4.2.

[d-torrance]

I successfully reproduced the issue using the brew build. I'll see if I can figure out what's going on.

[mahrud]

I'm surprised that it worked fine on all Macs, 10.15 through 12!

[mikestillman]

Here is an issue with development branch from today, built on Apple M1 Max laptop, MacOS 12.6. @jkyang92 Was I supposed to add in something for this? The trouble is that valgrind doesn't work with MacOS (at least not the M1 processor). Any suggestions for what to change? @DanGrayson Did you have a fix for this?

FAILED: Macaulay2/d/CMakeFiles/M2-interpreter.dir/M2mem.c.o 
/opt/homebrew/bin/ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DNDEBUG -DOM_NDEBUG -DSING_NDEBUG -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/d -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/c -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/Macaulay2/d -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/mathic -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/memtailor -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/mathicgb -I/opt/homebrew/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/libxml2 -I/opt/homebrew/include/eigen3 -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/usr-host/include -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/system -isystem /opt/homebrew/include -isystem /opt/homebrew/opt/tbb/include -isystem /opt/homebrew/opt/libffi/include -isystem /opt/homebrew/Cellar/fflas-ffpack/2.5.0/include -isystem /opt/homebrew/opt/libomp/include -isystem /opt/homebrew/Cellar/givaro/4.2.0/include -isystem /opt/homebrew/opt/gmp/include -O2 -g -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -mmacosx-version-min=12.6 -Wuninitialized -g --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/usr-host/include -I/Users/mike/src/M2-current-branches/M2-development/M2/include -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/include -Wall -Wextra -Wfatal-errors -Wcast-qual -Wno-unused-parameter -Wno-attributes -Wno-unused-command-line-argument -Wno-cast-qual -Wno-sign-compare -Wno-uninitialized -Wno-unused -Wno-array-bounds -Wno-parentheses-equality -Xpreprocessor -fopenmp /opt/homebrew/opt/libomp/lib/libomp.dylib -std=gnu11 -MD -MT Macaulay2/d/CMakeFiles/M2-interpreter.dir/M2mem.c.o -MF Macaulay2/d/CMakeFiles/M2-interpreter.dir/M2mem.c.o.d -o Macaulay2/d/CMakeFiles/M2-interpreter.dir/M2mem.c.o -c /Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/d/M2mem.c
/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/d/M2mem.c:209:5: fatal error: use of undeclared identifier 'OrigFn'
    OrigFn fn;
    ^

[mahrud]

Here is an issue with development branch from today, built on Apple M1 Max laptop, MacOS 12.6.

Is this related to the foreign function interface?

[jkyang92]

The proximate problem is valgrind.h doesn't support M1 Macs, so it doesn't correctly define OrigFn. I'll look at a good fix, but the general idea is still going to be to ifdef it off on unsupported systems. So no it has nothing to with FFI

[d-torrance]

I'm not sure if this is the same bug as the bug in the Ubuntu brew build (which I can reproduce, but am still stumped by...), but there's also an issue with ffi function closures in M1 Macs, as reported by @mikestillman. I'm borrowing an M1 laptop from the university and was able to reproduce it:

i1 : needsPackage "ForeignFunctions"

o1 = ForeignFunctions

o1 : Package

i2 : (value (foreignFunctionPointerType(int, int)) abs) 5
zsh: bus error  usr-dist/aarch64-Darwin-macOS-12.6.1/bin/M2

[mikestillman]

On development branch today, I just compiled using autotools, on Apple M1, commenting out the last 5 functions in Macaulay2/d/M2mem.c. At that point, it creates the exexcutable, and almost all examples work, except for ForeignFunctions, where there are 5 examples whose code fails.

Here is one of the 5 errors:

stdio:1:7:(3): error: dlopen(libmps.dylib, 0x0001): tried: '/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/arm64-appleclang/usr-dist/arm-Darwin-macOS-12.6/bin/../lib/Macaulay2/lib/libmps.dylib' (no such file), '/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/arm64-appleclang/usr-dist/arm-Darwin-macOS-12.6/bin/../lib/libmps.dylib' (no such file), '/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/arm64-appleclang/usr-dist/arm-Darwin-macOS-12.6/bin/../lib/Macaulay2/lib/libmps.dylib' (no such file), '/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/arm64-appleclang/usr-dist/arm-Darwin-macOS-12.6/bin/../lib/libmps.dylib' (no such file), 'libmps.dylib' (no such file), '/usr/local/lib/libmps.dylib' (no such file), '/usr/lib/libmps.dylib' (no such file), '/private/var/folders/wb/8v4mm0j52pq9pf5gkr8f23z40000gr/T/M2-83569-0/11-rundir/libmps.dylib' (no such file)

[feynhat]

@d-torrance Was this fixed not included in release 1.21? I am getting the same error on Arch linux.

[d-torrance]

@d-torrance Was this fixed not included in release 1.21? I am getting the same error on Arch linux.

Yes, it should have been included in 1.21. (It's not really a "fix" -- the feature was just disabled.)