WIP Let the GMT/MEX toolbox be an optional supplement in the main GMT project

[PaulWessel]

This is the first import of files and settings that seems relevant from the old autoconf/Makefile-driven gmtmex project.

First goal is to have this not break regular GMT built when GMT_BUILD_GMTMEX is not set. Looks like I have achieved that.

[PaulWessel]

This PR will set the right matlab/mex libraries and add them as optiona libraries, and add the mex include dir as an included dir.

*  Build module links         : 
*  MATLAB Application         : /Applications/MATLAB_R2019a.app
*  MATLAB include dir         : /Applications/MATLAB_R2019a.app/extern/include
*  MATLAB MEX library         : /Applications/MATLAB_R2019a.app/bin/maci64/libmex.dylib
*  MATLAB MX library          : /Applications/MATLAB_R2019a.app/bin/maci64/libmx.dylib
*  Found Ghostscript (gs)     : yes (9.50)

It seems to build the two object files under dbuild/src:

ls CMakeFiles/supplements.dir/gmtmex/
gmtmex.c.o		gmtmex_parser.c.o

but there are challenges since these are not really supplemental modules like meca and velo, but only library stubs apart from an executable (like gmt). So need to make some changes. First up are these failures

[7/269] Generating gmt_supplements_moduleinfo.h
CMake Warning at /Users/pwessel/GMTdev/gmt-dev/cmake/modules/GmtGenExtraHeaders.cmake:90 (message):
  THIS_MODULE_CLASSIC_NAME is empty in gmtmex/gmtmex.c
Call Stack (most recent call first):
  /Users/pwessel/GMTdev/gmt-dev/cmake/modules/GmtGenExtraHeaders.cmake:131 (gen_gmt_moduleinfo_h)

which is true since these are not modules. Thoughts on how to bypass that step for this supplement, @seisman ? Maybe a getter plan is to add the gmtmex_parser.c to the main GMT library building instead of having it inside the supplements.so? So conditionally add that file to libgmt.dylib. Can I do that inside the gmtmex CMakeLists.txt file?

[PaulWessel]

So this commit that you suggested causes no errors (I am not involving gmtmex.c yet and its product), and the 4 experted GMTMEX functions ended up in supplements.so:

nm -gU supplements.so | grep GMTMEX
00000000001887b0 T _GMTMEX_Get_Object
00000000001844d0 T _GMTMEX_Set_Object
0000000000184360 T _GMTMEX_objecttype
0000000000184330 T _GMTMEX_print_func

Since gmtmex certainly is "supplemental" that that is probably fine. I think maybe more things from gmtmex.c could go into the parser so that we could have as small a driver void mexFunction (int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) as possible.

[PaulWessel]

Hi @seisman, I am close (picking up on this PR again). Since it is not a supplement it wont be added to SUPPL_EXTRA_DIRS, and instead, like the MB glue, I just add it to libgmt via

if (GMT_BUILD_GMTMEX)
	set (GMT_GMTMEX_SRCS gmtmex/gmtmex_parser.c gmtmex/gmtmex.c gmtmex/gmtmex.h)
endif (GMT_BUILD_GMTMEX)

and add GMT_GMTMEX_SRCS to the list of things lin libgmt. This builds a libgmt with the right glue and otool -L shows links to my own matlab mx and mex libraries. Futhermore, this seems to be the end product! If I add a link gmtmex.mexmaci64 -> libgmt.6.dylib then matlab runs the gmt/mex toolbox, because all matlab does is (1) check for correct extension, (2) look for a function called mexFunction and launch it in matlab.

Instead of installing a program, I would like to add a symbolic link gmtmex.mexmaci64 -> libgmt.6.dylib in cmake. Is there a special command for that?

[seisman]

Try if this works:

file (CREATE_LINK libgmt.6.dylib gmtmex.mexmaci64 SYMBOLIC)

[PaulWessel]

It did create but in wrong dir, etc. This is the top of the build dir:

(base) pwessel@macnut:~/GMTdev/gmt-dev/dbuild-> ls -lg gmtmex.mexmaci64 lrwx--x--x 1 wessel 14 Sep 24 15:42 gmtmex.mexmaci64 -> libgmt.6.dylib

Maybe more like

file (CREATE_LINK${GMT_LIBDIR}/${LIB_PREFIX}gmt.${GMT_LIB_SOVERSION}.${CMAKE_SHARED_LIBRARY_SUFFIX} ${GMT_LIBDIR}//gmtmex.${MEX_EXT} SYMBOLIC)

Is there any shorter expressions for path to library instead of the above? Will try.

[PaulWessel]

Timing issues? Changing the dir seems like it is trying to create link before lib is ready:

CMake Error at src/CMakeLists.txt:570 (file): file failed to create symbolic link 'lib/gmtmex.mexmaci64': no such file or directory

[PaulWessel]

We have other symbolic links being made by cmake. hack based on this instead?

# symlink to gmt-wrapper in bindir and libdir:
if (UNIX AND GMT_INSTALL_NAME_SUFFIX)
	get_target_property(_gmt_wrapper_name gmt OUTPUT_NAME)
	install (CODE "
	execute_process (
		COMMAND ${CMAKE_COMMAND} -E create_symlink
		\"${_gmt_wrapper_name}\" \"\$ENV{DESTDIR}\${CMAKE_INSTALL_PREFIX}/${GMT_BINDIR}/gmt\")
	")
endif (UNIX AND GMT_INSTALL_NAME_SUFFIX)

[seisman]

symlink to gmt-wrapper in bindir and libdir:

if (UNIX AND GMT_INSTALL_NAME_SUFFIX) get_target_property(_gmt_wrapper_name gmt OUTPUT_NAME) install (CODE " execute_process ( COMMAND ${CMAKE_COMMAND} -E create_symlink "${_gmt_wrapper_name}" "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}/${GMT_BINDIR}/gmt") ") endif (UNIX AND GMT_INSTALL_NAME_SUFFIX)

Yes, should have an install command like this one.

[PaulWessel]

Hi @seisman, were did we leave your experiments on gmtmex etc? I want to return to this after the PR on basemap placements etc. @joa-quim and I have realized tehre is nothing in the gmtmex dir that is independent of Matlab libs and includes so all we plan to do is just build the gmtmex.extension shared lib and in that sense gmtmex is not a supplement but a directory with optional code for matlab.

[seisman]

were did we leave your experiments on gmtmex etc?

What do you mean? What experiments? Experiments on the macOS bundle https://github.com/GenericMappingTools/gmt/issues/1930#issuecomment-698683500?

[PaulWessel]

Maybe I am dreaming (well, since I cannot find the issue and there is no PR apart from my WIP), but I could swear you tested something that involved gmtmex and since I was in the middle on my PR I stopped to see how that went - I am sure motivated by the bundle failyre in PyGMT.

[joa-quim]

Yes, that relates with the gmtmex on Mac because it would greatly make it possible to distribute gmtmex in the bundle.

[seisman]

The only related issue I can find is https://github.com/GenericMappingTools/gmt/issues/1930.

[PaulWessel]

OK, maybe I extrapolated that too much. Were you successful in building a bundle that worked with pyGMT?

[seisman]

I can't build a working bundle from the master branch recently. The macOS bundle is successfully built but when running gmt, it says "Illegal instruction".

[PaulWessel]

Hi @seisman, @joa-quim, @meghanrjones , I have resurrected this WIP PR, updated from master, and simplified the mex code (Only two source files: gmtmex_api.c has everything, while gmtmex.c is 3-4 lines). If GMT_BUILD_GMTMEX is defined in ConfigUserAdvanced.cmake, then we determine the MATLAB libs and include files, and compile and include gmtmex_api.c into the GMT API. GMT_MexFunction is the only one exported. Now, it is very easy to build the actual MexFunction needed by Matlab via the gmtmex.c stub:

#include "mex.h"
extern void GMT_mexFunction (int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]);
void mexFunction (int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
	GMT_mexFunction (nlhs, plhs, nrhs, prhs);
}

I have added a CMakeLists.txt file to src/gmtmex to build the mexfunction. Here is status so far:

1. Configuring: This seems to work (as before) and with GMT_BUILD_GMTMEX we report (when run on my MacPro):

*  MATLAB Application         : /Applications/MATLAB_R2021a.app
*  MATLAB include dir         : /Applications/MATLAB_R2021a.app/extern/include
*  MATLAB MEX library         : /Applications/MATLAB_R2021a.app/bin/maci64/libmex.dylib
*  MATLAB MX library          : /Applications/MATLAB_R2021a.app/bin/maci64/libmx.dylib

2. Building libraries: Same as before, but now GMT_MexFunction is exported by libgmt:

nm -gU libgmt.dylib | grep mexF
0000000000232660 T _GMT_mexFunction

3. Building mexfunction shared library: This is not happening. I get no errors so presumably none of the things in gmtmex/CMakeLists.txt are attempted (?):

if (GMT_BUILD_GMTMEX)
	# Add the mexfunction as a shared library
	add_library (GMTmexfunction SHARED gmtmex.c)
	# Set the Matlab include file directory
	target_include_directories (GMTmexfunction ${MATLAB_INC})
	# Link with the Matlab and mex libraries and gmt library
	target_link_libraries (GMTmexfunction gmtlib ${MATLAB_LIB1} ${MATLAB_LIB2})
	# Set output properties of shared library
	set_target_properties (GMTmexfunction
		PROPERTIES
		OUTPUT_NAME gmtmex
		RUNTIME_OUTPUT_NAME gmtmex
		PREFIX "")
	# Add the install target
	install (TARGETS GMTmexfunction
		RUNTIME DESTINATION ${GMT_BINDIR}
		COMPONENT Runtime)
endif (GMT_BUILD_GMTMEX)

I am sure there are (a) mistakes in my gmtmex/CMakeLists.txt file and (b) maybe more is needed in src/CMakeLists.txt as well. Hopeing you experts can switch to this branch, set GMT_BUILD_GMTMEX and try it. @joa-quim: I know you have your own setup so perhaps not of interest, and presumably the "export" in the stub above needs something like __declspec(dllexport) anyway.

4. Manual build. I can do this on the command line (skipping the -L and -I args for clarity):

xcrun clang -undefined error -arch x86_64 -bundle -lmx -lmex -lgmt gmtmex.c -o gmtmex.mexmaci64

or

gcc -shared -lmx -lmex -lgmt gmtmex.c -o gmtmex.mexmaci64

or inside Matlab

>> mex gmtmex.c -lgmt

and all produce (apparently) good mexfunctions.

[PaulWessel]

So fixed many of these self-inflicted wounds. Now things build. However, a few remaining issues:

1. CM insists on putting the gmtmex.mexmaci64 file in lib not bin, even though I say BINDIR.
2. Matlab says it cannot find GMT_mexFunction when running the mexfunction that is linked to libgmt:

I guess (1) is solvable while (2) seems to indicate Matlab is looking for the library function to call in the mex file and not in the shared library.

>> gmtmex ('--help')
Invalid MEX-file '/Users/pwessel/GMTdev/gmt-dev/rbuild/gmt6/bin/gmtmex.mexmaci64': dlopen(/Users/pwessel/GMTdev/gmt-dev/rbuild/gmt6/bin/gmtmex.mexmaci64, 6): Symbol not found: _GMT_mexFunction
  Referenced from: /Users/pwessel/GMTdev/gmt-dev/rbuild/gmt6/bin/gmtmex.mexmaci64
  Expected in: /Users/pwessel/GMTdev/gmt-dev/rbuild/gmt6/lib/libgmt.6.dylib
 in /Users/pwessel/GMTdev/gmt-dev/rbuild/gmt6/bin/gmtmex.mexmaci64

[joa-quim]

But we don't want to make gmt dependent on having Matlab immutable. If gmt.dll includes the mex it will become dependent on finding the Matlab mex libs. If Matlab is removed and just drop out from path GMT will no longer work.

[PaulWessel]

This is mostly for me building and debugging GMT. We are not using GMT_BUILD_GMTMEX when making the bundle.

[PaulWessel]

OK, I think I got it all to work. Both gmt.m and gmt.mexmaci64 are placed in the bin dir. No mex inside the libs. gmt_init.c had mex.h optionally included - now removed. Will be testing the gmt mex commands to see if the shared lib issues are still present.

[PaulWessel]

Update May 18. 2021: This branch is in sync with master but has a few more things:

1. If a developer sets GMT_MEXBUILD to true in ConfigUserAdvanced.cmake then the src/gmtmex supplement is included in the build. This makes it easy to debug gmtmex in Xcode on macos.
2. If a user who installs the macos bundle from us runs gmt_mexbuild.sh in the terminal that opens when you double-click the GMT logo, then I do some gymnastics with all our shared libraries (basically making them use @rpath instead of @executable_path) and then I compile and build gmtmex. This requires the user to have Xcode CLI installed.

After that, the GMT/MEX toolbox works very well (i.e., no longer any path or shared library version crashes). There are a few minor things I noticed:

1. The WL_example_2.sh has odd characters rather than degree symbols in the PDF - probably a PS_CHAR_SET issue
2. The WL_example_3.sh gives sea surface temperature colorbar in 100th's of degrees (2500-3500 etc).

So this is a viable path and much better than what gmt_prepmex.sh does (renames libraries and duplicates to /opt/gmt which then is used to build gmtmex via a separate repo and non-cmake configure script. As long as the user installs the Xcode CLI then the GMT/MEX toolbox is just that one command away.

What I would like to do is to build the bundle with GMT_MEXBUILD enabled. This will build gmtmex.mexmaci64 which will have @rpath to the Matlab/mex shared libraries. However, right now, that file, as cmake and cpack builds it, has hardwired paths to my shared libraries:

/Users/pwessel/GMTdev/gmt-dev/build/src/gmtmex/gmtmex.mexmaci64 (compatibility version 0.0.0, current version 0.0.0)
/Users/pwessel/GMTdev/gmt-dev/build/src/libgmt.6.dylib (compatibility version 6.0.0, current version 6.2.0)
@rpath/libmx.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libmex.dylib (compatibility version 0.0.0, current version 0.0.0)
/opt/local/lib/libnetcdf.19.dylib (compatibility version 19.0.0, current version 19.0.0)
...

while the gmt executable has the expected @executable_path:

@executable_path/../lib/libgmt.6.dylib (compatibility version 6.0.0, current version 6.2.0)
...

The difference has to do with gmt being a program and gmtmex.mexmaci64 being a shared library (plugin). We could possibly figure out the target settings to make this go away.

The cmake docs talks about using rpath and has many RPATH related settings, but I cannot make sense of them, and no attempt so far changes the executable_path to instead use rpath. IF we can figure out how to have cpack do the right thing here then I think the bundle could come with a working GMT/MEX toolbox like the Windows installer does.

I don't want to hold up 6.2.0 because of this though but at least it looks like we can figure this out finally. Also, my gmt_mexbuild.sh commands could probably be inserted as a custom script into our cmake setup before the cpack step to do the rpath stuff for us, but it seems like this should be entirely unnecessary... Just pinging @joa-quim @meghanrjones @seisman on this.