about .ci/build_gdl.sh

[alaingdl]

Thanks to @pjb7687 and @GillesDuvert I discovered last days the great script .ci/build_gdl.sh (unclear for me who started to write that, thanks !))

I succeed to run it on several computers : Debian 9 & 10, Ubuntu 18.04.5 LTS, 20.04.2 LTS and Fedora 28.

I have few suggestions :

• having a verbose no sudo version to list missing packages (in the prep mode) (the idea behind is to be able to prepare a message for the IT/support team to ask/request they add some official packages)
• not immediately copy at the end of build, having another option to do that (and give clear way to pass -DCMAKE_INSTALL_PREFIX= (I do consider the current way write in un-expeted location some code, I am not comfortable on that.)
• being more explicit how to pass Cmake options (I noticed ${CMAKE_ADDITIONAL_ARGS[@] but did not succed to use it)
• maybe rename "check" to "test" to be consistent with new "make test"

Soon we will be able to remove my old scripts in scripts/ (minimum_script4gdl.sh, rpm4gdl.sh, aptget4gdl.sh & which_packages_to_add.sh)

On Debian 9 & Ubuntu 18, I do have few troubles with few packages : libncurses-dev libwxgtk3.0-gtk3-dev libtiff-dev libeccodes-dev shapelib

• shapeline (provided by proj-data ?), I can compile without it
• libwxgtk3.0-gtk3-dev : serious complains on Debian 9 (and not useful)
• libncurses-dev : libncurses5-dev instead ?
• libeccodes-dev : I can compile without it

Extra comments which may required others Issues :

• X11 still checked somewhere but not really used/needed (unclear for me ! @GillesDuvert ?? But we need to keep it for help,/device and many others situations
• MPI ? Are we really using it around ?
• gdl_idl_fl.pro is in testsuite/ but it is used in src/pro/utilities/gdl_status.pro (maybe moving the first one in the same sub dir ?)

[GillesDuvert]

Concerning X11, and indeed HELP,/DEVICE , the status is as follows:

• the default is to have the interactive graphic device use wxWidgets, to provide a uniform user interface on all machines.
• BUT this device is currently called "X" on linux, "WIN" on windows and "MAC" on mac, because some legacy codes rely on this name to test what kind of machine is present. I do not what to do (what the user expects). I believe I should modify the behaviour and have all and every GDL say that the device is 'X'. (and get back the other behaviour with a --switch) but need advice.
• X11 can be compiled in GDL as supplementary graphic device, and in this case, and only if the --no-use-wx is present, the PLOT windows will use X11, and the WIDGET_DRAW window will use (of course) wxWidgets. I am not sure if this is really useful. Without the wxWidgets library (custom install), GDL will use X11 on linux and mac (if present), and will be (at best) without graphics on Windows. When --no-use-wx is used, and wxWidgets is present, it is still possible to open a wxWidgets 'PLOT' window, as it is called 'MAC' :

$ gdl --no-use-wx
 GDL> help,/dev
Available Graphics Devices: NULL PS SVG Z MAC X 
Current graphics device: X

The reason why it is called 'MAC' instead of, e.g., 'WX' is because this is a known device name that will not (should not) cause troubles when running (serious) legacy code.

All this is perhaps superfluous and we propably should have only wxWidgets and call it 'X'. Please advice, as it may be the good time to really unify the code and behaviour.

[GillesDuvert]

definitely shapelib is needed. MPI I would leave it to custom install, but it can be handled automatically by the build script: if found, use it, if not, too bad, but do not enforce its presence at the 'prep' level? gdl_status should not need gdl_idl_fl as it is a gdl-only procedure. In truth, a number of test files depend on procedures testing of we are gdl, idl, or fl (?) but this is irrelevant and error-prone, as they do not work outside "testsuite". It is safer to use the simple test DEFSYSV, '!gdl', exists=isGDL everywhere it is needed, which is only IMHO in some comparisons to IDL outputs. Of course gdl_idl_fl will still be useful in the benchmark/ procedures. Strangely, banner_for_testsuite has also such idl vs. gdl tests that do not seem relevant those days?

[GillesDuvert]

being able to install gdl at some prefixed location with -DCMAKE_INSTALL_PREFIX is indeed much needed.

[pjb7687]

Regarding Shapelib, https://github.com/gnudatalanguage/gdl/commit/233ce87fbb7fac8632cb4eaad88b09150c2db285 should work to find shapelib on Debian 9.

I have tested on:

~/gdl# lsb_release -d
Description:    Debian GNU/Linux 9.13 (stretch)

and

~/gdl# lsb_release -d
Description:    Ubuntu 18.04 LTS

Both have no problems with other packages. It will be helpful if you can post the exact version of the distros so that I can test it.

[pjb7687]

And now I think it would be better to move build_gdl.sh to scripts directory. I started writing it in .ci because I thought it is going to be solely used by CI script... But now, since we are using it in other environments too, it would make sense to move it to a more visible place.

[pjb7687]

Regarding MPI, I started another discussion: https://github.com/gnudatalanguage/gdl/discussions/983

[alaingdl]

I do not remember why cmake should be above >= 3.7 but all is running perfectly on Ubuntu 16.04.7 LTS with Cmake 3.5.1 (I just add by myself libshp-dev for the script)

This is an important point for me :smile:

[pjb7687]

I do not remember why cmake should be above >= 3.7

https://github.com/gnudatalanguage/gdl/pull/976/files#diff-9bc9340de4c56dbc56003ebf42308b27934cbe02489aa5606eff312484fe0b64R20

We can revert back to make check if you think it is important to support CMake 3.5.

[GillesDuvert]

And now I think it would be better to move build_gdl.sh to scripts directory. I started writing it in .ci because I thought it is going to be solely used by CI script... But now, since we are using it in other environments too, it would make sense to move it to a more visible place.

indeed.

[alaingdl]

no no no !

I am not sure to understand you point.

I tried the current version of the script build_gdl.sh on Ubuntu 16.04.7 LTS with Cmake 3.5.1 and it worked fine to the end, including test (build_gdl.sh check)

and on the other side, I just need to make (cmake .. -DPYTHON=off ) and then make test without trouble

Maybe we can just add a simple flag on cmake version deactivating Python for cmake before 3.7 ?!

For me it is important to be able to say GDL can be compile in U16.04 LTS if you accept it is without python

[pjb7687]

The CMake 3.7 requirement is completely irrelevant to Python, it is only related to the test (to compile external library 'libtest_ce.so' before running the test).

It's weird you were successful to run build_gdl build with CMake 3.5.1, since it should fail here: https://github.com/gnudatalanguage/gdl/blob/master/CMakeLists.txt#L10

[alaingdl]

Yes, sorry if I do not clearly explain that : I edited the CMakeLists.txt file (changing 3.7 into 3.5) to go ahead.

Maybe 'libtest_ce.so' is not processed but I did not noticed and it is just a detail for me. (I will look later at that but I am focused on OSX now :grin:)

[alaingdl]

Sorry for the noise but successful compilation :yum: using the script build_gdl.sh (I cannot run then tests since I am connected through ssh)

$ sw_vers 
ProductName:	macOS
ProductVersion:	11.3.1
BuildVersion:	20E241

[alaingdl]

Various issues with the script :

• does not work on a fresh Debian Sid (testing) today

[build_gdl.sh] Building GDL (Release)...
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is unknown
[...]
CMake Error at CMakeLists.txt:12 (project):
  No CMAKE_CXX_COMPILER could be found.

May it is just that g++ is not installed on this new VM ? (dedicated to tests for GDL) Yes, just doing "sudo apt-get install g++" soled the problem --> To be add into the apt list :smile:

But question ! We don't need a C compiler, we just need a C++ compiler, isn't it ?

• Does not work on Mac OS X 10.14.6 (but works perfectly on BigSur 11.3.1 with same protocol ;)

-- The CXX compiler identification is Clang 12.0.0
CMake Error at /usr/local/Cellar/cmake/3.20.5/share/cmake/Modules/CMakeTestCCompiler.cmake:66 (message):
  The C compiler
    "/usr/local/opt/llvm/bin/clang"
  is not able to compile a simple test program.

• it is really a problem for me that the script works in ../../build because ** it might be outside the sub-dir where we do have the clone of gdl ** if a previous compilation was done (eg the day before) it does conflit with it

The question is : as an end-user, where should I be suppose to run this script ?

++ mkdir -p /Users/alain/GDL/gdl-1.0.0rc3-git210705/cibuild/../.ci/../../build

(I would prefer it works in cibuild/ then, evntually, copy somewhere different)

• I really would like we separate compilation and the copy

[pjb7687]

• does not work on a fresh Debian Sid (testing) today

At the moment you will need to install build-essentials and cmake beforehand. The script currently doesn't support installing toolchains yet - and yes we can add this functionality soon.

*Does not work on Mac OS X 10.14.6 (but works perfectly on BigSur 11.3.1 with same protocol ;)

It would be nice to see the failure log.

*it is really a problem for me that the script works in ../../build because

Yeah this is because I started writing the script as a CI script, not a general-purpose one... And I just didn't want to mess with the git source tree. I will try to fix it this weekend.

*I really would like we separate compilation and the copy

Copy that. 😆 I am also going to separate configure (cmake) and build too.

[alaingdl]

Concerning Mac OS X 10.14.6, because it is a shared computer, I have to check whether I have some issue on writing permissions on the Brew side.

Concerning Debian Sid, I did not know this build-essentials. Installing g++ was enough (but many others packages come with g++ and the others ones)

In any case, the script is a big step forward. We have to consider the BSD case too ... soon @thierry-FreeBSD :grin:

[alaingdl]

More details on my issue on OSX 10.14.5 : The message is :

    ld: unknown option: -platform_version

https://github.com/Homebrew/homebrew-core/issues/52461

[alaingdl]

Ok , thanks you @pjb7687, no trouble so far, as long we don't forgot to make a "configure" before a "build" :sweat_smile:

Few comments :

• may-be we have to mention when param when using the script are "stupid" ? (toto). But the current one line is smart when no param. May-be a small help ?!

• On the Linux side, no surprise on U 16, 18, 20, Deb 9 & 10, fc28. Perfect.

• On MSwin, expected some feedbacks soon @eloirozier @bouloukaouze : are Widgets OK ?

• On OSX, various troubles : clang 12.0.1 via Brew in OSX 10.14.5 / 10.14.6 is broken (as mention just above) but the default one (10) is OK, but without OpenMP. Basic plots OK, but no widgets.

I have a doubt : what are exactly GDL_DIR and ROOT_DIR ?

A.