Upgrade to SNAP 9.0.0 (#16)

[thierryweo]

Hello, I upgraded SNAP to 9.0.0 as in this ticket #16

There were a few changes necessary

1. /usr/local/snap/bin/snap --nosplash --nogui --modules --update-all https://github.com/mundialis/esa-snap/compare/master...thierryweo:esa-snap:master#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557L72 does not terminate. Fix is to write a short shell command to look for a special line and then terminate the process (update.sh)
2. /usr/local/snap/bin/snappy-conf /usr/bin/python3 command does no terminate so same as in point 1. wait for last line and terminate java process
3. openjdk8is not necessay in final Docker image as it is already baked in the base image

Test Image is pushed here to test https://hub.docker.com/repository/docker/thierrydoc/esa-snap/general I am not 100% sure everything is setup correctly:

1. Following message appears

INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL not found on system. Internal GDAL 3.2.1 from distribution will be used. (f0)
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.2.1 set to be used by SNAP.
Native library load failed.
java.lang.UnsatisfiedLinkError: /root/.snap/auxdata/gdal/gdal-3-2-1/lib/jni/libgdalalljni.so: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /root/.snap/auxdata/gdal/gdal-3-2-1/lib/jni/libgdal.so)
SEVERE: org.esa.s2tbx.dataio.gdal.GDALLoader: Failed to initialize GDAL native drivers. GDAL readers and writers were disabled.java.lang.reflect.InvocationTargetException
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.```


Please let me know if there are problems I will look into them happily

[mmacata]

HI @thierryweo, thank you for looking into this! Sorry for the long time until testing. When I run your changes locally, I get

sed: /src/snap/esa-snap_all_unix_9_0.sh: No such file or directory
sed: /src/snap/esa-snap_all_unix_9_0.sh: No such file or directory
sed: /src/snap/esa-snap_all_unix_9_0.sh: No such file or directory
sed: /src/snap/esa-snap_all_unix_9_0.sh: No such file or directory
sed: /src/snap/esa-snap_all_unix_9_0.sh: No such file or directory
sh: can't open '/src/snap/esa-snap_all_unix_9_0.sh': No such file or directory
sed: /usr/local/snap/etc/snap.conf: No such file or directory
/src/snap/install.sh: line 43: /usr/local/snap/bin/snappy-conf: not found
python3: can't open file 'setup.py': [Errno 2] No such file or directory
sed: snappy.ini: No such file or directory
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'snappy'
Traceback (most recent call last):
  File "/src/snap/about.py", line 3, in <module>
    from snappy import ProductIO
ModuleNotFoundError: No module named 'snappy'
Removing intermediate container b86b7894100f
 ---> 3f30f3824db9
Successfully built 3f30f3824db9
Successfully tagged esa-snap:16-master

The first errors appear because of the additional _0 which was added to the installer script name. When I correct these, I still get the error. Strange that they didn't appear in your build!?

Regarding GDAL: the sentinel-2 toolbox brings its own GDAL version which is too old for some use cases. Therefore we mainly build and use the branch named s1tbx which only installs the sentinel-1 toolbox. There already a newer alpine version is used together with openjdk from package sources again instead of the patched version.

I will look into the s1tbx branch to update with help of your changes.

Could you please try again to build your branch and see if you have the same errors? Thank you!

[johntruckenbrodt]

Hi all. Great to see this happening. I think the following adjustment needs to be made:

- sed -i 's+ bin/unpack200+ $JAVA_HOME/bin/unpack200+g' /src/snap/esa-snap_all_unix_${SNAPVER}_0.sh
+ sed -i 's+ bin/unpack200+ $JAVA_HOME/bin/unpack200+g' /src/snap/esa-snap_all_unix_${SNAPVER}_0_0.sh

[thierryweo]

@mmacata I will test and thanks @johntruckenbrodt for the hint. Not sure what i did in my build I will investigate

[thierryweo]

@mmacata and @johntruckenbrodt BUilding is fine. The only problem is the gdal warning that the embedded version is used. Which is normal as there is no other gdal installed. Did you try building without cache?

I use another image for version 9.0 as the Alpine images themselves are quite old and also java version is an old one. I will try to publish another PR soon with improved alpine images and also useing latest gdal.

[mmacata]

@thierryweo hello again, sorry for the noise. I tried again today and the build was more or less successful (except of the broken GDAL dependency) and I assume the problems yesterday appeared because of network instabilities from my side. Hope you didn't spend much time already investigating.

IIRC, GDAL conflicts where there before, so it would be fine for me to be merged. But if you like to look into them, that would also be great of course. As I wrote above - the s2tbx brings its own GDAL which conflicts with alpine GDAL - or at least it did a while ago which is why we came up with the s1tbx branch.

Either way I already tagged and pushed the now correctly built image to

• mundialis/esa-snap:9.0

And regarding the naming in the sed commands for ...${SNAPVER}_0.sh and ...${SNAPVER}_0_0.sh - before they were necessary so that the install script could run successfully. Now as they are not executed anymore, I assume the script changed in these parts or the changes are not needed anymore for other reasons and they can be removed. Only the last line in the block sh /src/snap/esa-snap_all_unix_${SNAPVER}_0.sh -q -varfile /src/snap/response.varfile might now lead to uncertain configuration as the varfile is not read correctly and I assume that default values are used instead, so it might be better to explicitly fix this line and use the config again.

[mmacata]

I updated for the s1tbx branch as well. And I realized that renaming ${SNAPVER}_0.sh to ${SNAPVER}_0_0.sh is not necessary in your case, as the file name is specified with wget and still is the old name as well.