Infinite loop while downloading DEM in mundialis/esa-snap:ubuntu docker image

[konstantinos2018]

Hi,

I am using the mundialis/esa-snap:ubuntu as a base image for building my own image. My algorithms run a typical pre-processing routine on Sentinel-1. When arriving at the Range-Doppler Terrain Correction step my workflow gets stuck in the following infinite loop where it tries to retrieve a DEM:

INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://download.esa.int/step/auxdata/dem/SRTM90/tiff/srtm_39_03.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving https://download.esa.int/step/auxdata/dem/SRTM90/tiff/srtm_39_03.zip
INFO: org.esa.snap.core.dataop.dem.ElevationFile: http retrieving http://skywatch-auxdata.s3-us-west-2.amazonaws.com/dem/SRTM90/tiff/srtm_39_03.zip
SEVERE: org.esa.snap.core.dataop.dem.ElevationFile: Server returned HTTP response code: 403 for URL: http://skywatch-auxdata.s3-us-west-2.amazonaws.com/dem/SRTM90/tiff/srtm_39_03.zip

I found some solutions in ESA STEP Forum such as this but they are not applicable in this case.

The strange part is that there is instability in the following sense:

Case 1

1. Run a container
2. Run my algorithm
3. Gets stuck and CTRL+C
4. Re-run algorithm
5. Gets stuck and CTRL+C
6. Problem never gets solved

Case 2

1. Run a container
2. Run my algorithm
3. Gets stuck and CTRL+C
4. Re-run algorithm
5. Works perfectly from now on and the infinite loop never shows up again

Case 3

Running Terrain Correction from GUI -from another machine though- always works.

Case 4 (UPDATE)

1. Run a container
2. Run my algorithm (with SRTM 3Sec as dem name)
3. Gets stuck and CTRL+C
4. Run my algorithm (with SRTM 3 Sec as DEM name: Notice the spacebar added)
5. Algorithm spits errors (see message below) and CTRL+C
6. Run my algorithm (with SRTM 3Sec as DEM name again)
7. Works perfectly from now on and the infinite loop never shows up again (same happens if I use another valid DEM instead of SRTM 3Sec for the first run of my algorithm)

Message in step 5:

INFO: org.esa.snap.core.gpf.common.WriteOp: Start writing product Subset_S1B_IW_GRDH_1SDV_20180810T051813_20180810T051838_012196_016778_3134_Orb_NR_Cal_TC to /home/AOIs/aoi_1/stage_1/S1B_IW_GRDH_1SDV_20180810T051813_20180810T051838_012196_016778_3134_Sub_orb_TNR_cal_TC
The DEM 'SRTM 3 Sec' is not supported.
The DEM 'SRTM 3 Sec' is not supported.
SEVERE: org.esa.snap.core.util.SystemUtils$SnapImagingListener: JAI error occurred: 'Problem occurs when computing a tile by the owner.' at com.sun.media.jai.util.SunTileScheduler@479225c9
org.esa.snap.core.gpf.OperatorException: The DEM 'SRTM 3 Sec' is not supported.
        at org.esa.snap.engine_utilities.gpf.OperatorUtils.catchOperatorException(OperatorUtils.java:440)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:1067)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:122)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:86)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
        at javax.media.jai.OpImage.getTile(Unknown Source)
        at javax.media.jai.PlanarImage.getData(Unknown Source)
        at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:449)
        at org.esa.snap.core.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:435)
        at org.esa.snap.core.gpf.internal.OperatorImage.computeRect(OperatorImage.java:75)
        at javax.media.jai.SourcelessOpImage.computeTile(Unknown Source)
        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(Unknown Source)
        at javax.media.jai.OpImage.getTile(Unknown Source)
        at com.sun.media.jai.util.RequestJob.compute(Unknown Source)
        at com.sun.media.jai.util.WorkerThread.run(Unknown Source)
Caused by: org.esa.snap.core.gpf.OperatorException: The DEM 'SRTM 3 Sec' is not supported.
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:846)
        ... 14 more
Caused by: java.io.IOException: The DEM 'SRTM 3 Sec' is not supported.
        at org.esa.snap.dem.dataio.DEMFactory.getDemDescriptor(DEMFactory.java:375)
        at org.esa.snap.dem.dataio.DEMFactory.createElevationModel(DEMFactory.java:77)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.getElevationModel(RangeDopplerGeocodingOp.java:490)
        at org.esa.s1tbx.sar.gpf.geometric.RangeDopplerGeocodingOp.computeTileStack(RangeDopplerGeocodingOp.java:843)
        ... 14 more

Notes

1. I have implemented this from 3 different public IPs, so it is not a "blacklisted IP issue"
2. It's not an issue of too many requests, because in a matter of seconds (i.e., deleting and rebuilding my image/container) the behavior can be different (e.g., sometimes it works after running my algorithm only once and some others it does not)
3. I am modifying the value of this key DEM.srtm3GeoTiffDEM_HTTP=http://download.esa.int/step/auxdata/dem/SRTM90/tiff/ both manually and automatically (using sed) prior to first run of my algorithm, without any luck.
4. Updating SNAP does not help