Quantized-mesh output support

[ahuarte47]

Provides some new features:

• New quantized-mesh format ouput (-f Mesh), optionally you could add VertexNormals with -N option.
• New metadata file output.
• New cesium-friendly option to automatically create missing root tiles (level 0).
• Consider nodata values when creating tiles.
• Catch GDAL 'Integer overflow' error when creating tiles of low levels.
• Refactoring of code to easily write new output formats.

Details: https://www.linkedin.com/pulse/fast-cesium-terrain-rendering-new-quantized-mesh-output-alvaro-huarte/

Update 2018/08/08: Available Cesium Terrain Builder Docker thanks to @BWibo

[bjpirt]

I've just built this branch and generated quantized mesh tiles for some elevation data I've been working with and can confirm all seems to work well. Would be good to get it merged in

Thanks to @ahuarte47 for a quick fix and @markerikson for help in getting it built

[ahuarte47]

Thanks @bjpirt , my apologies to everybody, I wasn't able to reopen the original PR #55 to keep saved all comments already written

[LHolst]

i successfully build a terrain as Meshformat with Normals plus Cesiumfriendly option. Its working great @ahuarte47 , to use the normal information in cesium i had to set the "extensions": ["octvertexnormals"] in the layer.json

maybe that could be done by ctb-tile when the layer.json is generated by setting the -l option

[ahuarte47]

Thanks @LHolst for the feedback! I will add the "extensions" entry.

[LHolst]

i have noticed some inconsistency between tiles normals at the edges.

it could be an issue in cesium...

[LHolst]

The extensions entry works. Thanks @ahuarte47

When using the cesiumFriendly option, the missing toplevel 0.terrain file is copied, but not reflected in the layer.json by the first entry in available. Therefore cesium only loads half of the globe, because the startX and endX of z=0 misses the copied file.

Lastly, it would be nice when using 'Normal' and 'CesiumFriendly' together, the normals of the copied missing 0.terrain were flipped.

see wrong normals on the left, i modified the layer.json by setting startX=0

[ahuarte47]

Thanks @LHolst for your feedback and advices! About flipping normals in that copied missing root tile, it needs some more of work

[LHolst]

i really appreciate your help, root tiles working now :+1:

[ahuarte47]

Hi @LHolst, finally I have changed how CTB creates the missing root tile, now it is created from a temporary empty elevation that spatially overlaps the missing tile. I hope it works

[LHolst]

I am getting an bounds error Exception when using -l layer only option

ctb-tile -C -N -o . -f Mesh -l  /data/dtm_converted.tif                                                                                                                                                                                                     
0...10...20...30...40...50...60...70...80...90...100 - done.
0...10...20...30...40...50...60...70...80...90...100 - done.
terminate called after throwing an instance of 'ctb::CTBException'
  what():  The value is greater than the maximum X value
Aborted (core dumped)

gdalinfo /data/dtm_converted.tif
Driver: GTiff/GeoTIFF
Files: /data/dtm_converted.tif
       /data/dtm_converted.tif.aux.xml
Size is 8750, 6000
Coordinate System is:
PROJCS["CH1903+ / LV95",
    GEOGCS["CH1903+",
        DATUM["CH1903+",
            SPHEROID["Bessel 1841",6377397.155,299.1528128,
                AUTHORITY["EPSG","7004"]],
            TOWGS84[674.374,15.056,405.346,0,0,0,0],
            AUTHORITY["EPSG","6150"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4150"]],
    PROJECTION["Hotine_Oblique_Mercator_Azimuth_Center"],
    PARAMETER["latitude_of_center",46.95240555555556],
    PARAMETER["longitude_of_center",7.439583333333333],
    PARAMETER["azimuth",90],
    PARAMETER["rectified_grid_angle",90],
    PARAMETER["scale_factor",1],
    PARAMETER["false_easting",2600000],
    PARAMETER["false_northing",1200000],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","2056"]]
Origin = (2685625.000000000000000,1287000.000000000000000)
Pixel Size = (0.500000000000000,-0.500000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( 2685625.000, 1287000.000) (  8d34'51.96"E, 47d43'45.82"N)
Lower Left  ( 2685625.000, 1284000.000) (  8d34'49.86"E, 47d42' 8.69"N)
Upper Right ( 2690000.000, 1287000.000) (  8d38'21.90"E, 47d43'43.70"N)
Lower Right ( 2690000.000, 1284000.000) (  8d38'19.69"E, 47d42' 6.58"N)
Center      ( 2687812.500, 1285500.000) (  8d36'35.85"E, 47d42'56.21"N)
Band 1 Block=8750x1 Type=Float32, ColorInterp=Gray
  Min=399.590 Max=679.660 
  Minimum=399.590, Maximum=679.660, Mean=547.999, StdDev=62.302
  NoData Value=-9999
  Metadata:
    STATISTICS_MAXIMUM=679,65997314453
    STATISTICS_MEAN=547,99864003316
    STATISTICS_MINIMUM=399,58999633789
    STATISTICS_STDDEV=62,30161802239

without the layers option tiles get generated is there a way to generate the layer.json along with the tiles?

[LHolst]

when i use my old layer.json, the normals are improved.

[ahuarte47]

Hi @LHolst, I fixed a bug, I hope it works now

[LHolst]

https://github.com/ahuarte47/cesium-terrain-builder/commit/4bb78ba2e2c13cb0b4631b355cafb1aa94676f65 generates layer.json correctly, thanks @ahuarte47 :+1: :tada:

also https://github.com/geo-data/cesium-terrain-builder/issues/15#issuecomment-390916918 did the trick to run gdal 2.2

[geoportallux]

for anyone who wants to test this PR using docker:

FROM debian
RUN apt-get update && apt-get -y install cmake build-essential gdal-bin git libgdal-dev
RUN cd /root && git clone https://github.com/ahuarte47/cesium-terrain-builder.git && cd cesium-terrain-builder && git checkout master-quantized-mesh && mkdir build && cd build && cmake .. && make install . && ldconfig

works great for us. many thanks

[winsento]

Generate terrain from dem.tif EPSG:4326

ctb-tile  -f Mesh -s 24 -e 0 -C -c 4 -l --output-dir /terrain /dem.tif

But when set CesiumTerrainProvider in Cesium - result is empty. Example here: https://jsfiddle.net/winsent/7kp7y9k2/6/

[LHolst]

@winsento you need to run the command without -l to generate the tiles, and with to generate the layer.json

[JoJoeGoing]

Failed to obtain terrain tile...... I do not know how to fix it.

[ahuarte47]

It would be useful more information to try help you. What command parameters did you used? or what raster file did you used?

[JoJoeGoing]

@ahuarte47 The raster file I used is .tif。And the command is : ctb-tile -o ./terrain -f Mesh -C -N -c 4 ./out.tif .（I have get the layer.json by this command ctb-tile -o ./terrain -f Mesh -C -N -c 4 -l ./out.tif). It works well and I get the terrain files . the gdal version is 2.1.1

[ahuarte47]

Ok I see, the error comes from Cesium no from CTB. Do you have defined the Content-Encoding: gzip content negotiation header?

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding

[JoJoeGoing]

@ahuarte47 I really appreciate your help. It works well now。

[mrgloom]

Seems like Integer overflow problem is not gone https://github.com/geo-data/cesium-terrain-builder/issues/28

[xtassin]

Hi! Thank you for the great project.

I successfully processed SRTM 250m data from http://gisweb.ciat.cgiar.org/TRMM/SRTM_Resampled_250m/

However, the file for the western part of the world (http://gisweb.ciat.cgiar.org/TRMM/SRTM_Resampled_250m/SRTM_W_250m_TIF.rar) triggers this error: Error: The minimum X value is greater than the maximum X value

I believe the range of coordinate (< -180.0) is causing the issue. Any suggestion on how to solve this?

[ahuarte47]

I would use gdal_translate with -projwin parameter to clip the raster to (-180 / -29.98)

[xtassin]

Thanks for the suggestion! I did it with gdalwarp eventually.

[BWibo]

Hey there, today, I released a Docker image based on this pull request including a brief guide on how to create quantized-mesh terrain datasets. I am happy for any kind of feedback.

github.com/tum-gis/cesium-terrain-builder-docker

[xtassin]

I noticed that generated tiles have some elevations errors at the boundaries (see screenshot). Would you have an idea why?

[ahuarte47]

Do those walls match the limits of your input raster? If you do not have data outside that borders you have to set the nodata flag of these raster cells in order to skip them when creating the tiles. You could also add some height to that cells to avoid a so big break with the real terrain.

[xtassin]

I assumed the walls match the raster tiles but I would have to confirm this. There is definitely data outside of the border as I am using a global SRTM source. These artifacts are visible on most tile boundaries.

On Thu, Sep 6, 2018 at 6:55 PM, Alvaro Huarte [email protected] wrote:

Do those walls match the limits of your input raster? If you do not have data outside that borders you have to set the nodata flag of these raster cells in order to skip them when creating the tiles. You could also add some height to that cells to avoid a so big break with the real terrain.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/geo-data/cesium-terrain-builder/pull/64#issuecomment-419166325, or mute the thread https://github.com/notifications/unsubscribe-auth/AASGfGFsF2QJQS5TYwfZm8df0tA9IifTks5uYVN5gaJpZM4T7BGN .

[84ace]

Does anyone have the issue with small area terrain tile generation causing the tiles to be somehow misplaced and overscaled? I have commented on issue #72 with the commit (https://github.com/geo-data/cesium-terrain-builder/commit/29e0ba8dfd626eab485a9addf150fc4ce3135cf3) that first caused the issue and a link to a tiff that can reproduce the problem in that commit or any commit after that. If anyone can shed light on it i'd be grateful. Thanks!