treee111
Contour lines
Hi, actually it is not possible. I heard this idea once but never digged into it...
Well it is - kind of - possible. But needs extra steps.
- Use Program srtm2osm (https://wiki.openstreetmap.org/wiki/Srtm2Osm), to download the the elevation data for a specified area and convert it into an OSM-format contour file
- I can't remember if there was a step to merge the data to one OSM File?
- This is the OSM data that needs to be fed to the existing application.
Disclaimer:
- I didn't check the code of wahooMapsCreator
- Didn't test the thesis.
Hello,
PHYGHTMAP seems to be a better choice no ?
A good exemple from https://oruxmaps.forumotion.com/t2216-contourlines-added-for-oruxmapsforge
#test of merging osm + srtm and render in mapsforge using a small aera in LowerAustria along the river Danube
#Merging Srtm + osm: Thanks to user Fireball2 at http://forum.openstreetmap.org/viewtopic.php?id=5359
#Contours with PHYGHTMAP (output is in version 0.6 so no further conversion is needed)
C:\Python27\Scripts\phyghtmap -a 14.0:48.0:16.5:48.5 -o D:\maps\Mapsforge\at_test\srtm6 --source=view1,view3,srtm3 -s 25 -c 500,100 --start-node-id=2000000000 --start-way-id=1000000000 --write-timestamp --max-nodes-per-tile=0 --max-nodes-per-way=0
#cut out osm-data from austria.osm
osmosis --rx D:\maps\OSM_Data\austria.osm.bz2 --tee 1 --bounding-box top="48.50" bottom="48.00" left="14.00" right="16.50" --wx D:\maps\Mapsforge\at_test\at_test.osm
#merge srtm + osm
osmosis --rx D:\maps\Mapsforge\at_test\at_test.osm --sort-0.6 --rx D:\maps\Mapsforge\at_test\srtm6.osm --sort-0.6 --merge --wx D:\maps\Mapsforge\at_test\at_test_srtm.osm
#mapsforge output (use mapwriter plugin WITH dependencies)
osmosis --rx file=D:\maps\Mapsforge\at_test\at_test_srtm.osm --mw file=D:\maps\Mapsforge\at_test\at_test_srtm.map bbox=48.0,14.0,48.5,16.5 type=hd tag-conf-file=D:\maps\Mapsforge\at_test\tag-mapping-srtm.xml
#Custom tag-mapping-srtm.xml:
#added lines to tag-mapping.xml:
<!-- Elevation TAGS -->
<ways>
<osm-tag key="contour_ext" value="elevation_minor" zoom-appear="13" />
<osm-tag key="contour_ext" value="elevation_medium" zoom-appear="12" />
<osm-tag key="contour_ext" value="elevation_major" zoom-appear="12" />
</ways>
#Custom [render].xml
#added lines:
<!-- contour lines -->
<rule e="way" k="contour_ext" v="*">
<rule e="way" k="contour_ext" v="elevation_major">
<line stroke="#0a630c" stroke-width="1.1" stroke-dasharray="7,5"/>
</rule>
<rule e="way" k="contour_ext" v="elevation_medium">
<line stroke="#0a630c" stroke-width="0.8" stroke-dasharray="7,5"/>
</rule>
<rule e="way" k="contour_ext" v="elevation_minor">
<line stroke="#0a630c" stroke-width="0.4" stroke-dasharray="7,5"/>
</rule>
</rule>
I will try to test.
Have a good day.
@geekitus: that would be great, if it works. Because being also coded in python would make a better integration than with the srtm2osm executable. Please keep us update about the results of your test. BR Gü
Hi @geekitus, did you have any success / intermediate result? I would work on top of them or tryout by myself with the information in this issue. greetings!
Did some testing today, but I got stuck with PHYGHTMAP on windows with the latest matplotlib not having the _contour function anymore for a long time now it seems. So I got a phyton import error on _contour.
I did a test with srtm2osm and got a osm file out of it to test further with. Maybe later this week.
Did some testing today, but I got stuck with PHYGHTMAP on windows with the latest matplotlib not having the _contour function anymore for a long time now it seems. So I got a phyton import error on _contour.
I did a test with srtm2osm and got a osm file out of it to test further with. Maybe later this week.
Just install matplotlib 3.4.3 via pip
@vanyasvl I'm am at work right now but I ran into an error yesterday evening (late). Don't have the error message at hand but I'm hoping you know what I'm running into... I used a variation of this line: phyghtmap -a 14.0:48.0:16.5:48.5 -o D:\maps\Mapsforge\at_test\srtm6 --source=view1,view3,srtm3 -s 25 -c 500,100 --start-node-id=2000000000 --start-way-id=1000000000 --write-timestamp --max-nodes-per-tile=0 --max-nodes-per-way=0. First I got an error that I needed a license for the srtm3 data. Ok, registered and tried again. Now I get an error on a textfile in the cache directory that does not exist yet. From memory something about an idex file. There are other files in there after the first run btw. Hope this means something to you, if not I post the error message tonight and try further. This elevation stuff is pretty new to me.
@vanyasvl I'm am at work right now but I ran into an error yesterday evening (late). Don't have the error message at hand but I'm hoping you know what I'm running into... I used a variation of this line: phyghtmap -a 14.0:48.0:16.5:48.5 -o D:\maps\Mapsforge\at_test\srtm6 --source=view1,view3,srtm3 -s 25 -c 500,100 --start-node-id=2000000000 --start-way-id=1000000000 --write-timestamp --max-nodes-per-tile=0 --max-nodes-per-way=0. First I got an error that I needed a license for the srtm3 data. Ok, registered and tried again. Now I get an error on a textfile in the cache directory that does not exist yet. From memory something about an idex file. There are other files in there after the first run btw. Hope this means something to you, if not I post the error message tonight and try further. This elevation stuff is pretty new to me.
Check phyghtmap command-line arguments in my build script for Garmin here https://github.com/vanyasvl/cyprus-map-sid/blob/master/build.sh
Thanks, but here you seem to manually download the elevation tiles. I thought phyghtmap was supposed to do that for you? I'll, try to, look further into it. If it works it seems to be pretty easy to incorporate into the Wahoo map maker
@vanyasvl phyghtmap -a 6:44:9:47 -o high_alps --source=view1 (from the man page example) gives FileNotFoundError: [WinError 2] The system cannot find the file specified: 'hgt\viewfinderHgtIndex_1.txt' view3 results in FileNotFoundError: [WinError 2] The system cannot find the file specified: 'hgt\viewfinderHgtIndex_3.txt' and srtm3 in FileNotFoundError: [WinError 2] The system cannot find the file specified: 'hgt\hgtIndex_3_v3.0.txt'
Maybe I need to do something with --rewrite-indices, but how?
Something like below gives:
PS D:\Windows-Wahoo-Map-Creator-Osmosis> phyghtmap -a 6:44:9:47 -o high_alps --source=srtm3 --rewrite-indices
generating index in hgt\hgtIndex_1_v2.1.txt ... Traceback (most recent call last):
File "C:\Users\ebodd\AppData\Local\Programs\Python\Python39\Scripts\phyghtmap-script.py", line 33, in
I'm sure I'm doing something stupid, but what?
phyghtmap has an issue with hsgt data downloading. Download files manually like I do in my script
@vanyasvl Just got it to work (at least it downloads and does something) Turned out all I needed to do was a pip install lxml...
Still running into some troubles, but also some success! While rendering Scotland osmosis errors out halfway through at tile 30 of 72 and there are some weird diagonal(ish) lines which I don't understand yet. Just proof of concept, if working would also need somebody with a newer unit than a bolt 1 to look how it is displayed on the device(s) The screenshot is of cruiser on my pc with a modified roam 1 mapsforge theme.
And finally success. Had one of those rare bright moment ;-). I had taken over the start node and way number of the posted example. But, that was in 2012 so changed --start-node-id=2000000000 to --start-node-id=3000000000 and did the same for ways and voila. The complete map set for bonnie Scotland rendered just fine and no artifacts (here) anymore.
@Ebe66 : I don't go into every details of the tread above. And even if I did: In some month I will not remember the solution any more. As many might experience the same: Any chance that you create a summary of the additons/changes needed to get contour lines added? Or even add it to the repo readme? Thanks in advance.
@limex I don't know if Treee111 wants to incorporate this, but sure I can make a list of the things I ran into (on windows). The method used was the phyghtmap way as described above near the top (or more precise the later version of it posted in the mentioned thread).
- The first issue I ran into was that matplotlib does not have the _contour function/functionality anymore. Solution: install an older version with pip install matplotlib==3.4.3
- Then it turns out you now need to register to use the srtm3 data.
Solution: make a free account here https://ers.cr.usgs.gov/login?redirectUrl=https://ers.cr.usgs.gov/
and add the command line arguments for it --earthexplorer-user=
and --earthexplorer-password= - Now phyghtmap ended with a runtime error. Took me some time but... Solution: All I needed to do was pip install lxml which was not mentioned on the programs manual page.
Basically it was running now and I could generate my tile maps. But...
- Strange diagonal(ish) lines in the output. See my first screenshot above. Solution: the --start-node-id and --start-way-id used in the example script near the top where probably fine in 2012 when it was posted but are now somewhere in the used range of id's. So id's were conflicting with already existing map data. Upping them to a higher value resulted in the tile from the screenshot rendering good.
- But the next tile again had some lines, less but lines. Solution: Find out what the actual highest node and way id's used are and use start id's above that number. I found them on this page: https://osmstats.neis-one.org/?item=elements
Mind, I'm most certainly not a expert in this field but my current command line looks something like this: phyghtmap -a left:bottom:right:topcoordinates -o outFile -s 10 -c 100,50 --source=view1,view3,srtm3 --jobs=8 --viewfinder-mask=1 --start-node-id=20000000000 --max-nodes-per-tile=0 --start-way-id=2000000000 --write-timestamp --no-zero-contour --earthexplorer-user= --earthexplorer-password=
Hope it helps out someone.
Edit: Have had feedback that the matplotlib _countour problem is also on macOS but fortunately the fix is also the same
@Ebe66 helped me to get it working and integrated into @treee111 's code. Thanks again! I tested on macOS BigSur and it works fine so far. I use python3.10 via brew. Additionally to before I had to do:
pip3 install bs4 pip3 install lxml
Download and unpack matplotlib-3.4.3.tar.gz from https://pypi.org/project/matplotlib/3.4.3/#files sudo python3 setup.py install
Download and unpack http://katze.tfiu.de/projects/phyghtmap/phyghtmap_2.23.orig.tar.gz sudo python3 setup.py install
And here you find a working version (some things can be made better). In osm_maps_functions.py, line 551, insert your account data from https://ers.cr.usgs.gov. Here it would be cool to use arguments/parameters to not have the account data in source code.
https://github.com/masc4ii/wahooMapsCreator
More testing and optimization on theme with elevation lines code done. Here an example (Bolt v2, Gran Canaria). Find code in my fork - as always.
Hi you both - @Ebe66 and @masc4ii, thanks a lot for your effort in getting it running! Very nice and as more or less always some geniuses getting it to run and I integrate it into the repo, make it equal to the existing coding, have validations etc.
That's done from my side for contour lines. @masc4ii it would be nice if you could test it on your end if my solution is also working for you! I also switched back to python 3.10 to get matplotlib 3.4.3 running. I write a solid PR now, if you are faster you can see all changes (including setting everything up, might not be needed for you anyway) in the PR #188
Please make sure that shapely is installed via conda-forge as written in this link!
Tests can go on with branch contour-lines or alpha version installed via ~~pip install wahoomc==4.0.0a3~~ ~~pip install wahoomc==4.0.0a5~~ pip install wahoomc==4.0.0a8
If you have any comments, don't hesitate to use the PR review functionality!
@treee111 : Thank you for implementing! Now I found the time to test. I tried to create canary islands.
python3 -m wahoomc cli -co canary_islands -con
The processing stops with this error:
INFO:# Generate land for each coordinate
INFO:+ (tile 1 of 17) Coordinates: 114,106
ERROR:subprocess:'File "/Users/masc/Desktop/wahooMapsCreator-contour-lines/wahoomc/resources/shape2osm.py", line 197'
ERROR:subprocess:'print("<osm version=\'0.6\'>", file=open_file)'
ERROR:subprocess:'^'
ERROR:subprocess:'SyntaxError: invalid syntax'
ERROR:! Error creating land.osm for tile: 114,106
When commenting this line, I get a very similar error in line 215. Something I missed to install? I use the macOS brew python 3.10. Installed shapely is 1.8.5. In my fork this line looks identical and no such error pops up.
Edit: found it. "python" in code always has to be "python3" on my macOS. Changed -> works. Cool! Where do you save the user data needed for contour lines?
Hi @masc4ii! Sorry for the late reply..
thanks a lot for testing! Nice, that you found that out by yourself!
I assume you have your installation in your main/standard Python environment and not via Anaconda. That might be the reason for you to have python3. In earlier times I had this in the coding as well but it turned out to be not needed.
I'd recommend using Anaconda and the setup I described in the docs, its very easy to setup and you have it isolated from the rest of your system.
Anyway, I am glad that it worked also in your setup 👍
I'll save the userdata in ~/wahooMapsCreatorData/.config.json file where other tool-related settings are stored.
Thanks for your answer. Found the file now!
I found some areas where the contour lines failed. One example is near Plauen (136/86). In the middle of the tile the lines are missing. In the near of this, towards Czech Republic that happens often.
I have the same in my maps I saw. At first I thought it might be a fault with phyghtmap because of the square cutout but at the Czech border it follows the border. When I load the generated height .pbf file in maperitive for this tile you can clearly see the voids.
I think I will manually try some different source settings.
Well, trying srtm opens the python module hell for me. It needs ogr and osr. After changing the source to use "from osgeo import ogr" and the same for osr I get loads of other errors. Switching to view3 does give a complete picture. But the viewfinder site confuses me and sometime contradicts itself it seems. But if this part is still valid it might even be better to use the view3 data: "NASA has only released 1" SRTM data for US territory. I have created HGT files with a resolution of 1" for some parts of Europe from topographic maps. For most of the Alps and the most rugged parts of the Pyrenees, these were created from the most detailed sources, and should be accurate. Elsewhere, I have 1" DEM data for Great Britain, Ireland, France (excluding Corsica), Germany, Benelux, Denmark, Norway, Sweden and north east Spain. But these were created from Russian maps of scale 1:100,000, which are less accurate than SRTM. Consequently, for these areas, where 3" DEM data is available from this page or from SRTM, they should be generally better than the 1" data, despite the lower resolution. "
nice finding! So can there be applied a general coding/config change to get better results over all? Or is it specific to certains regions/countries?
Hard to say for me. That is what I mean by the contractionary information. The obvious choice would be to use srtm1 data (if someone can make it work) but the info on the viewfinder site on the one hand says that only the US is available but on the other hand at a different place the whole planet excluding north/south of 60". As long as srtm is not working the best option appears to be using view3. You, having a gui, could consider making it a checkbox option? On the viewfinder homepage it says "Last update 15 May 2022 with addition of more and improved 1" DEM for Europe." which is fairly recent considering other dates on there but also explicitly implies it is not complete.
Got srtm1 to work. turns out that I also had to update my gdal installation to 3.6.3
And, the Czech part now present
No real suggestion on what elevation source should be used. I'm no expert on this too, but reading the viewfinder site it seems all have their pros and cons. Completeness can only be checked visually and accuracy...??? Best option is probably put something in the readme and make it a checkbox on what to use