tilemaker
tilemaker copied to clipboard
systemed
Out of memory error
I am not sure if this is a issue with tilemaker or hardware/software.
I am trying to create level 18 tiles of south america(3gb) with highway data on a virtual machine (virtualbox). The virtual machine spec: cpu 4cores i7 4790k memory 10gb ddr3 hard drive ssd(40gb)&hdd(250gb)
I have tried a few different settings like less memory, fixedspace drive, different drives, ssd, always in combination with --store ... the result is always the same: "Out of memory: Killed process"
The process on ssd does not take a lot, node creation and sorting, same as ways is finished in 15 minutes, but instantly after i get the error. --store creates around 10gb+ of space on ssd from 3gb osm.pbf.
For very small regions its working fine.
Log:
Jul 1 11:12:03 userx kernel: [ 7231.709528] tilemaker invoked oom-killer: gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0 Jul 1 11:12:03 userx kernel: [ 7231.709534] CPU: 2 PID: 2287 Comm: tilemaker Not tainted 5.15.0-40-generic #43-Ubuntu Jul 1 11:12:03 userx kernel: [ 7231.709536] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 Jul 1 11:12:03 userx kernel: [ 7231.709537] Call Trace: Jul 1 11:12:03 userx kernel: [ 7231.709538]Jul 1 11:12:03 userx kernel: [ 7231.709540] show_stack+0x52/0x58 Jul 1 11:12:03 userx kernel: [ 7231.709543] dump_stack_lvl+0x4a/0x5f Jul 1 11:12:03 userx kernel: [ 7231.709546] dump_stack+0x10/0x12 Jul 1 11:12:03 userx kernel: [ 7231.709548] dump_header+0x53/0x224 Jul 1 11:12:03 userx kernel: [ 7231.709550] oom_kill_process.cold+0xb/0x10 Jul 1 11:12:03 userx kernel: [ 7231.709551] out_of_memory+0x106/0x2e0 Jul 1 11:12:03 userx kernel: [ 7231.709555] __alloc_pages_slowpath.constprop.0+0x97a/0xa40 Jul 1 11:12:03 userx kernel: [ 7231.709558] __alloc_pages+0x30d/0x320 Jul 1 11:12:03 userx kernel: [ 7231.709559] alloc_pages_vma+0x9d/0x380 Jul 1 11:12:03 userx kernel: [ 7231.709562] do_anonymous_page+0xee/0x3b0 Jul 1 11:12:03 userx kernel: [ 7231.709565] handle_pte_fault+0x1fe/0x230 Jul 1 11:12:03 userx kernel: [ 7231.709566] __handle_mm_fault+0x3c7/0x700 Jul 1 11:12:03 userx kernel: [ 7231.709568] handle_mm_fault+0xd8/0x2c0 Jul 1 11:12:03 userx kernel: [ 7231.709570] do_user_addr_fault+0x1c5/0x670 Jul 1 11:12:03 userx kernel: [ 7231.709573] exc_page_fault+0x77/0x160 Jul 1 11:12:03 userx kernel: [ 7231.709575] ? asm_exc_page_fault+0x8/0x30 Jul 1 11:12:03 userx kernel: [ 7231.709578] asm_exc_page_fault+0x1e/0x30 Jul 1 11:12:03 userx kernel: [ 7231.709580] RIP: 0033:0x7f6e316e0180 Jul 1 11:12:03 userx kernel: [ 7231.709582] Code: 81 fa 80 00 00 00 76 d2 c5 fe 7f 40 40 c5 fe 7f 40 60 48 83 c7 80 48 81 fa 00 01 00 00 76 2b 48 8d 90 80 00 00 00 48 83 e2 c0 Jul 1 11:12:03 userx kernel: [ 7231.709590] Mem-Info: Jul 1 11:12:03 userx kernel: [ 7231.709591] active_anon:562 inactive_anon:1450239 isolated_anon:0 Jul 1 11:12:03 userx kernel: [ 7231.709591] active_file:0 inactive_file:0 isolated_file:0 Jul 1 11:12:03 userx kernel: [ 7231.709591] unevictable:6976 dirty:0 writeback:0 Jul 1 11:12:03 userx kernel: [ 7231.709591] slab_reclaimable:14090 slab_unreclaimable:12941 Jul 1 11:12:03 userx kernel: [ 7231.709591] mapped:2286 shmem:290 pagetables:3489 bounce:0 Jul 1 11:12:03 userx kernel: [ 7231.709591] kernel_misc_reclaimable:0 Jul 1 11:12:03 userx kernel: [ 7231.709591] free:22689 free_pcp:0 free_cma:0 Jul 1 11:12:03 userx kernel: [ 7231.709594] Node 0 active_anon:2248kB inactive_anon:5800956kB active_file:0kB inactive_file:0kB unevictable:27904kB isolated(anon):0kB isolated(file):0kB mapped:9144kB dirty:0kB writeback:0kB shmem:1160kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:3872kB pagetables:13956kB all_unreclaimable? yes Jul 1 11:12:03 userx kernel: [ 7231.709596] Node 0 DMA free:14336kB min:172kB low:212kB high:252kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB Jul 1 11:12:03 userx kernel: [ 7231.709599] lowmem_reserve[]: 0 3360 5826 5826 5826 Jul 1 11:12:03 userx kernel: [ 7231.709602] Node 0 DMA32 free:48108kB min:38876kB low:48592kB high:58308kB reserved_highatomic:0KB active_anon:0kB inactive_anon:3498060kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:3653568kB managed:3553464kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB Jul 1 11:12:03 userx kernel: [ 7231.709604] lowmem_reserve[]: 0 0 2466 2466 2466 Jul 1 11:12:03 userx kernel: [ 7231.709606] Node 0 Normal free:28312kB min:28528kB low:35660kB high:42792kB reserved_highatomic:0KB active_anon:2248kB inactive_anon:2302336kB active_file:0kB inactive_file:0kB unevictable:27904kB writepending:0kB present:2640896kB managed:2525328kB mlocked:27904kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB Jul 1 11:12:03 userx kernel: [ 7231.709609] lowmem_reserve[]: 0 0 0 0 0 Jul 1 11:12:03 userx kernel: [ 7231.709611] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB (M) 3*4096kB (UM) = 14336kB Jul 1 11:12:03 userx kernel: [ 7231.709618] Node 0 DMA32: 9*4kB (UM) 8*8kB (UM) 21*16kB (UME) 15*32kB (UME) 12*64kB (UME) 7*128kB (UE) 1*256kB (E) 1*512kB (M) 2*1024kB (ME) 1*2048kB (U) 10*4096kB (M) = 48404kB Jul 1 11:12:03 userx kernel: [ 7231.709627] Node 0 Normal: 658*4kB (UME) 559*8kB (UME) 287*16kB (UME) 157*32kB (UE) 55*64kB (UE) 23*128kB (UME) 9*256kB (UE) 5*512kB (ME) 1*1024kB (M) 0*2048kB 0*4096kB = 29072kB Jul 1 11:12:03 userx kernel: [ 7231.709636] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Jul 1 11:12:03 userx kernel: [ 7231.709637] 2705 total pagecache pages Jul 1 11:12:03 userx kernel: [ 7231.709638] 0 pages in swap cache Jul 1 11:12:03 userx kernel: [ 7231.709638] Swap cache stats: add 0, delete 0, find 0/0 Jul 1 11:12:03 userx kernel: [ 7231.709639] Free swap = 0kB Jul 1 11:12:03 userx kernel: [ 7231.709639] Total swap = 0kB Jul 1 11:12:03 userx kernel: [ 7231.709640] 1577614 pages RAM Jul 1 11:12:03 userx kernel: [ 7231.709640] 0 pages HighMem/MovableOnly Jul 1 11:12:03 userx kernel: [ 7231.709641] 54076 pages reserved Jul 1 11:12:03 userx kernel: [ 7231.709641] 0 pages hwpoisoned Jul 1 11:12:03 userx kernel: [ 7231.709642] Tasks state (memory values in pages): Jul 1 11:12:03 userx kernel: [ 7231.709642] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name Jul 1 11:12:03 userx kernel: [ 7231.709644] [ 446] 0 446 11942 761 98304 0 -250 systemd-journal Jul 1 11:12:03 userx kernel: [ 7231.709646] [ 483] 0 483 72365 6876 110592 0 -1000 multipathd Jul 1 11:12:03 userx kernel: [ 7231.709648] [ 485] 0 485 6695 1212 69632 0 -1000 systemd-udevd Jul 1 11:12:03 userx kernel: [ 7231.709650] [ 692] 104 692 22325 572 77824 0 0 systemd-timesyn Jul 1 11:12:03 userx kernel: [ 7231.709651] [ 740] 101 740 4017 645 73728 0 0 systemd-network Jul 1 11:12:03 userx kernel: [ 7231.709652] [ 742] 102 742 6337 1319 94208 0 0 systemd-resolve Jul 1 11:12:03 userx kernel: [ 7231.709654] [ 753] 0 753 1723 578 45056 0 0 cron Jul 1 11:12:03 userx kernel: [ 7231.709655] [ 754] 103 754 2192 984 53248 0 -900 dbus-daemon Jul 1 11:12:03 userx kernel: [ 7231.709656] [ 759] 0 759 20674 748 61440 0 0 irqbalance Jul 1 11:12:03 userx kernel: [ 7231.709658] [ 760] 0 760 8194 2887 106496 0 0 networkd-dispat Jul 1 11:12:03 userx kernel: [ 7231.709659] [ 761] 0 761 58621 793 90112 0 0 polkitd Jul 1 11:12:03 userx kernel: [ 7231.709660] [ 762] 107 762 55600 1070 86016 0 0 rsyslogd Jul 1 11:12:03 userx kernel: [ 7231.709661] [ 763] 0 763 465953 6440 495616 0 0 dockerd Jul 1 11:12:03 userx kernel: [ 7231.709662] [ 765] 0 765 366338 3747 311296 0 -900 snapd Jul 1 11:12:03 userx kernel: [ 7231.709664] [ 767] 0 767 5870 655 73728 0 0 systemd-logind Jul 1 11:12:03 userx kernel: [ 7231.709665] [ 769] 0 769 98139 1121 126976 0 0 udisksd Jul 1 11:12:03 userx kernel: [ 7231.709666] [ 776] 0 776 1957 665 53248 0 0 login Jul 1 11:12:03 userx kernel: [ 7231.709667] [ 805] 0 805 3854 1086 69632 0 -1000 sshd Jul 1 11:12:03 userx kernel: [ 7231.709668] [ 817] 0 817 27433 2837 110592 0 0 unattended-upgr Jul 1 11:12:03 userx kernel: [ 7231.709670] [ 823] 0 823 60819 1274 114688 0 0 ModemManager Jul 1 11:12:03 userx kernel: [ 7231.709671] [ 1080] 0 1080 430781 3469 335872 0 0 containerd Jul 1 11:12:03 userx kernel: [ 7231.709672] [ 1115] 0 1115 693 242 45056 0 0 bpfilter_umh Jul 1 11:12:03 userx kernel: [ 7231.709673] [ 1321] 1000 1321 4253 1049 69632 0 0 systemd Jul 1 11:12:03 userx kernel: [ 7231.709675] [ 1322] 1000 1322 42324 949 98304 0 0 (sd-pam) Jul 1 11:12:03 userx kernel: [ 7231.709676] [ 1339] 1000 1339 2185 957 53248 0 0 bash Jul 1 11:12:03 userx kernel: [ 7231.709677] [ 2276] 1000 2276 2873 699 61440 0 0 sudo Jul 1 11:12:03 userx kernel: [ 7231.709679] [ 2277] 1000 2277 2873 221 57344 0 0 sudo Jul 1 11:12:03 userx kernel: [ 7231.709680] [ 2278] 0 2278 1520442 1423957 11591680 0 0 tilemaker Jul 1 11:12:03 userx kernel: [ 7231.709681] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-1.scope,task=tilemaker,pid=2278,uid=0 Jul 1 11:12:03 userx kernel: [ 7231.709711] Out of memory: Killed process 2278 (tilemaker) total-vm:6081768kB, anon-rss:5695396kB, file-rss:432kB, shmem-rss:0kB, UID:0 pgtables:11320kB oom_score_adj:0 Jul 1 11:12:03 userx kernel: [ 7231.819827] oom_reaper: reaped process 2278 (tilemaker), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kBfd 7f 02 c5 fd 7f 42 20 c5 fd 7f 42 40 c5 fd 7f 42 60 48 83 ea Jul 1 11:12:03 userx kernel: [ 7231.709583] RSP: 002b:00007f6e29f145b8 EFLAGS: 00010287 Jul 1 11:12:03 userx kernel: [ 7231.709585] RAX: 00007f6cbdf9e011 RBX: 00007f6cbdf9e010 RCX: 00007f6e3165dbd7 Jul 1 11:12:03 userx kernel: [ 7231.709586] RDX: 00007f6cbe435000 RSI: 0000000000000000 RDI: 00007f6cc1ca71b0 Jul 1 11:12:03 userx kernel: [ 7231.709587] RBP: 00007f6cc1ca7230 R08: 00007f6cbdf9e010 R09: 0000000003d09220 Jul 1 11:12:03 userx kernel: [ 7231.709587] R10: 0000000000000022 R11: 0000000000000246 R12: 00007f6e14006330 Jul 1 11:12:03 userx kernel: [ 7231.709588] R13: 00007f6e29f14690 R14: 00007f6e143afb70 R15: 0000558629691140 Jul 1 11:12:03 userx kernel: [ 7231.709590]
Is there a particular reason you want to create z18 tiles?
Generally with vector tiles, you'll create up to z14, but these z14 tiles will have all the information needed to render at z18 (and beyond). See https://github.com/systemed/tilemaker/blob/master/docs/VECTOR_TILES.md#the-format
Yes i need z18 tiles for my application. It propably would work to create mbtiles for each country, but that would be a lot of work foreach country on planet.
Is there no workaround for "out of memory" issue ? Is this issue caused because of limited memory of 10gb ?
I would genuinely be interested to know what your application is as I've never heard of anyone needing z18 vector tiles before!
tilemaker will have a lot of overhead creating z18 tiles for such a large area, and so the memory usage will be high. If you're just using a raw extract from (say) Geofabrik, outlying islands will probably cause a lot of sea tiles to be generated, so you could save some memory by supplying a bounding box that's closer to the main landmass only.
I dont display the map in my application but need the data for research, z18 tiles have a perfect size, z17 are already to big.
Might it would help to reduce data content, lower details ? I dont need sharp edges etc. For me it would be fine if a street run completly straight inside of a z18 tile instead of making small curves...
It might help a bit, but ultimately tilemaker keeps an index of the features that are needed in each individual tile, and the more tiles you have, the more memory that needs.
For research/analysis, the most usual approach is to put OSM data into a Postgres database (using osm2pgsql or similar) and then query that using the PostGIS analysis functions - tilemaker isn't really designed for this sort of use-case I'm afraid.
According info page osm2pgsql take a lot of memory aswell (~64gb memory for 64gb osm file) and according my fast research the output dont seem to be in a similar format like tilemaker is able to (z18,y,x,data; which is perfect for me, tilemaker is awesome). Osmium-renumber takes a lot of memory aswell else --compact mode would have been an option and i havent seen a osmium-renumber prepared planet.osm.pbf file or continents.
Well i will stick to tilemaker and use the --bbox option to generate parts of each continent step by step. I think thats the best solution for me.
Is there a way to lower details to reduce overhead a little bit ? What does -DFAT_WAYS does ? Or maybe can you link me a documentation of available build options ? Thanks a lot.
-DFAT_WAYS isn't needed any more. The only build option is -DFAT_TILE_INDEX which I think you're probably using already - that's the one that enables output at z17+. --bbox sounds like the best solution for now, I think!
I dont display the map in my application but need the data for research, z18 tiles have a perfect size, z17 are already to big.
- the default png tiles precision is 256x256 pixel
- https://tile.openstreetmap.org/18/140148/97424.png
- vector tiles default precision is 4096x4096
- it is easy to generate from z14(4096x4096) the z18(256x256) via overzooming
- or you can use 8192x8192 with the optional "high-resolution" flag ( https://github.com/systemed/tilemaker/pull/384/files )
- and you can use maxzoom15 or maxzoom16 or maxzoom17
- And not so hard to create 16384x16384 tiles
- based on this PR https://github.com/systemed/tilemaker/pull/384/files - just replace 8192 -> 16384
https://github.com/mapbox/vector-tile-spec/blob/master/2.1/README.md
- "For example, if a tile has an extent of 4096, coordinate units within the tile refer to 1/4096th of its square dimensions. A coordinate of 0 is on the top or left edge of the tile, and a coordinate of 4096 is on the bottom or right edge. Coordinates from 1 through 4095 inclusive are fully within the extent of the tile, and coordinates less than 0 or greater than 4096 are fully outside the extent of the tile. A point at (1,10) or (4095,10) is within the extent of the tile. A point at (0,10) or (4096,10) is on the edge of the extent. A point at (-1,10) or (4097,10) is outside the extent of the tile."
@ImreSamu thanks for your post. I need z18 tiles informations locally inside my mbtiles file with the correct tile_x(z18),tile_y(z18) and data(z18) value. It would require a tool to rewrite the whole database and split up each z14 tile into many small tiles(~256 ?) and fetch the matching data to it. I have not found anything like this and could imagine my setup isnt good enough for conversion of the whole planet.
I have build tilemaker with -DFAT_TILE_INDEX which allows to create z18 tiles on the fly.
I was able to build whole south america thanks to bbox option and 5 runs.
@Deam94
( for research purposes ) in addition to postgis/osm2pgsql .. most people using osm-qa-tiles - Zoom12 Vector tiles and it has a ~1cm precision.
- "(almost) unsimplified geometries"
- zoom12 (2^12) AND d20 precision ( 2^20 = 1048576x1048576 )
- so 1/(2^32) of the size of Earth, or about ~1cm,
- Planet = 36 GB; weekly updated.
I need z18 tiles informations locally inside my mbtiles file
I don't know what other program gets this z18 data, but I would check if it supports overzooming.
In the end i have created a shell script and made z18 vector tiles for each country, which worked perfectly beside russia. Now im merging them with DB browser.