imposm3 It does not update the geometry in the db after correcting the shape of an object

It does not update the geometry in the db after correcting the shape of an object

Open geofrizz opened this issue 4 years ago • 2 comments

Context

Updated position of the nodes in a rounded building !! I discovery some objects with a geometry deformed (circle deformed in a line) eg: osm_id = 600860114 and 345949779

before the update (from the imposm 3 cache):

{
  "ways": {
    "600860114": {
      "ID": 600860114,
      "Tags": {
        "building": "yes"
      },
      "Metadata": null,
      "Refs": [
        5712178893,
        5712178894,
        5712178895,
        5712178896,
        5712178897,
        5712178898,
        5712178899,
        5712178900,
        5712178882,
        5712178883,
        5712178886,
        5712178887,
        5712178888,
        5712178889,
        5712178890,
        5712178891,
        5712178892,
        5712178884,
        5712178885,
        5712178893
      ],
      "Nodes": null,
      "nodes": {
        "5712178882": {
          "ID": 5712178882,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871589628094341
        },
        "5712178883": {
          "ID": 5712178883,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871591136836912
        },
        "5712178884": {
          "ID": 5712178884,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871639668056304
        },
        "5712178885": {
          "ID": 5712178885,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.87163899750405
        },
        "5712178886": {
          "ID": 5712178886,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871595327788498
        },
        "5712178887": {
          "ID": 5712178887,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.87160178185394
        },
        "5712178888": {
          "ID": 5712178888,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.87160966084295
        },
        "5712178889": {
          "ID": 5712178889,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871618126565153
        },
        "5712178890": {
          "ID": 5712178890,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.87162625701123
        },
        "5712178891": {
          "ID": 5712178891,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871633046352798
        },
        "5712178892": {
          "ID": 5712178892,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871637656399542
        },
        "5712178893": {
          "ID": 5712178893,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871635644742781
        },
        "5712178894": {
          "ID": 5712178894,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871630364143783
        },
        "5712178895": {
          "ID": 5712178895,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.87162332334512
        },
        "5712178896": {
          "ID": 5712178896,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871615444356138
        },
        "5712178897": {
          "ID": 5712178897,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871607481548125
        },
        "5712178898": {
          "ID": 5712178898,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871600273111369
        },
        "5712178899": {
          "ID": 5712178899,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.87159448959818
        },
        "5712178900": {
          "ID": 5712178900,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743381797351958,
          "Long": 13.871590885379817
        }
      }
    }
  }
}

in the db:

sde_pg_3=> select  id,osm_id,class,fclass,tags,st_astext(geom),st_geohash(geom,1) from osm_planet.polygons_building_yes where osm_id=600860114 ;
    id     |  osm_id   |  class   | fclass |       tags        |   st_astext   | st_geohash 
-----------+-----------+----------+--------+-------------------+---------------+------------
 339224248 | 600860114 | building | yes    | "building"=>"yes" | POLYGON EMPTY | 
(1 row)

I updated only the position of every nodes, I don't added or deleted nodes !! After updated nodes position in OSM:

in the imposm 3 cache:

{
  "ways": {
    "600860114": {
      "ID": 600860114,
      "Tags": {
        "building": "yes"
      },
      "Metadata": null,
      "Refs": [
        5712178893,
        5712178894,
        5712178895,
        5712178896,
        5712178897,
        5712178898,
        5712178899,
        5712178900,
        5712178882,
        5712178883,
        5712178886,
        5712178887,
        5712178888,
        5712178889,
        5712178890,
        5712178891,
        5712178892,
        5712178884,
        5712178885,
        5712178893
      ],
      "Nodes": null,
      "nodes": {
        "5712178882": {
          "ID": 5712178882,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743379031323883,
          "Long": 13.871596585073974
        },
        "5712178883": {
          "ID": 5712178883,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743385066294195,
          "Long": 13.871597255626227
        },
        "5712178884": {
          "ID": 5712178884,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743384395741941,
          "Long": 13.871633968362147
        },
        "5712178885": {
          "ID": 5712178885,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743378276952598,
          "Long": 13.871634387457306
        },
        "5712178886": {
          "ID": 5712178886,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743390598350288,
          "Long": 13.87159985401621
        },
        "5712178887": {
          "ID": 5712178887,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743394956939937,
          "Long": 13.871604128786856
        },
        "5712178888": {
          "ID": 5712178888,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743397639148952,
          "Long": 13.871609744661981
        },
        "5712178889": {
          "ID": 5712178889,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743398561158301,
          "Long": 13.871615863451296
        },
        "5712178890": {
          "ID": 5712178890,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743397471510889,
          "Long": 13.871622066059643
        },
        "5712178891": {
          "ID": 5712178891,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743394454025747,
          "Long": 13.871627430477673
        },
        "5712178892": {
          "ID": 5712178892,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743389843979003,
          "Long": 13.871631537610227
        },
        "5712178893": {
          "ID": 5712178893,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.74337249343941,
          "Long": 13.87163262725764
        },
        "5712178894": {
          "ID": 5712178894,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743367464297506,
          "Long": 13.87162919067734
        },
        "5712178895": {
          "ID": 5712178895,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743363860079143,
          "Long": 13.871624161535436
        },
        "5712178896": {
          "ID": 5712178896,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743361932241413,
          "Long": 13.871618126565153
        },
        "5712178897": {
          "ID": 5712178897,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743362099879477,
          "Long": 13.871611923956806
        },
        "5712178898": {
          "ID": 5712178898,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.74336419535527,
          "Long": 13.871606140443618
        },
        "5712178899": {
          "ID": 5712178899,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743368051030728,
          "Long": 13.87160127893975
        },
        "5712178900": {
          "ID": 5712178900,
          "Tags": null,
          "Metadata": null,
          "Lat": 13.743373163991663,
          "Long": 13.871597926178481
        }
      }
    }
  }
}

But in the db the geometry update is not applied !!

sde_pg_3=> select  id,osm_id,class,fclass,tags,st_astext(geom),st_geohash(geom,1) from osm_planet.polygons_building_yes where osm_id=600860114 ;
    id     |  osm_id   |  class   | fclass |       tags        |   st_astext   | st_geohash 
-----------+-----------+----------+--------+-------------------+---------------+------------
 339224248 | 600860114 | building | yes    | "building"=>"yes" | POLYGON EMPTY | 
(1 row)

Expected Behavior

Update of all elements of the geometry in the db !!

Actual Behavior

Imposm update only the imposm 3 cache but not the db !!

Possible Fix

Temporary for resolve the problem it's possible to add a new tag in the way, eg: name

{
  "ways": {
    "600860114": {
      "ID": 600860114,
      "Tags": {
        "building": "yes",
        "name": "test for update"
      },
      "Metadata": null,
      "Refs": [
        5712178893,
        5712178894,
	...

In the db:

sde_pg_3=> select  id,osm_id,class,fclass,tags,st_astext(geom),st_geohash(geom,1) from osm_planet.polygons_building_yes where osm_id=600860114 ;
   id    |  osm_id   |  class   | fclass |                     tags                     |                                                                                                                                                                                                                                                                                                                                                   st_astext                                                                                                                                                                                                                                                                                                                                                   | st_geohash 
---------+-----------+----------+--------+----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------
 1882534 | 600860114 | building | yes    | "name"=>"test for update", "building"=>"yes" | POLYGON((13.871596585074 13.7433790313239,13.8715972556262 13.7433850662942,13.8715998540162 13.7433905983503,13.8716041287869 13.7433949569399,13.871609744662 13.743397639149,13.8716158634513 13.7433985611583,13.8716220660596 13.7433974715109,13.8716274304777 13.7433944540257,13.8716315376102 13.743389843979,13.8716339683621 13.7433843957419,13.8716343874573 13.7433782769526,13.8716326272576 13.7433724934394,13.8716291906773 13.7433674642975,13.8716241615354 13.7433638600791,13.8716181265652 13.7433619322414,13.8716119239568 13.7433620998795,13.8716061404436 13.7433641953553,13.8716012789397 13.7433680510307,13.8715979261785 13.7433731639917,13.871596585074 13.7433790313239)) | s
(1 row)

Steps to Reproduce

Other objects have the same problem: 611280979 (way) 621705015 (way) 621705026 (way) 644554965 (way) 644554981 (way) ... and other

Context

Your Environment

Version used: imposm 3 v 0.10.0
Environment name and version (e.g. PostgreSQL x.y.z with PostGIS a.b.c): postgresql: PostgreSQL 11.9 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit postgis: 2.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1
Server type and version: VMWare virtual server
Operating System and version: Red Hat Enterprise Linux Server release 7.8 (Maipo)

Sep 24 '20 14:09 geofrizz

Steps to Reproduce

imho: if you can add a

minimal reduced osm xml file ( 1 osm way with a minimal nodes )
minimal osc file ( minimal osm xml change file )
minimal imposm3 mapping file

then it would be helpful in the debugging and the fixing and we can add this file as a test file ..

like the "single_table" test in the /test directory ..

or you can use the ./test/expire_tiles_mapping.yml as an example

Sep 24 '20 15:09 ImreSamu

File .osm:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
  <node id='5787817829' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186661' />
  <node id='5787817830' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186679' />
  <node id='5787817831' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186664' />
  <node id='5787817832' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186094' />
  <node id='5787817833' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186061' />
  <node id='5787817834' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.818606' />
  <node id='5787817835' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186093' />
  <node id='5787817836' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186616' />
  <node id='5787817837' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186541' />
  <node id='5787817838' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186447' />
  <node id='5787817839' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186344' />
  <node id='5787817840' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186244' />
  <node id='5787817841' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186158' />
  <node id='5787817842' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186155' />
  <node id='5787817843' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.818624' />
  <node id='5787817844' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186339' />
  <node id='5787817845' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.818644' />
  <node id='5787817846' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186534' />
  <node id='5787817847' timestamp='2020-03-03T09:40:34Z' uid='6334327' user='mhrana98' visible='true' version='2' changeset='81718087' lat='22.6177832' lon='89.8186611' />
  <way id='611280979' timestamp='2018-07-26T19:14:05Z' uid='4439816' user='mrburatti' visible='true' version='1' changeset='61098226'>
    <nd ref='5787817842' />
    <nd ref='5787817843' />
    <nd ref='5787817844' />
    <nd ref='5787817845' />
    <nd ref='5787817846' />
    <nd ref='5787817847' />
    <nd ref='5787817829' />
    <nd ref='5787817830' />
    <nd ref='5787817831' />
    <nd ref='5787817836' />
    <nd ref='5787817837' />
    <nd ref='5787817838' />
    <nd ref='5787817839' />
    <nd ref='5787817840' />
    <nd ref='5787817841' />
    <nd ref='5787817832' />
    <nd ref='5787817833' />
    <nd ref='5787817834' />
    <nd ref='5787817835' />
    <nd ref='5787817842' />
    <tag k='building' v='yes' />
  </way>
</osm>

file .osc:

<?xml version='1.0' encoding='UTF-8'?>
<osmChange version="0.6" generator="Osmosis 0.47.4">
  <modify>
    <node id="5787817829" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177629" lon="89.8186479"/>
    <node id="5787817830" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177664" lon="89.8186535"/>
    <node id="5787817831" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177715" lon="89.8186575"/>
    <node id="5787817832" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177984" lon="89.8186334"/>
    <node id="5787817833" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177957" lon="89.8186272"/>
    <node id="5787817834" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177913" lon="89.8186224"/>
    <node id="5787817835" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177857" lon="89.8186193"/>
    <node id="5787817836" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177775" lon="89.8186595"/>
    <node id="5787817837" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177838" lon="89.8186593"/>
    <node id="5787817838" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177897" lon="89.8186569"/>
    <node id="5787817839" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177945" lon="89.8186526"/>
    <node id="5787817840" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177978" lon="89.8186468"/>
    <node id="5787817841" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177991" lon="89.8186401"/>
    <node id="5787817842" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177795" lon="89.8186184"/>
    <node id="5787817843" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177733" lon="89.8186197"/>
    <node id="5787817844" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177679" lon="89.8186231"/>
    <node id="5787817845" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177638" lon="89.8186283"/>
    <node id="5787817846" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177615" lon="89.8186346"/>
    <node id="5787817847" version="3" timestamp="2020-09-24T16:30:01Z" uid="9069087" user="UNGSC-DTLM-GeoFrizz" changeset="91452432" lat="22.6177611" lon="89.8186414"/>
  </modify>
</osmChange>

mapping.yml:

tags:
  load_all: true
  exclude: 
  - source*
  - comment*
  - attribution*

areas:
  area_tags: [building, landuse, leisure, natural, place]
  linear_tags: [highway, barrier, waterway, railway]
  
tables:
  polygons_building_yes:
    columns:
    - name: osm_id
      type: id
    - name: class
      type: mapping_key
    - name: fclass
      type: mapping_value
    - name: tags
      type: hstore_tags
    - name: geom
      type: validated_geometry
    mapping:
      building:
      - "yes"
    type: polygon

Sep 24 '20 17:09 geofrizz

imposm3 imposm3 copied to clipboard

It does not update the geometry in the db after correcting the shape of an object

Context

Expected Behavior

Actual Behavior

Possible Fix

Steps to Reproduce

Context

Your Environment

imposm3
imposm3 copied to clipboard