imposm2 icon indicating copy to clipboard operation
imposm2 copied to clipboard

Add support for PostgreSQL HSTORE

Open aronbierbaum opened this issue 11 years ago • 5 comments

This adds a use_hstore parameter that will cause all OSM tags to be added to a hstore tags column. This can be useful when you want to query the features for all available data.

aronbierbaum avatar Nov 06 '13 01:11 aronbierbaum

Cool. I think you still need to get the filters to not filter out any tags at all, if you want your HStore to contain everything.

olt avatar Nov 07 '13 09:11 olt

Thank you for pointing that out. I have added a function set_hstore_all that should disable all filtering. Does this seem like a reasonable approach?

aronbierbaum avatar Nov 07 '13 14:11 aronbierbaum

Using a global similar to the name type was the simplest way I could come up with for disabling tag filtering. This makes we wonder if we should still have a use_hstore property on each mapping though. If I understand things correctly it does still add some value though because there are two levels of tag filtering. The first level determines what is parsed and the second determines what is actually added to the database.

aronbierbaum avatar Nov 07 '13 15:11 aronbierbaum

The first filtering reduces the data in the imposm cache files (cache only tags you're using later). In the PostGIS package it decides which tag needs to be inserted into the actual tables.

You still need to do that. But only by disabling the first filter, you're able to insert all tags into the hstore column of a table.

olt avatar Nov 07 '13 15:11 olt

I'm not sure if a global variable and use_hstore is the best solution. What if someone wants to have only a list of keys in the hstore column, or if he wants to filter out some columns? I would like to see hstore as another field type with the option to include/exclude keys, or to take all.

E.g:

amenities = Points(
    name='amenities',
    fields=('tags', HStore(include=['opening_hours', ....]),),
    mapping = {
        'amenity': (
            '__any__',
        ),
})

HStore.extra_fields could return a special key to signal the _tag_filter that it wants all tags.

olt avatar Nov 07 '13 15:11 olt