api icon indicating copy to clipboard operation
api copied to clipboard

use parent field values from same layer when deduplicating

Open missinglink opened this issue 3 years ago • 1 comments

In some cases the parent hierarchy contains tokens which are relevant for deduplication.

For example if we have a geonames record on layer=locality with the name.default="Land Berlin" and also parent.locality=["Berlin"].

In this case we can use the parent field provided by the PIP service to assist in the deduplication process.

missinglink avatar Aug 17 '21 11:08 missinglink

Coming back to this now after thinking it over...

I think it's good to merge, the concept that the parent names at the same layer as the feature can be used for deduplication is logical and unlikely to cause error.

The issue description was missing the concrete example which was included in the tests:

Given the feature geonames:region:2950157 ("Land Berlin", en: "State of Berlin") along with another feature at the same level whosonfirst:region:85682499 ("Berlin") in the results.

We can use the parent.region = ["Berlin"] property of geonames:region:2950157 to establish that it is equivalent to whosonfirst:region:85682499

The parent.region_id = [85682499] property of geonames:region:2950157 further indicates that the two are duplicate, so we could consider only using the IDs, I'm open to both methods.

missinglink avatar Aug 30 '21 13:08 missinglink