fiware-cygnus icon indicating copy to clipboard operation
fiware-cygnus copied to clipboard

[cygnus-ngsi][NameMapping] Allow deep Json attribute mapping.

Open pmo-sdr opened this issue 4 years ago • 6 comments

It will be nice to be able to map specific NGSI Json attributes to a final persisted field.

Example:

We have a NGSI entity with an address field like this:

"address": {
    "type": "PostalAddress",
    "addressLocality": "Santander",
    "addressRegion": "CA",
    "postalCode": "39002",
    "streetAddress": "39002 Plaza Ayuntamiento Nº 1"
  }

But we want to have a Mysql table with fields, city and region instead of a single field address within whole json expression. To do this, namemapping config should look like:

"attributeMappings": [
                        {
                           "originalAttributeName": "address.addressLocality",
                           "newAttributeName": "city",
                           ...
                        }, 
                        {
                           "originalAttributeName": "address.addressRegion",
                           "newAttributeName": "region",
                           ...
                        },

Mapping could use JsonPath-like expressions.

pmo-sdr avatar Apr 16 '20 08:04 pmo-sdr

Solution "Highlights":

  • Mapping interceptor "knows" if provided expression is a regex or a Jsonpath expression.
  • If JsonPath, original attribute name is extracted from JsonPath expression
  • Mapping can be configured not to stop at first match.

pmo-sdr avatar Apr 16 '20 09:04 pmo-sdr

It seems to be a interesting feature. What do you mean with "Mapping can be configured not to stop at first match"? does it mean that when an attribute regexp is matched, it doesn't go on with the next attribute?

manucarrace avatar Apr 20 '20 06:04 manucarrace

Yes, nowadays, name mapping stops checking mapping rules when it finds a match. This is, Name Mapping takes an attribute, and tries to match it with each mapping rule, stopping when it finds the first matching rule, and not going on with the rest of the rules.

It makes sense, since it's the most common behavior, and stopping it produces a significant performance improvement.

But sometimes, we may need to "clone" or "split" NGSI attributes to fit final persistence structure, this is why the new feature will be configurable at agent level.

pmo-sdr avatar Apr 20 '20 07:04 pmo-sdr

New section added to docs explaining how it will work. Advanced Mapping section, Clone attributes and Using JsonPath Expressions

https://github.com/Stdri/fiware-cygnus/blob/JsonPath/doc/cygnus-ngsi/installation_and_administration_guide/name_mappings.md

pmo-sdr avatar Apr 20 '20 07:04 pmo-sdr

Nice. For me it's a very interesting feature but only for attributes. I see no point in apply it to entities, subservices or services, for the time being ...

manucarrace avatar Apr 22 '20 09:04 manucarrace

PR https://github.com/telefonicaid/fiware-cygnus/pull/1860

fgalan avatar Apr 22 '20 10:04 fgalan