fiware-draco
fiware-draco copied to clipboard
feat: extend NGSI-LD support for NGSIToPostgreSQL processor (and other global upgrades)
Note: submitting the PR in draft status since there are still some things to finish (see TODOs below), but new features are already in good shape and tested on a real deployment.
Proposed changes
Globally extend the NGSI-LD support of the NGSIToPostgreSQL processor and also do some global upgrades (NiFi version, Java version):
- chore: upgrade NiFi version to 1.14.0 (only tested for NGSIToPostgreSQL processor)
- chore: set base Java version to 11 for nifi-ngsi-bundle
- chore: replace part of System.out statements by log statements
- feat: extract keys to ignore in payloads into a constant list for improved maintainability
- feat: add support for multi-attributes (section 4.5.5 of the NGSI-LD spec) to avoid too long (and truncated) names
- a configurable prefix can be removed from the dataset ids
- feat: add support for sub-attributes of all types of attributes (was previously only supported for properties)
- feat: use types as much as possible for columns (e.g. TIMESTAMPTZ for dates or NUMERIC for numeric values)
- currently using TEXT (default), TIMESTAMPTZ and NUMERIC (more could be added in the future)
- it won't break existing installations because columns already exist, but they could benefit from such a new feature
TODOs before removing the draft status
- align NGSIv2 code where needed (e.g., in
getValuesForInsert
,listOfFields
) - add some specific tests for the new features
- upgrade bundle version (to which version?)
Types of changes
What types of changes does your code introduce to the project: Put an x
in the boxes that apply
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
Checklist
- [x] I have read the CONTRIBUTING doc
- [x] I have signed the CLA
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] I have added necessary documentation (if appropriate)
- [ ] Any dependent changes have been merged and published in downstream modules
I have read the CLA Document and I hereby sign the CLA