ngsi-timeseries-api icon indicating copy to clipboard operation
ngsi-timeseries-api copied to clipboard

How to deal with nulls?

Open c0c0n3 opened this issue 5 years ago • 1 comments

Quantum Leap may store a null attribute value even if the attribute wasn't originally sent by the device. This is especially so in batch notifications where we take the union of all the attributes for each entity type in the batch---see #191.

How should a DB null be interpreted? As it stands now it could be either because the device sent an attribute with an empty value or it could be we added that null ourselves.

This may not be a problem if we add a query parameter to filter nulls out. The only problem I can think of would be the following, but am not sure this is actually something that could happen in practice---my IoT experience is very limited, but @chicco785 @pooja1pathak may care to comment.

Say I have a device with two sensors, one for temperature and the other for pressure, and say it sends temperature measurements every minute whereas pressure gets measured only every second minute. I may wind up with entity data that looks like this:

pressure:    null    4   null  null   ...
temperature:  10    15    16    18    ...
time:          1     2     3     4    ...

The nulls at time points 1 and 3 where introduced by Quantum Leap whereas the one at time point 4 was the result of the device failing to measure pressure so it sent us an empty attribute. Now say I'd like figure out the pressure sensor fault rate. Just looking at nulls won't cut it nor can I filter out all the nulls in any meaningful way...

c0c0n3 avatar Jun 07 '19 12:06 c0c0n3

Stale issue message

github-actions[bot] avatar Mar 10 '21 02:03 github-actions[bot]