[Feature Request] Please add full support of NaN and +-Inf.

[zelandoramen1]

Please add full support of NaN and +-Inf. They are part of IEEE 754 double-precision specification ie 7fff ffff ffff ffff = NaN https://en.wikipedia.org/wiki/Double-precision_floating-point_format Sometimes NaN and InF are used in data. For example NaN may be used to show no data from sensor. NaN is commonly used in financial time series to store no rate event. At the moment in Influxdb 0.9.3 we can write NaN into DB, but it is not possible to select any seiries that include NaN. See https://github.com/influxdb/influxdb/issues/3994 and https://github.com/influxdb/influxdb/issues/3539 As far as I undestand there is no support of Nan in Json. https://github.com/golang/go/issues/3480 May be it is possible to add full IEEE 754 support to InfluxDB line protocol.

[mitar]

+1

We do measurements of our sensors at regular intervals. It is very useful to be able to store that there was a measurement cycle run but no value was obtained (because sensors was not available) vs. that there was no measurement cycle itself because the monitoring daemon was down. With other data backends we use NULL/NaN values to signal the first condition.

This also allows us to draw graphs properly. When there is a NaN we know that we can break the line between datapoints. Otherwise we can show a line between datapoints.

[mitar]

BTW, is NaN and null value the same for the purpose of this ticket?

[kostko]

BTW, is NaN and null value the same for the purpose of this ticket?

Yes, actual null values are ignored by InfluxDB. So having NaN would be great.

[mitar]

Or, alternatively, one should be able to store null values. :-)

[mitar]

But when doing for example COUNT, it is good that NaN values are ignored the same as null values.

[kostko]

Yes, with aggregations, NaN should behave the same as null. The problem is only for the highest granularity (without any aggregations). Because when you do aggregations you already get nulls for empty buckets.

[HorstBort]

I would really love to see this implemented. I'm currently setting up InfluxDB + Grafana to monitor a cold atom experiment, and some of our sensors can return Inf.

[wcork]

+1 to this. Some of our data sources could return NaN or Inf. This should be handled somehow in InfluxDB.

[daviskirk]

+1 The same goes for invalid data or measurement errors. An invalid point is different than a missing point and is useful information.

Perhaps a more specific case is when data is corrected after it is written. If I overwrite a point with a value it is sent out through all udp/http subscriptions. Deleting the points will not trigger anything as far as I can see, so any actions dependent on subscriptions like kapacitor tasks will also not pick these points up.

[SanderMertens]

-1 to this.

NaN is not meant to represent missing values, but is a mechanism to cope with a side effect of the IEEE floating point encoding. If a sensor / datasource reports a NaN, the insert query should reformat this to something representing a missing value (null for example).

It is useful to be able to indicate absence of data, but IMO this should be done in a way that is consistent across types (boolean or integer fields do not have NaN).

Edit: Having inf and -inf would be useful.

[simaamis]

Hello: has been issue been resolved? Can we store NaN values in a float type columns? The distinction between null values and Nan is also important for us.

[juliusv]

To better support storing of Prometheus data, it would be important to support not only NaN in general, but to be able to store exact bit representations of different NaN values. See this Wikipedia article about the different NaN types: https://en.wikipedia.org/wiki/NaN#Signaling_NaN

Prometheus uses a normal (non-signalling) NaN for e.g.:

• A quantile value for which there were no observations yet.
• The outcome of certain floating-point operations.

In addition, Prometheus uses a signalling NaN to indicate that a series has gone stale. E.g. a series was present in a target in the last scrape, but now it's not. Or when a target disappears in an orderly way, we write out stale markers for all of its series. This is then used by Prometheus to immediately not return stale-marked series anymore after the staleness timestamp.

Here is the definition of the two NaNs used in Prometheus: https://github.com/prometheus/prometheus/blob/master/pkg/value/value.go#L20-L29

[ravz]

+1

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[timhallinflux]

this is planned as part of 2.0

[Fluepke]

Please don't make it a default, make it an option. +/- Inf is explicitly not supported by JSON as of RFC8259 and GoLang should adhere to standards.

RFC8259 Section 6 Numeric values that cannot be represented in the grammar below (such as Infinity and NaN) are not permitted.

If you want to change the JSON standard, propose a new RFC obsoleting RFC8259.

[mitar]

What has JSON support for those values with wanting to store those values into a database?

[foobar]

nice to have in 1.x

[flash0077]

I see Influx version 2.0 is reIeased , Is (+,-)Infinity/NaN Support Added in Influx 2.0?

[timhallinflux]

We didn't make these changes at storage tier to address this in the initial 2.0 release. The change ended up being pretty invasive in a way that would have really prevented an in-place upgrade from 1.x to 2.x.

However, we did announce a new project which will address these items (and more). This will become an opt-in feature at some point down the line for 2.x. Follow the details here: https://github.com/influxdata/influxdb_iox

[pauldix]

That's right, this will be coming as part of InfluxDB IOx. At least for the underlying storage system. The bigger question is how you want this exposed via the Write and Query API. InfluxDB IOx supports the 2.0 Line Protocol write and the CSV return format. But it'll also support new ways of writing data in and new ways of pulling data out.

With JSON, this is tricky because you can't represent those. You end up doing something horrible like using a string and then dealing with it at parse time.

[blueforesticarus]

Another use-case:

I have data that resets to zero occasionally, but when viewing I would like to remove the resets. It is easy enough to write a flux query which will changes over the time range you provide: difference() |> cumulativeSum()

if I could write an NaN into the database when the value resets, then this would behave ideally: the difference would have two null values, before and after the NaN, and then fill(value: 0) |> cumulativeSum() would recreate the trace except without the large jumps at reset. This is all entirely sensible, the NaN is used to represent a break in the continuity of the data, and this causes the delta to be null (correct behavior).

instead, there is absolutely no way to write null/NaN into the database, so I (as a hack) write in -1, and am left to try to make some crazy flux query which filters the jump.

|> map(fn: (r) => ({ r with _value: (if r._value == -1.0 then math.NaN() else r._value) }))
|> difference()```