stream-reactor
stream-reactor copied to clipboard
Influxdb-sink connector: adding an extra field with sys_time()
@stheppi in the influxdb-sink connector, we are currently using a KCQL query to extract the timestamp from a field:
INSERT INTO mytopic1 SELECT * FROM mytopic1 WITHTIMESTAMP producedTimeStamp
in our system produceTimeStamp
corresponds to the time when the the data was produced.
It would be really useful if we could write sys_time()
to an extra field to record the time when the point is written to InfluxDB. Then sys_time - procudedTimeStamp
would help us to characterize the latency in our system.
Looking at the InfluxDB Sink connector documentation it seems that the only alternative is:
INSERT INTO mytopic1 SELECT * FROM mytopic1 WITHTIMESTAMP sys_time()
but then sys_time()
is used as the InfluxDB timestamp and that is not what we want.
Is it possible to write a KCQL to add an extra field with sys_time()
? if not, how much work is involved in changing the connector code to add this extra field?
hey @stheppi do you have any comments on this issue? or any pointer to the connector code in case we want to add an extra field with the value of sys_time()
?