mtools icon indicating copy to clipboard operation
mtools copied to clipboard

Add InfluxDB support

Open mroch opened this issue 9 years ago • 2 comments

I'd like to write directly to InfluxDB and log with grafana.

I think the format should look like (in InfluxDB line format):

measurement_name XXXX123_ch1_aws=1234.0,XXXX123_ch2_aws=345.0 123456789000000000
measurement_name XXXX789_ch1_aws=1.0,XXXX789_ch2_aws=2.0 123456789000000000

That is, each channel reading should be a field, named for the serial number and channel. (note that since the timestamps and measurement name + tags match, this is a single measurement with 4 fields... i just split it for readability).


An alternative would be

measurement_name,serial=XXXX123 ch1_aws=1234.0,ch2_aws=345.0 123456789000000000
measurement_name,serial=XXXX789 ch1_aws=1.0,ch2_aws=2.0 123456789000000000

which is two separate series. I don't think this is as valuable because there's no reason you'd want to query for all the ch1_aws values across tags.

Another alternative would be

measurement_name,serial=XXXX123,id=ch1_aws value=1234.0 123456789000000000
measurement_name,serial=XXXX123,id=ch2_aws value=345.0 123456789000000000
measurement_name,serial=XXXX789,id=ch1_aws value=1.0 123456789000000000
measurement_name,serial=XXXX789,id=ch2_aws value=2.0 123456789000000000

The problem with this format is that each channel is a separate series, so you can't do interesting things like add two values together. For example, the plugs in my kitchen are on 3 different branch circuits, so I'd like to have a "Kitchen" value that is mean("ch1_aws") + mean("ch2_aws") + mean("ch3_aws") but you can't do that. This is possible with the proposed format above, though.

mroch avatar Aug 27 '16 19:08 mroch

My current usage is most like the bottom example and, you're right, it gets tricky when attempting to look at multiple series together.

I'm not sure how avoidable that is. I run 2x btmon's, one covers a GEM on the subpanel and the other for the Main panel. With this config, I'd get 2 (or 2n) series anyhow (either row or column oriented, depending upon the other choices.

I've only lightly played with Grafana/InfluxDB, so I'm not sure what options are available there to get the rollups needed to make it work nicely.

BTW: For me, I'd like to do all the logical mappings up front on the collection/btmon end. This avoids having the impl channel/device#s (serial/id) on the persistence end.

eg. ch1_aws -> kitchen_main

If I ever sub-out the devices for something else, it means the data streams will remain the same.

It may be possible to get both by having a format-string, with well-known substitutions. Folks wanting to push specific tag values over could then tweak it to push that data over, or some alternate.

Presumably folks could use that mechanism, if combined with the column oriented persistence, to tag-like-crazy and pull the data out using those tags.

eg. measurement_name,...,room=kitchen,circuit=coffee_aws value=

mrguessed avatar Sep 01 '16 16:09 mrguessed

@mrguessed I updated #4 with an option to do either the top or bottom formats.

i'm also using 2 GEMs (main and subpanel) but only one instance of btmon. I'm using trust_device_clock = true so the timestamps come from the GEM to avoid drift from processing delays. each GEM updates every 10 seconds, but one updates on the 7's (:07, :17, ...) and the other updates on the 8's (:08, :18, ...). when using influxdb_mode = row, I have a single measurement with columns for "kitchen_lights", "family_room_lights", etc, that ends up looking like

measurement_name kitchen_lights=12345 <timestamp for :07>
measurement_name family_room_lights=12345 <timestamp for :08>

GROUP BY time(10s) will round the timestamps, so it ends up looking like a single point:

measurement_name kitchen_lights=12345 family_room_lights=12345 <timestamp for :00>

which lets me calculate things like "power used by all lights".

mroch avatar Sep 06 '16 02:09 mroch