kapacitor icon indicating copy to clipboard operation
kapacitor copied to clipboard

Variable Field Names

Open octalthorpe opened this issue 8 years ago • 4 comments

I can accomplish this task by having seperate tick scripts, but since the scripts are doing the same thing but on different fields of a measurement, I figured I would combine them into one TICK script.

The issue I am having is two fold.

1 )AWS CW metrics sometimes will push out metrics that do not have the all fields populated. i.e. one point may have a tag for "domain_name" but the field names vary all over the place, so I needed to filter out only the points that have a "domain_name" and "cluster_status.yellow_maximum" or "cluster_status.red_maximum" So I am doing a eval/keep to filter that out. This works fine as long as I don't use a variable for the field, i.e.

    |eval(lambda: "domain_name", lambda:  "cluster_status.yellow_maximum")
       .as('_domain_name', '_cluster_status')
       .keep()
       .quiet()

So problem #1 solved.

  1. Now the variable issue if I want to have a single TICK script the above would be:
    |eval(lambda: "domain_name", lambda:  measurement_field)
       .as('_domain_name', '_cluster_status')
       .keep()
       .quiet()

where measurement_field is a variable, a string to be exact.

var measurement_field = "cluster_status.yellow_maximum"

Now this means that the metrics push into the next node would the proper domain name, but simply the string value of "measurement_field"

[elasticsearch_status-yellow:log5] 2017/01/23 20:33:48 I!  {"Name":"cloudwatch_aws_es","Database":"local","RetentionPolicy":"default","Group":"","Dimensions":{"ByName":false,"TagNames":null},"Tags":{"deployment":"local","domain_name":"test"},"Fields":{"_cluster_status":"cluster_status.yellow_maximum","_domain_name":"test","cluster_status.yellow_maximum":1},"Time":"2017-01-23T20:33:48.51813888Z"}

So my question is in a lambda expression, how do I go about expanding the variable variable? Kinda like a double eval??

I am running kapacitor 1.1.1

Any help would be greatly appreciated.

octalthorpe avatar Jan 23 '17 20:01 octalthorpe

I agree that being able to dereference field and tag names stored in a TICK variable within a lambda would be very useful and really expand the ability to template tasks.

AdamSLevy avatar Dec 24 '17 00:12 AdamSLevy

Any update on this issue ? I'm working with: InfluxDB shell version: 1.5.2 Kapacitor OSS 1.5.0 (git: HEAD 4f10efc41b4dcac070495cf95ba2c41cfcc2aa3a) RH version: Red Hat Enterprise Linux Server release 7.2 (Maipo)

baruchih-amdocs avatar Jul 31 '18 11:07 baruchih-amdocs

Related to https://github.com/influxdata/kapacitor/issues/1511.

Unfortunately, there doesn't seem to be a workaround.

lafrech avatar Jun 20 '19 09:06 lafrech

This could be very useful for scalability. I'm guessing there is no workaround still. I'd be happy to work on this feature, if someone is up to giving me some pointers.

cisakoglou avatar Jan 17 '23 18:01 cisakoglou