logstash-filter-ruby
logstash-filter-ruby copied to clipboard
The Logstash Ruby plugin documentation is incomplete.
(This issue was originally filed by @dbeckham at https://github.com/elastic/logstash/issues/2227)
The Ruby plugin documentation is incomplete and only documents the basic configuration option details.
Specifically, the code configuration option is the most important part of the configuration options, but it is one of the least documented options.
Issues:
- There are no links to documentation or Logstash code that describes what the
event
variable is, or even what type of variable it is. We can assume it's some sort of hash or object based on the single example, but other than that, there are no details. What attributes or methods are available? Is it immutable? Can you alter the contents of other fields? - What constraints are placed on Ruby code used with this plugin? i.e. are there any special keywords that can not be used, limits on methods used, or any particular characters that need special escaping?
- What happens when Ruby code throws an exception? Does Logstash silently drop the event, or does the filter just fail?
A simple link to the description of the event variable, or a wider range of examples may help solve several of these issues.
Dang - 6 months later and I still cannot find a simple description of the event variable!
same problem troubled me too. My problem is specific as below.
the event message is:
{"a":[{"aa":"v1"},{"bb":"v2"}]}
how to add a field ,namely, "a_size" and its value is array a's length. so the output is like below:
{"a":[{"aa":"v1"},{"bb":"v2"}],"a_size":2 }
I found this issue looking for the same info about the event
variable. After a little more digging I found the docs for the Event API. This is exactly what I needed. It would be nice if there was a direct link to this API documentation from the Ruby plugin documentation.
Ooh, was tearing my hair out trying to figure out where any documentation on the ruby filter API was. Had to rely on issues like https://github.com/logstash-plugins/logstash-filter-ruby/issues/21 to figure out how to set fields. That Event API documentation is really useful, but it'd be nice if it were linked from the Ruby filter docs.
The great thing is that the docs are open source, so I'll submit a PR to add a link. The thing that's a bit unfortunate is that I'm still on 2.x so the docs there actually aren't relevant to my version, but I'm working on upgrading to 5 soon...
Edit: Hooray, my PR was accepted! https://github.com/logstash-plugins/logstash-filter-ruby/pull/34 I still think more documentation would be helpful but that should at least get people on the right track.
In particular, I'd like to know if i can do things like define functions, classes, etc. I'm guessing you probably can, but the examples given are quite short.
One other question to answer here: you can indeed define methods inside the filter. The filter is basically being evaluated inside the body of a lambda; see: https://github.com/logstash-plugins/logstash-filter-ruby/blob/master/lib/logstash/filters/ruby.rb#L38
And apparently you can define methods in a lambda block. So it works, not sure whether there are any efficiency/overhead concerns.
~~Just kidding, looks like you perhaps cannot. I thought it was working for a second but realized I didn't deploy the extra filter, but when I did include a filter which defines methods in the code parameter, I got this error: Ruby exception occurred: class or module required
~~
Just kidding again, that was a typo in my own code.