logstash-filter-verifier icon indicating copy to clipboard operation
logstash-filter-verifier copied to clipboard

Would be useful to have support for something like drip for faster startup.

Open choffee opened this issue 7 years ago • 4 comments

I tried using drip, https://github.com/ninjudd/drip, to speed up the tests, each one currently taking 20seconds. It seems not to work with my setup but sounds like a good thing.

USE_DRIP=1 logstash-filter-verifier tests/ logstash.conf --keepenv=USE_DRIP

Also noted that the logstash readme says it does not work with stdin which might make this a dead end anyway.

Another option might be to have just one logstash process and pass all the test through one by one but that may lead to it waiting for a test to finish. ( Timeouts could deal with that? )

choffee avatar Jan 16 '17 15:01 choffee

20 seconds to process a single test case file? That sounds very slow.

But yes, I suspect lack of support for stdin would make use of drip hard. I suppose one could use a single input file with a sentinel event sent between the sequences of input lines from each test case file.

magnusbaeck avatar Jan 22 '17 15:01 magnusbaeck

@choffee The 20 seconds are because of the slow Logstash startup time. Do you already use the possibility to have multiple input and expect in the same test case file? This is possible, as long as these test cases share the same fields and codec settings. This speeds up the processing dramatically.

breml avatar Jan 23 '17 07:01 breml

I have switched to using the --socket option in 1.2 and this speeds things up greatly. I would say this is less urgent now. It might still be useful for development work but all the tests run very quickly now after the initial logstash start up.

choffee avatar Feb 22 '17 14:02 choffee

@choffee Good the hear that you like the new option. I'm thinking about to speed up LFV even more, but this is not yet ready.

breml avatar Feb 22 '17 15:02 breml