Quotes on query fields are not working

[azimut]

In the example provided the field "operation" is matched against the one "opid" on the new event. But, if opid is for example an url, and as such contains ":" or "/", it would fail.

   elasticsearch {
      hosts => ["es-server"]
      query => "type:start AND operation:%{[opid]}"
      fields => ["@timestamp", "started"]
   }

This could be fixed by quoting the whole "opid".

      query => "type:start AND operation:\"%{[opid]}\""

But this or other attempts to quote the search string are not working.

message=>"Failed to query elasticsearch for previous event", :query=>"type:rss AND link:\\\"https://www.domain.com/path/\\\""

Please advice.

[jordansissel]

As a workaround, you can use single quotes:

query => 'type:start AND operation:"%{[opid]}"'

[muaazsaleem]

@jordansissel I have been stuck on this for two days, finally your comment is what helped!

[v01t]

@jordansissel your workaround doesn't work in my case: query => '{"query":{"bool":{"must":[{"match":{"_type":{"query":"shares","type":"phrase"}}},{"range":{"date_start":{"lte":"%{[ts_date]}"}}},{"range":{"date_end":{"gt":"%{[ts_date]}"}}},{"match":{"carrier":{"query":"%{[carrier]}","type":"phrase"}}},{"match":{"market":{"query":"%{[market]}","type":"phrase"}}},{"match":{"content_type":{"query":"%{[content_type]}","type":"phrase"}}}]}}}'

:error=>#<Elasticsearch::Transport::Transport::Errors::BadRequest: [400] { error":{"root_cause":[{"type":"query_parsing_exception","reason":"Failed to parse query [{\"query\":{\"bool\":{\"must\":[{\"match\":{\"_type\":{\"query\":\"shares\",\"type\":\"phrase\"}}},{\"range\":{\"date\":{\"lte\":\"2016-07-15\"}}},{\"range\":{\"date_end\":{\"gt\":\"2016-07-15\"}}},{\"match\":{\"carrier\":{\"query\":\"carrier-1\",\"type\":\"phrase\"}}},{\"match\":{\"market\":{\"query\":\"market-0\",\"type\":\"phrase\"}}},{\"match\":{\"content_type\":{\"query\":\"content-0\",\"type\":\"phrase\"}}}]}}}]","index":"sys"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"sys","node":"nyR9DmsxTmimTQxvBXZFNQ","reason":{"type":"query_parsing_exception","reason":"Failed to parse query [{\"query\":{\"bool\":{\"must\":[{\"match\":{\"_type\":{\"query\":\"shares\",\"type\":\"phrase\"}}},{\"range\":{\"date\":{\"lte\":\"2016-07-15\"}}},{\"range\":{\"date_end\":{\"gt\":\"2016-07-15\"}}},{\"match\":{\"carrier\":{\"query\":\"telefonica-uk\",\"type\":\"phrase\"}}},{\"match\":{\"market\":{\"query\":\"gcb\",\"type\":\"phrase\"}}},{\"match\":{\"content_type\":{\"query\":\"in-app\",\"type\":\"phrase\"}}}]}}}]","index":"sys","caused_by":{"type":"parse_exception","reason":"Cannot parse '{\"query\":{\"bool\":{\"must\":[{\"match\":{\"_type\":{\"query\":\"shares\",\"type\":\"phrase\"}}},{\"range\":{\"date\":{\"lte\":\"2016-07-15\"}}},{\"range\":{\"date_end\":{\"gt\":\"2016-07-15\"}}},{\"match\":{\"carrier\":{\"query\":\"telefonica-uk\",\"type\":\"phrase\"}}},{\"match\":{\"market\":{\"query\":\"gcb\",\"type\":\"phrase\"}}},{\"match\":{\"content_type\":{\"query\":\"in-app\",\"type\":\"phrase\"}}}]}}}': Encountered \" \"}\" \"} \"\" at line 1, column 77.\nWas expecting one of:\n \"TO\" ...\n <RANGE_QUOTED> ...\n <RANGE_GOOP> ...\n ","caused_by":{"type":"parse_exception","reason":"Encountered \" \"}\" \"} \"\" at line 1, column 77.\nWas expecting one of:\n \"TO\" ...\n <RANGE_QUOTED> ...\n <RANGE_GOOP> ...\n "}}}}]},"status":400}>

any advice?

[thaibt]

@v01t Have you ever found a work around for that?