logstash-filter-elasticsearch icon indicating copy to clipboard operation
logstash-filter-elasticsearch copied to clipboard

Backslashes in the query string not supported?

Open Dom-nik opened this issue 8 years ago • 1 comments
trafficstars

  • Version: ES/Logstash 5.0.2, plugin: 3.1.0
  • Operating System: CentOS 7
  • Config File (the query I used is in the comment):
input {
        elasticsearch {
                hosts => "10.x.y.z:9200"
                index => "data_for_mapping_test"
                query => '
{
  "query": {
    "term": {
      "_id": {
        "value": "1"
      }
    }
  }
}
'
        }
}

filter {
     elasticsearch {
              hosts => ["10.x.y.z:9200"]
              index => "mappings"
              query_template => "/etc/logstash/conf.d/mapping_test.dsl"
              # query => '{ "query": { "term": { "AssetType": "%{AssetType}"} } }'
              fields => {"AssetTypeGrouping" => "AssetTypeGroupingMapped"}
              enable_sort => false
     }
}

output {
    elasticsearch {
        hosts => ["10.x.y.z:9200"]
        index => "mappings_test"
    }
}
  • Sample Data: My input data contains a field like this: "AssetType":"\\Demo\\Something" My mappings index contains documents like this:
{
        "AssetType": "\\Demo\\Something",
        "AssetTypeGrouping": "marketing assets"
}
  • Steps to Reproduce: Run the pipe ;]
  • Error:

:error=>#<LogStash::Json::ParserError: Unrecognized character escape 'D' [...]

NOTE: It helped (worked as expected) when I added the following step in the query parsing in the plugin code:

query_tmp = event.sprintf(@query_dsl).gsub!('\\', '\\\\\\')

So it seems that the query parser doesn't understand backslashes in the query string. Is it the solution or am I doing something completely stupid?

Dom-nik avatar Dec 22 '16 19:12 Dom-nik

Sorry about the delay here :-(

Did the fix in ElasticSearch help with the issue on the Logstash side?

webmat avatar Apr 30 '18 18:04 webmat