ES 6 Support

[timwsuqld]

Elasticsearch 6 has been released. I've not yet done any testing, but will do before we upgrade our cluster.

If anyone else has tested elktail with ES6 and can report on if it works here, then we can track any required changes here.

[timwsuqld]

Initial attempts against a docker image that should have some data in it. I'll try with our testing environment sometime over the next week.

$ /usr/local/bin/elktail/elktail --url http://127.0.0.1:9200/ -v3 -i 'mysu-.*'
TRACE: elktail.go:456: Not saving query terms. Total terms: 0
TRACE: client.go:734: GET /_all/_settings HTTP/1.1
Host: 127.0.0.1:9200
User-Agent: elastic/5.0.30 (linux-amd64)
Accept: application/json
Accept-Encoding: gzip


TRACE: client.go:734: HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{".monitoring-es-6-2017.11.14":{"settings":{"index":{"codec":"best_compression","number_of_shards":"1","provided_name":".monitoring-es-6-2017.11.14","format":"6","creation_date":"1510693833105","number_of_replicas":"1","uuid":"ko-rPgWPQ9SIRANq27PSCg","version":{"created":"6000099"}}}},"mysu-2017.11.09":{"settings":{"index":{"creation_date":"1510695083123","number_of_shards":"5","number_of_replicas":"1","uuid":"SaHV0otwQIO0uggR2X-5_Q","version":{"created":"6000099"},"provided_name":"mysu-2017.11.09"}}},"mysu-2017-11-09":{"settings":{"index":{"creation_date":"1510695166465","number_of_shards":"5","number_of_replicas":"1","uuid":"6ZY2VdMqRaeddh7F5OjEQQ","version":{"created":"6000099"},"provided_name":"mysu-2017-11-09"}}},"syslog-2017.11.15":{"settings":{"index":{"creation_date":"1510694233934","number_of_shards":"5","number_of_replicas":"1","uuid":"dgt29ZT0SZ6NoW0r07K2sQ","version":{"created":"6000099"},"provided_name":"syslog-2017.11.15"}}}}
INFO: elktail.go:141: Using indices: [mysu-2017.11.09]
TRACE: elktail.go:280: Running query match all query.
TRACE: client.go:734: POST /mysu-2017.11.09/_search HTTP/1.1
Host: 127.0.0.1:9200
User-Agent: elastic/5.0.30 (linux-amd64)
Transfer-Encoding: chunked
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

1
{
55
"from":0,"query":{"match_all":{}},"size":50,"sort":[{"@timestamp":{"order":"desc"}}]}
0


ERROR: elktail.go:148: Error in executing search query. elastic: Error 400 (Bad Request): all shards failed [type=search_phase_execution_exception]

[knes1]

Oh man, just as v5 was sorted out... 😄

Anyhow - we need a v6 branch and v6 version of olivere/elastic imported in the imports. Then we'd need to figure out what breaking changes affect elktail (elktail uses really a minimal set of ES functionality, but it seems it will not work with new version anyway without doing adjustments...)

Ideally, the project should have Travis CI configured to do integration tests against new versions of ES and ease the development/testing. If there is anyone reading this who is familiar Travis CI and is willing to tackle this task, it would be great. Perhaps the setup could be borrowed from olivere/elastic, it doesn't look too compliacted: https://github.com/olivere/elastic/blob/release-branch.v6/.travis.yml