og-aws
og-aws copied to clipboard
open-guides
Cover Elasticsearch
AWS' own Elasticsearch service is growing in popularity. Discuss basics, when to manage yourself vs with AWS' managed service, etc.
This would probably need some comment on how AWS is lacking behind quite a bit regarding the available ES versions and the implications to get a system running securely in a VPC
@zoellner yeah, what I've heard too. That's exactly the sort of tip we should put in!
PR welcome, or if you'd like to put that tip here (any links/details to add?) we can incorporate it.
In addition to version lagging (which right now is not actually too bad, they bumped up to 2.3 I believe in July), they have scripts disabled. And there are some relatively straightforward uses that require scripts in elasticsearch, e.g. sorting an aggregation by query score is one I just ran into.
I can help here. I think Elasticsearch Service is one of the most powerful yet unknown service...as long as I ramp up my GitHub game lol ;-)
dorrin [8:22 AM]
I'd like to hear stories from people who switched from managed ElasticSearch to building and maintaining their own cluster
rjhintz [10:01 AM]
@dorrin Regarding: Elasticsearch. This was a recent topic on Reddit/aws. It seemed to draw some informed opinion (as opposed to "normal" Reddit.) https://www.reddit.com/r/aws/comments/55dpn6/elasticsearch_on_aws_managed_or_roll_your_own/
Price can also be a factor. ElasticSearch service's instances are more expensive than regular EC2 instances.
Piping in from the field here. AWS ES does not allow use of the /_cluster/state/metadata endpoint. This causes problems with supporting tools, like Elasticsearch Curator.
See https://github.com/elastic/curator/issues/717
Troubleshooting the ES service is also a problem. You can't see the instances behind the ES cluster so if you can't fix an issue by adding more memory or storage then you should contact support.
AWS has a list of ES service limits as well : http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html Mainly no scripting, plugins, heap size limit, max instance count.
Another major security gotcha to list: http://www.zdnet.com/article/first-came-mass-mongodb-ransacking-now-copycat-ransoms-hit-elasticsearch/