vector
vector copied to clipboard
vectordotdev
Support AWS OpenSearch Serverless for `elasticsearch` sink
AWS OpenSearch Serverless Service endpoint is aoss instead of es.
Please add a config to distinguish ES from AOSS, and fix sign_request call.
https://github.com/vectordotdev/vector/blob/c7a4d7b46a4b351f936b4ac675db3b12ece59734/src/sinks/elasticsearch/common.rs#L258
I'm not sure if there are future changes needs to be done afterwards.
@AvihaiSam Could you please test the fix from the branch - https://github.com/vectordotdev/vector/pull/16590 ? I have no ability to test it on my own right now. You need to build Vector from the provided PR and test with your setup with AWS OpenSearch Serverless.
Thanks in advance!
Hey @zamazan4ik, I have actually compiled a different version a week ago and tried it. I have noticed several things:
- the health API is not available
- the version API is not available (at this point, my guess is that all the
/_clusterAPIs are not available) - my sigv4-auth signature was wrong, using the assume-role auth-option (maybe my version made it fail, i'm no rust expert...) For the last thing - I have opened a support ticket to AWS and they are looking into it. Since this issue makes the cluster impossible to work with we have decided to delete the cluster due do its high costs. So - unfortunately, I can no longer test it.
Hope to hear from AWS and will let you know if/when I will have access to a new cluster...
I have actually compiled a different version a week ago and tried it.
What version did you compile? Does it have the same fixes for AOSS as from my branch?
@AvihaiSam 👋🏻 I have interest in getting vector working with OpenSearch Serverless; any objection to me taking what you've got in your tree and seeing what the current gaps are, and me getting it committed with you listed as primary author if I can get to success? Or do you have current plans or thoughts?
@handlerbot sure. I left it there hoping someone will find the time to work on that.
