Optional domain ARN parameter for OpenSearch sink

[graytaylor0]

Is your feature request related to a problem? Please describe.

• As a user of Amazon OpenSearch Service, I would like to configure my opensearch sink of Data Prepper with the domain ARN rather than the host endpoint.

Describe the solution you'd like The ability to pass the domain ARN to the opensearch sink plugin. Potential configuration solutions include

1. Keep the parameters the same and allow for the ARN to be passed to the hosts field. This is the easiest change to make, but it may be a slightly confusing user experience that the host is equivalent to an ARN

- opensearch:
        hosts: [ "arn:aws:es:us-west-1:987654321098:domain/test-domain/" ]
        aws_region: "us-east-1"
        aws_sts_role_arn: "role"

2. Make hosts conditionally optional based on a aws_domain_arns parameter

- opensearch:
        aws_domain_arns: [ "arn:aws:es:us-west-1:987654321098:domain/test-domain/" ]
        aws_region: "us-east-1"
        aws_sts_role_arn: "role"

[dlvenable]

Why would the aws_domain_arns be plural and a list?

I'd think:

- opensearch:
    aws_domain_arn: "arn:aws:es:us-west-1:987654321098:domain/test-domain"

Also, it could just be aws_domain and the ARN part is implied.

---

It might be a good idea to change the OpenSearch sink to use a nested configuration (perhaps for the upcoming 2.0 release). This is what we did in the S3 source.

- opensearch:
  aws:
    domain: "arn:aws:es:us-west-1:987654321098:domain/test-domain"
    region: us-east-1
    sts_role: "role"

[graytaylor0]

Nesting the configuration here makes a lot of sense. Do you think domain should be required if aws is used? Or should it still just be conditionally optional based on hosts?