Not being able to configure auth without aws credentials

[GrigoreMihai]

We have a cluster which is configured to use an user and password for authentication. In the previous versions of your package we were able to use it by using the testMode constant. eg:

 pushStream( { event, endpoint: "https://" + ES_DOMAIN, testMode:true, index: "dynamodb-sources", type: "dynamo", transformFunction: modifyRecord, useBulk:true, 
        elasticSearchOptions: {auth: {
            username: ES_USER,
            password: ES_PASS
        } }   
    } )

In this commit this flag is removed: https://github.com/matrus2/dynamodb-stream-elasticsearch/commit/c99e008750520a3c3dd7d12b16c84b9da443fc2f#diff-ac1fa144d8a26929871e9bd0a2dc01736ee07d205abedd47bf638c69907fdfd7R8 .

It would be nice to have still have some flag that will allow to skip the default AWSConnection and connect using the user and password authentication.

[matrus2]

I know :) The initial purpose of this parameter was for testing.

Let me bring back this one and at the same time add one more e.g. 'esMode' as a replacement in the future.

[matrus2]

@GrigoreMihai Nevertheless, I believe you can still use elasticSearchOptions even without testMode flag as AWSConnection adds only aws specific headers to requests to elasticsearch, which are neglected if es is not AWS based. This is why all tests pass with both elasticsearch containers.

[GrigoreMihai]

@GrigoreMihai Nevertheless, I believe you can still use elasticSearchOptions even without testMode flag as AWSConnection adds only aws specific headers to requests to elasticsearch, which are neglected if es is not AWS based. This is why all tests pass with both elasticsearch containers.

In our case es is AWS based, but we still need to use the auth option as we had setup an user and password for the cluster, I think the tests are not covering this.

[matrus2]

You can pass auth option in the same fashion: https://github.com/matrus2/dynamodb-stream-elasticsearch/blob/master/src/index.js#L45 https://github.com/matrus2/dynamodb-stream-elasticsearch/blob/master/src/index.js#L53 https://github.com/matrus2/dynamodb-stream-elasticsearch/blob/master/src/utils/es-wrapper.js#L14

[GrigoreMihai]

We tried to use auth as before but with the latest version it is failing because the package is always adding the AWSConnection headers (https://github.com/matrus2/dynamodb-stream-elasticsearch/blob/master/src/utils/es-wrapper.js#L12) and the request will try to use that instead of the specified auth option resulting in some permission errors. ie. specifying the auth option without a flag to remove the default credentials will not work.

[tong-bluehill]

I got same issue. I rollback to v3.0.2. It works now.