Agent can't send data to VPC endpoint for Kinesis Data Firehose

[rkosyk-newfire]

I have the latest version of aws-kinesis-agent on Amazon Linux 1 installed and it can successfully send data to the default public firehouse endpoint "firehose.endpoint": "firehose.us-east-1.amazonaws.com" But if I try to send data to my VPC endpoint for Kinesis Data Firehose "firehose.endpoint": "vpce-0ee422b7e0f2d75a5-c47g84kf.firehose.us-east-1.vpce.amazonaws.com" it fails with the error: [ERROR] AsyncPublisher[fh:emr_delivery_stream:/tmp/app.log*]:RecordBuffer(id=2,records=1,bytes=4) Retriable send error (com.amazonaws.services.kinesisfirehose.model.AmazonKinesisFirehoseException: Credential should be scoped to a valid region, not 'vpce'. (Service: AmazonKinesisFirehose; Status Code: 400; Error Code: InvalidSignatureException; Credential should be scoped to a valid region, not 'vpce'

Has anyone dealt with this situation before ? looking to hear about some solutions.

[thesunmin]

any workarounds?

[rkosyk-newfire]

@thesunmin As a temporary solution, I have added to the file /etc/hosts one record:

192.168.255.191 firehose.us-east-1.amazonaws.com

Now public endpoint firehose.us-east-1.amazonaws.com locally points to my VPC endpoint IP 192.168.255.191, and it works.

[thesunmin]

@rkosyk-newfire Thank you!!

[tall3n]

See Commit a44e373f5b0a7470522188b92d65cf7d050ca904

They recently just merged a fix that allows you to pass region when using a private custom dns endpoint.