No support for http proxies

[samcday]

I can't see any mention of http proxying anywhere in the Java bindings or in the underlying C++ code.

What I've gathered from a cursory glance is that the HTTP requests to Kinesis are happening in C++. I definitely don't feel confident enough to roll up my sleeves and crank out a PR to add support for http proxies in there ;)

[kevincdeng]

Hi Sam,

You can try using the customEndpoint option to divert the requests to a different place.

Unfortunately there's no way to disable TLS at the moment, so this might not work with a standard HTTP proxy. You can however disable certificate checking with the verifyCertificate option.

[samcday]

Using a different endpoint is a possible workaround yes, but I'll keep this ticket open as a request to support "proper" proxying through something like Squid (using CONNECT for TLS socket passthrough connections etc)

[pfifer]

With the 0.12.x version, which uses the AWS C++ SDK and Curl, proxies should work. I will need to investigate how you would pass proxy configuration to the native process. This may still require some change to the configuration for the native process.

For everyone if this is affecting you please comment, or add a reaction to assist us in prioritizing the change.

Thanks.

[x418]

Hi,

We are evaluating the usage of KPL & KCL for Kinesis stream processing. All our services access aws resources via an internal proxy. It would be great if you could update this issue about passing on proxy configuration to the native process. Right now, we see the following exceptions in the logs

2017-08-02 17:47:30,626 [kpl-daemon-0003] [trid=, url=] logLevel=ERROR com.amazonaws.services.kinesis.producer.LogInputStreamReader - [2017-08-02 17:47:30.626540] [0x00005332][0x00007f4170dfa700] [error] [AWS Log: ERROR](CurlHttpClient)Curl returned error code 56

Is it possibly related ?

[x418]

A quick note to say that while trying to test KPL's buffering, I tried capturing the requests via a proxy and found that it infact doesn't use the proxy.

We would love to have the ability to pass in the proxy config to the native process.

KCL on the other hand, does use the proxy.

[kunickiaj]

Still hoping this gets done sooner rather than later.

[Pyons]

Any news on proxy support for KPL?

[pfifer]

I really need to tell people to place reactions on the top post. GitHub's issue sorting only appears to work if the reactions are on the initial post.

Curl, which is used by the C++ SDK, does support proxies so it should be possible to add.

I don't have an ETA for this, but I'm looking into the priority of it.

[DracoBlue]

We fixed it in our setup with an environment variable for the socks proxy like this:

export ALL_PROXY=socks5://proxy.example.org:1080

[conilas]

Any updates on the PR trying to solve it?

[kabennett]

Would the utilization of a Kinesis VPC Endpoint be a way to work around this limitation? I am currently experiencing a similar issue where I can't connect to the service endpoint via a proxy configuration.

[jdcaperon]

This can be closed with https://github.com/awslabs/amazon-kinesis-producer/pull/351?