fs2-kafka
fs2-kafka copied to clipboard
Spinoco
Support for SSL
Hi. I was hoping to use the library against cloudkarafka but it uses an SSL connection. It looks like the library does not support that. Is there any plan to?
https://www.cloudkarafka.com/docs-java.html
@justinhj there should not be big deal to add SSL support. In fact currently the SSL support for fs2 is pending review, once that will be done, it will be just question to plug it here and testing. I would expect this to be available in few weeks.
Awesome I'll check it out later then :)
On 11 November 2017 at 23:03, Pavel Chlupacek [email protected] wrote:
@justinhj https://github.com/justinhj there should not be big deal to add SSL support. In fact currently the SSL support for fs2 is pending review, once that will be done, it will be just question to plug it here and testing. I would expect this to be available in few weeks.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Spinoco/fs2-kafka/issues/12#issuecomment-343717818, or mute the thread https://github.com/notifications/unsubscribe-auth/AAt9o74ZMrCdxsYrkyD7QRMe50GTxLo9ks5s1pg6gaJpZM4QaoQT .
We are also looking to use SSL connection with kafka, could you please let us know if there is any update on this? Thanks :)
@ranjanibrickx hey there, with the release of fs2-crypto, this should be really easy to PR, I can get it together over the weekend if that is soon enough for you?
@AdamChlupacek - thank you so much for the response! That's great news - in the next week or so would be awesome! :) Looking forward to it.
@ranjanibrickx hey, so in theory I have a working version of this. Dont really have any test for this tho :/ Would you happen to know about an easy way (preferably docker) how to get kafka running with a SSL turned on on it? I dont see any other way than setting it all up manually, which is just bothersome.
The only kafka docker image I know of is https://github.com/wurstmeister/kafka-docker and that does not seem to support SSL
I was testing against CloudKarafka and they offer free accounts that only support SSL (no plaintext).
There is also this image but I haven't tried it. If I get chance this evening I may give it a go. I can also pull your branch and try it out.
https://github.com/trastle/docker-kafka-ssl
On 17 April 2018 at 12:56, Adam Chlupacek [email protected] wrote:
@ranjanibrickx https://github.com/ranjanibrickx hey, so in theory I have a working version of this. Dont really have any test for this tho :/ Would you happen to know about an easy way (preferably docker) how to get kafka running with a SSL turned on on it? I dont see any other way than setting it all up manually, which is just bothersome.
The only kafka docker image I know of is https://github.com/ wurstmeister/kafka-docker and that does not seem to support SSL
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Spinoco/fs2-kafka/issues/12#issuecomment-382121725, or mute the thread https://github.com/notifications/unsubscribe-auth/AAt9ox5Rs0cbK1DRc0Ah5ajsodvHt6Pmks5tpkkFgaJpZM4QaoQT .
@justinhj Oh, yeah, thanks for the link, I think I now know to integrate the SSL enabled kafka into our tests. Will get it done tonight.
@justinhj @ranjanibrickx So I have a version of something that in theory should work on feature/ssl at the moment the kafka broker and client handshake successfully but afterwards kafka refuses to unwrap any data from us. If anyone wanna get a stab at it be free to do so. (the fs2-crypto snapshot there is just a snapshot with added one fix from 0.1 brach, it should generally work with the current 0.2.0 fs2-crypto).
Awesome I’ll take a look at the weekend if I get time
On Apr 19, 2018, at 9:01 AM, Adam Chlupacek [email protected] wrote:
@justinhj @ranjanibrickx So I have a version of something that in theory should work on feature/ssl at the moment the kafka broker and client handshake successfully but afterwards kafka refuses to unwrap any data from us. If anyone wanna get a stab at it be free to do so. (the fs2-crypto snapshot there is just a snapshot with added one fix from 0.1 brach, it should generally work with the current 0.2.0 fs2-crypto).
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
If there's anything I can do to help get this in, let me know. I'd love to see this happen as we're wanting to abandon the Apache client :)
hey @felixmulder You can checkout the branch I mentioned in previous comment. Currently it is stuck at kafka replying something along: "Message cannot be parsed" inside of SSL error. It happens right after handshake at the first point kafka is receiving application data. I suspect the message format is a bit different for SSL connection, but had no luck in finding how different it is. Sadly dont have any time to spend on this at the moment.
@AdamChlupacek - looks to me like Kafka isn't responding to the handshake. The unwrap method in TLSEngine is never called, and as such the Wrap#handshakeComplete function is never called - thus it is locked in waiting for the next part of the handshake.
If it's as you say - how can I see the handshake from Kafka's side? Are you attaching yourself to the kafka docker container?