is camel-smtp connector ready for production?

[saherahwal]

Hi I have few questions regarding camel-smtp connector.

1. Where are the instructions to setup camel smtp Kafka Connect?

2. What makes it better than implementing our own consumer that uses SMTP client?

3. Is the smtp connector production ready?

[oscerd]

Hello, it's always good to start with hello so: Hello.

1. This is the way you have to test the connector locally: https://camel.apache.org/camel-kafka-connector/latest/try-it-out-locally.html and the options related to the connector are here: https://camel.apache.org/camel-kafka-connector/latest/connectors/camel-smtp-kafka-sink-connector.html for sink and https://camel.apache.org/camel-kafka-connector/latest/connectors/camel-smtp-kafka-source-connector.html for source

2. Basically it's up to you, this is just a tool to move data into a Kafka Broker or from a Kafka Broker. Writing your own smtp client can work anyway, but it will be errors prone. Under the hood the connector will use the camel-smtp component which it has been battle tested for more than 10 years.

3. I'd say no. This project is still under heavy development, so there are connectors much more integration tested than others and smtp has still to be tested extensively.

[saherahwal]

Hello oscerd Thank you so much for the detailed and very clear answers. Very much appreciated.

Thanks Saher

[valdar]

@saherahwal if I may add my 2c to what @oscerd said:

3. see for yourself, if camel-smtp-kafka-connector covers your usecase chances are, since camel-smtp-kafka-connector is based on camel-smtp, that would work well in production environment...

[saherahwal]

Thanks @valdar Unfortunately it doesn't. Our use case requires dynamic email details (subject, recipients (to, cc, bcc), body, ...etc) based on the Kafka messages coming in. It seems that the connector doesn't allow for dynamic attributes for emails.

These details are currently only placed in the config file for the connector which doesn't work with our use case.

[oscerd]

You can use still use the connector by injecting headers into your kafka record. You just need to set [email protected] and it will be picked up from the connector. You can do this with kafkacar or through a producer programatically

[saherahwal]

Hi @oscerd thanks for your quick response.

Do you have any examples on that? How do you override these config values from within

thanks

[valdar]

@saherahwal if I have understood your use case correctly you don't need to override the values, you need to send kafka events with the correct headers in the topic your sink connector listen from

[dberardo-com]

it would be great to have some more "use case" oriented documentation of this connector, or at least including a sample configuration file somewhere, but i think this is a "design choice" of camel connectors, to have developers struggle in finding their way out.

anyway, i didnt know about the kafka message header, sounds like a quite useful feature, would be nice to get a link for more documentation on that topic.

coming back to the SMTP(s) connector, i am trying to pass more custom params to enable ssltls, NTLM auth, etc, so i thought of using this param:

camel.sink.endpoint.additionalJavaMailProperties

how should properties should be passed here?

i thought of doing it this way:

"camel.component.smtps.javaMailProperties":"mail.smtp.starttls.enable=true,another.java.prop=123",

is this correct?

[oscerd]

Instead of doing useless observation like "to have developers struggle in finding their way out.", try to have a look at how the component works in camel and try to setup a working configuration.

The connectors you see in this project, are autogenerated: it is not possible to have an example for each one of them. Whenever someone ask how a connector works, or try to use a connector, we try to base an integration test or an example, on what he is doing and in that way, we test/find/write documentation and/or transforms, converters.

In this project there are 350 connectors and our time is not for free. So help, instead of complaining.

[oscerd]

If you don't like the approach or you prefer something else, there are alternatives and/or commercial offerings you could look at.

[dberardo-com]

allow us some frustration, i think it is still a valuable feedback (sure i will not repeat myself in this and try to be objective form now on).

anyway, i was trying to figure out the required params by looking at the source code but i am not able to find it, do you have any hint for me where i can find the source files of specific connectors?

[oscerd]

The source code of components is here: https://github.com/apache/camel/tree/master/components

For each component there is a corresponding connector.

[energenious-eu]

thanks for the link, unfortunately i cannot find camel-smtp in the list, can that be nested somewhere in the directory?

[oscerd]

It's in camel-mail https://github.com/apache/camel/blob/master/components/camel-mail/

[energenious-eu]

ok, so i assume that camel-mail is a more recent version of camel-smtp for Camel 1.0. Please correct me if i am wrong.

I find the examples in the doc inside the component useful. I will have a look and see that they match with the actual smtp(s) components specifications.

will get back in this chat in case issues arise.

[oscerd]

camel-mail contains support for different protocol, it was available starting from camel 1.0, but camel-kafka-connector is using camel 3.8.0 as base version for autogenerating the connectors.

So the version you should look at is 3.8.0 and ckc at version 0.8.0

[energenious-eu]

ok, my confusion arose from the fact that i was initially trying to look for the mail connector in this list: https://camel.apache.org/camel-kafka-connector/latest/connectors.html but could not find any, so i thought there was a versioning mismatch between the smtp(s) connectors and the mail one.

Is there a download link for the .jar of the connector, so it can be used with regular kafka connect?

[oscerd]

is the camel-smtp connector. https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-smtp-kafka-connector/0.8.0/camel-smtp-kafka-connector-0.8.0-package.tar.gz

here https://camel.apache.org/camel-kafka-connector/latest/connectors.html

[oscerd]

is the camel-smtp connector. https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-smtp-kafka-connector/0.8.0/camel-smtp-kafka-connector-0.8.0-package.tar.gz

here https://camel.apache.org/camel-kafka-connector/latest/connectors.html

[energenious-eu]

ah ok, so they are the same connector?

but if the mail.connector unifies protocols, that one is more interesting. Is there a .jar release of the mail connector available for download?

[oscerd]

The connector is a thing, the component is a different thing. The connector is created for working on kafka connect, the component is created for running on multiple runtimes (plain, spring boot, karaf, quarkus (through extension), kubernetes/openshift), but not on kafka-connect.

There isn't a connector for kafka-connect including all the protocols in one single package.

[energenious-eu]

can i still refer to the documentation for the mail component in order to understand the one for smtp connector? or are they very different?

[oscerd]

The options are mapped one to one in the connector, so I'd say yes.

If you have time or want to help, you can add an example here: https://github.com/apache/camel-kafka-connector-examples

if you don't have time, even reporting problem here would be helpful. Thank you.

[energenious-eu]

i am trying to fix a smtps connection problem of the connector. Once i am over with it i could share my solution in an example.

for that i will just create a folder with the connector's name (smtp) and examples in it.