apache
CamelSalesforcesourceSourceConnector issue
Apache Camel Salesforce Source Connector: v3.18.2 , kafka-clients v3.1.2
Issue: Upgrading the apache kafka salesforce source connector version from 0.11.0 to 3.18.2 and getting below error while deploying the application.
"trace":"org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context\n\tat org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:184)\n\tat org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:232)\n\tat org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)\n\tat org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:241)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: org.apache.camel.FailedToStartComponentException: Failed to start component salesforce because of loginUrl must be specified\n\tat org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:3272)\n\tat org.apache.camel.impl.engine.AbstractCamelContext.doStartContext(AbstractCamelContext.java:3033)\n\tat org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2988)\n\tat org.apache.camel.support.service.BaseService.start(BaseService.java:119)\n\tat org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2649)\n\tat org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:262)\n\tat org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)\n\tat org.apache.camel.support.service.BaseService.start(BaseService.java:119)\n\tat org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:180)\n\t... 8 more\nCaused by: java.lang.IllegalArgumentException: loginUrl must be specified\n\tat org.apache.camel.util.ObjectHelper.notNull(ObjectHelper.java:156)\n\tat org.apache.camel.component.salesforce.SalesforceLoginConfig.validate(SalesforceLoginConfig.java:237)\n\tat org.apache.camel.component.salesforce.internal.SalesforceSession.
Below is the configuration of my source connector:
{ "name": "einv-contract-sales-ipl", "config": { "connector.class": "org.apache.camel.kafkaconnector.salesforcesource.CamelSalesforcesourceSourceConnector", "camel.kamelet.salesforce-source.topicName": "subscribe:/event/IPL_Contract__e?rawPayload=true&", "transforms.tojson.salesforce.object":"IPL_Contract__e", "transforms.tojson.topics":"dv.aws-use1.sales-ipl", "topics": "${bamboo_ENV_PREFIX}.aws-use1.sales-ipl.fct.sales-contract.v1", "transforms.ValueToKey.fields":"Record_Id__c", "transforms.tojson.id.columns": "Record_Id__c", "camel.source.endpoint.backoffIncrement": "5000", "camel.source.maxPollDuration": "1000", "camel.source.maxBatchPollSize": "5000", "errors.log.include.messages": "true", "tasks.max": "1", "value.converter.schema.registry.basic.auth.user.info": "<Redacted>", "transforms": "tojson,ValueToKey", "value.converter.basic.auth.credentials.source": "USER_INFO", "value.converter": "io.confluent.connect.avro.AvroConverter", "camel.source.pollingConsumerBlockTimeout": "20000", "errors.log.enable": "true", "key.converter": "org.apache.kafka.connect.storage.StringConverter", "camel.error.handler.redelivery.delay": "5000", "transforms.ValueToKey.type": "org.apache.kafka.connect.transforms.ValueToKey", "camel.source.endpoint.fallBackReplayId": "-2", "camel.source.maxNotCommittedRecords": "5000", "errors.retry.timeout": "-1", "errors.retry.delay.max.ms": "20000", "transforms.tojson.type": "org.apache.camel.kafkaconnector.transforms.SourceJsonStringToAvroTransformPEv1", "transforms.tojson.converter.type": "value", "camel.kamelet.salesforce-source.password":"<Redacted>", "camel.kamelet.salesforce-source.replayId": "-2", "camel.kamelet.salesforce-source.loginUrl": "https://login.salesforce.com/", "camel.kamelet.salesforce-source.userName":"<Redacted>", "camel.kamelet.salesforce-source.clientId":"<Redacted>", "camel.kamelet.salesforce-source.clientSecret":"<Redacted>", "camel.kamelet.salesforce-source.eventType":"platform", "camel.kamelet.salesforce-source.query": "SELECT * FROM IPL_Contract__e", "camel.kamelet.salesforce-source.rawPayload": "true", "camel.component.salesforce.authenticationType": "USERNAME_PASSWORD", "key.converter.schemas.enable": "false", "value.converter.schema.registry.url": "<Redacted>", "value.converter.schemas.enable": "false", "camel.component.salesforce.longPollingTransportProperties.maxMessageSize": "8388608", "camel.source.pollingConsumerQueueSize": "2000", "linger.ms": "2000", "camel.component.salesforce.httpClientIdleTimeout":"30000", "camel.error.handler.max.redeliveries":5, "transforms.tojson.schema.auth.user.info":"<Redacted>", "transforms.tojson.schema.registry.url":"<Redacted>", "transforms.tojson.salesforce.username":"<Redacted>", "transforms.tojson.salesforce.passwordtoken":"<Redacted>", "transforms.tojson.salesforce.consumer.key":"<Redacted>", "transforms.tojson.salesforce.consumer.secret":"<Redacted>}", "transforms.tojson.salesforce.instance":"<Redacted>" } }
Upgrading from 0.11.0 to 3.18.2 is a big jump. I would suggest to update to version 4.x if possible or 3.20.x which is an LTS.
This is the json for configuring the connector: https://github.com/apache/camel-kafka-connector/blob/camel-kafka-connector-3.18.2/connectors/camel-salesforce-source-kafka-connector/src/generated/resources/camel-salesforce-source-source.json
@oscerd Thanks for the reply. I tried with 3.20.x . I was facing the kamelet exception with this version. Then i found that there is "operationName" property was added in v3.19.x and its causing many issue in connector. Hence I am trying with 3.18.2.
https://github.com/apache/camel-kafka-connector/issues/1587
With 4.x, i am facing Jetty version compatibility issue because we are using confluentinc/cp-kafka-connect:7.9.0 as base image version for the build of the application.
3.x is not supported, only 4.x it is, so there is no solution to the problem you see with cp-kafka-connect. I don't control confluent container image and we won't fix or release any 3.x version.