Azure authentication erros on 3.18.x

[bsvicente]

Hi

After upgrade from 1.0.x to 3.18.x, AzureBlobStorageSink started having erros in authentication.

2022-09-28 12:51:34,273 INFO [azure-test-sink|task-0] Package versions: jackson-annotations=2.13.3, jackson-core=2.13.3, jackson-databind=2.13.3, jackson-dataformat-xml=2.13.3, jackson-datatype-jsr310=2.13.2, azure-core=1.28.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot (com.azure.core.implementation.jackson.JacksonVersion) [task-thread-azure-test-sink-0] 2022-09-28 12:51:34,779 INFO [azure-test-sink|task-0] Azure Identity => Attempted credential EnvironmentCredential is unavailable. (com.azure.identity.ChainedTokenCredential) [parallel-1] 2022-09-28 12:51:34,893 INFO [azure-test-sink|task-0] Azure Identity => Attempted credential ManagedIdentityCredential is unavailable. (com.azure.identity.ChainedTokenCredential) [parallel-1] 2022-09-28 12:51:34,980 INFO [azure-test-sink|task-0] Azure Identity => Attempted credential SharedTokenCacheCredential is unavailable. (com.azure.identity.ChainedTokenCredential) [Thread-19] 2022-09-28 12:51:34,982 INFO [azure-test-sink|task-0] Azure Identity => Attempted credential IntelliJCredential is unavailable. (com.azure.identity.ChainedTokenCredential) [Thread-19] 2022-09-28 12:51:35,026 INFO [azure-test-sink|task-0] Azure Identity => Attempted credential VisualStudioCodeCredential is unavailable. (com.azure.identity.ChainedTokenCredential) [Thread-19] 2022-09-28 12:51:35,037 INFO [azure-test-sink|task-0] Azure Identity => Attempted credential AzureCliCredential is unavailable. (com.azure.identity.ChainedTokenCredential) [Thread-19] 2022-09-28 12:51:35,043 INFO [azure-test-sink|task-0] Azure Identity => Attempted credential AzurePowerShellCredential is unavailable. (com.azure.identity.ChainedTokenCredential) [Thread-20] 2022-09-28 12:51:35,043 ERROR [azure-test-sink|task-0] Failed to acquire a new access token. (com.azure.core.implementation.AccessTokenCache) [Thread-20] 2022-09-28 12:51:35,048 ERROR [azure-test-sink|task-0] Failed delivery for (MessageId: 13B494CCCE1E39D-0000000000000000 on ExchangeId: 13B494CCCE1E39D-0000000000000000). Exhausted after delivery attempt: 1 caught: com.azure.identity.CredentialUnavailableException: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/java/identity/environmentcredential/troubleshoot ManagedIdentityCredential authentication unavailable. Connection to IMDS endpoint cannot be established. SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache. IntelliJ Authentication not available. Please log in with Azure Tools for IntelliJ plugin in the IDE. Failed to read Vs Code credentials from Linux Key Ring. AzureCliCredential authentication unavailable. Azure CLI not installed.To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/java/identity/azclicredential/troubleshoot Unable to execute PowerShell. Please make sure that it is installed in your systemTo mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azure-identity-java-default-azure-credential-troubleshoot

Message History (source location and message history is disabled)

Source ID Processor Elapsed (ms) route1/route1 955 ... azure-storage-blob-sink.kamelet.yaml:91 azure-storage-blob-sink-2/to2 azure-storage-blob://{{accountName}}/{{containerNa 0

Stacktrace

(org.apache.camel.processor.errorhandler.DefaultErrorHandler) [task-thread-azure-test-sink-0] com.azure.identity.CredentialUnavailableException: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/java/identity/environmentcredential/troubleshoot ManagedIdentityCredential authentication unavailable. Connection to IMDS endpoint cannot be established. SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache. IntelliJ Authentication not available. Please log in with Azure Tools for IntelliJ plugin in the IDE. Failed to read Vs Code credentials from Linux Key Ring. AzureCliCredential authentication unavailable. Azure CLI not installed.To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/java/identity/azclicredential/troubleshoot Unable to execute PowerShell. Please make sure that it is installed in your systemTo mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azure-identity-java-default-azure-credential-troubleshoot at com.azure.identity.ChainedTokenCredential.lambda$getToken$3(ChainedTokenCredential.java:95) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44) at reactor.core.publisher.Mono.subscribe(Mono.java:4400) at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:82) at reactor.core.publisher.MonoNext$NextSubscriber.onComplete(MonoNext.java:102) at reactor.core.publisher.FluxFlatMap$FlatMapMain.checkTerminated(FluxFlatMap.java:846) at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:608) at reactor.core.publisher.FluxFlatMap$FlatMapMain.innerComplete(FluxFlatMap.java:894) at reactor.core.publisher.FluxFlatMap$FlatMapInner.onComplete(FluxFlatMap.java:997) at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:2058) at reactor.core.publisher.Operators.complete(Operators.java:137) at reactor.core.publisher.MonoEmpty.subscribe(MonoEmpty.java:46) at reactor.core.publisher.Mono.subscribe(Mono.java:4400) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onError(FluxPeekFuseable.java:234) at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258) at reactor.core.publisher.FluxPeekFuseable$PeekConditionalSubscriber.onError(FluxPeekFuseable.java:903) at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onError(Operators.java:2063) at reactor.core.publisher.Operators.error(Operators.java:198) at reactor.core.publisher.MonoError.subscribe(MonoError.java:53) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.Mono.subscribe(Mono.java:4400) at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:82) at reactor.core.publisher.MonoNext$NextSubscriber.onComplete(MonoNext.java:102) at reactor.core.publisher.FluxFlatMap$FlatMapMain.checkTerminated(FluxFlatMap.java:846) at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:608) at reactor.core.publisher.FluxFlatMap$FlatMapMain.innerComplete(FluxFlatMap.java:894) at reactor.core.publisher.FluxFlatMap$FlatMapInner.onComplete(FluxFlatMap.java:997) at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:2058) at reactor.core.publisher.Operators.complete(Operators.java:137) at reactor.core.publisher.MonoEmpty.subscribe(MonoEmpty.java:46) at reactor.core.publisher.Mono.subscribe(Mono.java:4400) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onError(FluxDoFinally.java:136) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:172) at reactor.core.publisher.MonoCompletionStage.lambda$subscribe$0(MonoCompletionStage.java:77) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1705) at java.base/java.lang.Thread.run(Thread.java:829) Suppressed: java.lang.Exception: #block terminated with an error at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) at reactor.core.publisher.Mono.block(Mono.java:1707) at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:177) at com.azure.storage.blob.specialized.BlockBlobClient.uploadWithResponse(BlockBlobClient.java:421) at com.azure.storage.blob.specialized.BlockBlobClient.uploadWithResponse(BlockBlobClient.java:363) at org.apache.camel.component.azure.storage.blob.client.BlobClientWrapper.uploadBlockBlob(BlobClientWrapper.java:121) at org.apache.camel.component.azure.storage.blob.operations.BlobOperations.uploadBlockBlob(BlobOperations.java:188) at org.apache.camel.component.azure.storage.blob.BlobProducer.process(BlobProducer.java:89) at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172) at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:477) at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:189) at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:61) at org.apache.camel.processor.Pipeline.process(Pipeline.java:182) at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:399) at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:96) at org.apache.camel.impl.engine.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:214) at org.apache.camel.impl.engine.SharedCamelInternalProcessor$1.process(SharedCamelInternalProcessor.java:111) at org.apache.camel.impl.engine.DefaultAsyncProcessorAwaitManager.process(DefaultAsyncProcessorAwaitManager.java:83) at org.apache.camel.impl.engine.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:108) at org.apache.camel.support.cache.DefaultProducerCache.send(DefaultProducerCache.java:199) at org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:176) at org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:148) at org.apache.camel.kafkaconnector.CamelSinkTask.put(CamelSinkTask.java:205) at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:581) at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:333) at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:234) at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:203) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:188) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ... 1 more

[yuhusolutions]

This is related to the change introduced in https://issues.apache.org/jira/browse/CAMEL-18099

You can look at the doc of the Azure Storage Blob component: https://camel.apache.org/components/3.18.x/azure-storage-blob-component.html

The default behavior starting from >=3.18 is relying on AzureIdentity in alignment with understand-authorization-for-data-operation.

If you wish to use shared account key, you would need to set credentialType=SHARED_ACCOUNT_KEY.

It looks like the extra property was not mapped when the deps were bumped to 3.18.

https://github.com/apache/camel-kafka-connector/blob/main/connectors/camel-azure-storage-blob-sink-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/azurestorageblobsink/CamelAzurestorageblobsinkSinkConnectorConfig.java

https://github.com/apache/camel-kafka-connector/blob/main/connectors/camel-azure-storage-blob-source-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/azurestorageblobsource/CamelAzurestorageblobsourceSourceConnectorConfig.java

[oscerd]

This needs to be done at kamelet level

[oscerd]

This is done at Kamelet level. We should release the kamelet catalog 0.9.1 this week, then we could update the 3.18.x branch and release a 3.18.2 version.