kestra icon indicating copy to clipboard operation
kestra copied to clipboard

Azure storage fail to upload big files with IllegalStateException

Open loicmathieu opened this issue 4 months ago • 0 comments

Describe the issue

When trying to execute a flow with a FILE input using a large file (4MB for ex), it fails with an error java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread default-nioEventLoopGroup-1-3 with the Azure internal storage.

java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread default-nioEventLoopGroup-1-3
	at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:87)
	at reactor.core.publisher.Mono.block(Mono.java:1779)
	at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:146)
	at com.azure.storage.blob.BlobClient.uploadWithResponse(BlobClient.java:337)
	at com.azure.storage.blob.BlobClient.uploadWithResponse(BlobClient.java:298)
	at com.azure.storage.blob.BlobClient.upload(BlobClient.java:243)
	at io.kestra.storage.azure.AzureStorage.put(AzureStorage.java:199)
	at io.kestra.core.storages.StorageInterface.put(StorageInterface.java:75)
	at io.kestra.core.storages.StorageInterface.from(StorageInterface.java:96)
	at io.kestra.core.runners.FlowInputOutput.lambda$readData$2(FlowInputOutput.java:166)

Upstream issue TBD

Environment

  • Kestra Version: 0.20.0-SNAPSHOT
  • Operating System (OS/Docker/Kubernetes):
  • Java Version (if you don't run kestra in Docker):

loicmathieu avatar Oct 09 '24 12:10 loicmathieu