kestra icon indicating copy to clipboard operation
kestra copied to clipboard

feat(core): create namespace file dir when uploading new files

Open loicmathieu opened this issue 4 months ago • 1 comments

This allows to avoid having to do it when we put the file.

On a contrived example, when using ForEachItem with 1000 subflows, which means 1000 files uploaded by the split, flow execution time for MinIO goes down from 5mn to 2mn y avoiding a call to create the directory if not exist which was done systematically before.

See the following PRs on the storage: https://github.com/kestra-io/storage-s3/pull/114 https://github.com/kestra-io/storage-azure/pull/61 https://github.com/kestra-io/storage-gcs/pull/142 https://github.com/kestra-io/storage-minio/pull/93

Fixes https://github.com/kestra-io/kestra/issues/5354

loicmathieu avatar Oct 09 '24 12:10 loicmathieu