kestra
kestra copied to clipboard
feat(core): create namespace file dir when uploading new files
This allows to avoid having to do it when we put the file.
On a contrived example, when using ForEachItem with 1000 subflows, which means 1000 files uploaded by the split, flow execution time for MinIO goes down from 5mn to 2mn y avoiding a call to create the directory if not exist which was done systematically before.
See the following PRs on the storage: https://github.com/kestra-io/storage-s3/pull/114 https://github.com/kestra-io/storage-azure/pull/61 https://github.com/kestra-io/storage-gcs/pull/142 https://github.com/kestra-io/storage-minio/pull/93
Fixes https://github.com/kestra-io/kestra/issues/5354