dolphinscheduler-sdk-python icon indicating copy to clipboard operation
dolphinscheduler-sdk-python copied to clipboard

[impv] Upload the resource to the special folder instead of the Root path

Open jieguangzhou opened this issue 2 years ago • 2 comments

Now, we can upload files to the resource center, but if we cannot manage the path of the resource, there will make some errors. For example, different files have the same name.

I think we can upload the file to a special folder, such as PYDS/workflow_name/workflow_version/task_name/task_version.

But it will waste a lot of space storing the same files

jieguangzhou avatar Dec 21 '22 07:12 jieguangzhou

The best approach depends on the specific requirements of your system and workflow. But methods that we can use to eliminate duplication of files are :-File hashing, Data compression, Symbolic Links, Automatic cleanup etc.

khushi-98 avatar Jan 15 '24 13:01 khushi-98

The best approach depends on the specific requirements of your system and workflow. But methods that we can use to eliminate duplication of files are :-File hashing, Data compression, Symbolic Links, Automatic cleanup etc.

Hi @khushi-98 thanks for your reply, but we should keep the original filename for the resources file, because we may need find them in web UI in resource center page

zhongjiajie avatar Jan 16 '24 01:01 zhongjiajie