dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Feature][resource center] Can the resource center support Tencent Cloud COS

Open kiwi-field opened this issue 1 year ago • 8 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Description

I am currently using HDFS in the resource center and I want to migrate to Tencent Cloud COS uniformly. What changes do I need to make? Tencent Cloud's bucket is public read

Use case

After switching to Tencent Cloud COS, you can perform upload, download, and preview operations in the resource center

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

kiwi-field avatar Jul 30 '24 13:07 kiwi-field

Currently there are no plans to support it.

SbloodyS avatar Jul 31 '24 00:07 SbloodyS

Can I connect to Tencent Cloud COS through the dependency of S3Utils and AWS Java SDK S3

kiwi-field avatar Jul 31 '24 01:07 kiwi-field

You can try it, but cos doesn't work well with S3.

SbloodyS avatar Jul 31 '24 01:07 SbloodyS

image The above picture shows our program configuration, and when downloading from the resource center, the following error occurred。 image

kiwi-field avatar Jul 31 '24 01:07 kiwi-field

It is obvious that AWS Java SDK S3 is designed for Amazon S3 cloud services, not the S3 protocol

kiwi-field avatar Jul 31 '24 01:07 kiwi-field

image The optional regions are shown in the above figure

kiwi-field avatar Jul 31 '24 01:07 kiwi-field

@SbloodyS

kiwi-field avatar Jul 31 '24 02:07 kiwi-field

for me ,it work

fuchanghai avatar Aug 25 '24 15:08 fuchanghai

Hi, I am currently working on this feature (#16396) to support Tencent Cloud COS as a storage plugin in DS.

Could you please assign this issue to me?

This work is due by September 30, 2024.

Mighten avatar Sep 01 '24 11:09 Mighten

Hi, I am currently working on this feature (#16396) to support Tencent Cloud COS as a storage plugin in DS.

Could you please assign this issue to me?

This work is due by September 30, 2024.

done

fuchanghai avatar Sep 02 '24 10:09 fuchanghai