doris
doris copied to clipboard
[enhancement](broker) broker load support tencent cos
Signed-off-by: nextdreamblue [email protected]
Proposed changes
Issue Number: close #xxx
Problem summary
broker load support tencent cos.
Checklist(Required)
- Does it affect the original behavior:
- [ ] No
- Has unit tests been added:
- [ ] No Need
- Has document been added or modified:
- [ ] Yes
- Does it need to update dependencies:
- [ ] No
- Are there any changes that cannot be rolled back:
- [ ] No
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Why not access cos with s3?
Why not access cos with s3?
使用腾讯cos对象存储的协议去访问cos相比原生s3a协议的优势:
- 比如针对对象存储的数据读取,又进行数据的预读操作
- 针对文件路径的查询操作,在判断目录是否是目录时,有进行list-1的优化操作
- 支持了对象存储这边的融合桶支持
- 支持了ranger的访问控制协议
- 其他兼容性优化
@morningman please review, thanks
TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.75 seconds load time: 434 seconds storage size: 17151837064 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221122085621_clickbench_pr_50573.html
PR approved by at least one committer and no changes requested.
PR approved by anyone and no changes requested.
PR approved by at least one committer and no changes requested.
PR approved by at least one committer and no changes requested.