ozone icon indicating copy to clipboard operation
ozone copied to clipboard

Published 20 hours ago • verified publisher icon apache

HDDS-5869. [Ozone-Streaming] Added support for stream on S3Gateway wr…

Open guohao-rosicky opened this issue 3 years ago • 11 comments

What changes were proposed in this pull request?

Added support for stream on S3Gateway write path

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-5869

guohao-rosicky avatar Nov 22 '21 07:11 guohao-rosicky

hi @szetszwo @captainzmc , can you help review this patch?

guohao-rosicky avatar Nov 22 '21 07:11 guohao-rosicky

@guohao-rosicky , the compilation failed. Please take a looks. Thanks.

szetszwo avatar Nov 29 '21 12:11 szetszwo

@szetszwo This PR is implemented using buffer copy, and then I'll turn on another PR to optimize for zero copy

https://issues.apache.org/jira/browse/HDDS-6133

guohao-rosicky avatar Dec 22 '21 08:12 guohao-rosicky

Added the enable Stream configuration on the Gateway side.

Please take a look. @szetszwo

guohao-rosicky avatar Dec 22 '21 11:12 guohao-rosicky

CI that fails is successful in personal branch. https://github.com/guohao-rosicky/ozone/actions/runs/2154613367 Hi @szetszwo Could you help review this PR again?

captainzmc avatar Apr 12 '22 14:04 captainzmc

Thanks @guohao-rosicky update this. @szetszwo Can you help take another look?

captainzmc avatar Jun 24 '22 06:06 captainzmc

Are the crypto changes related to S3Gateway? Could we separated it out since this is very big?

Hi, @szetszwo I can work this in https://issues.apache.org/jira/browse/HDDS-5892

guohao-rosicky avatar Jun 28 '22 02:06 guohao-rosicky

@guohao-rosicky , @captainzmc , this pull request is around for quite a long time. We should plan about how to get this committed. One problem is that this involves too much code change. Let's separate the individual methods put(..) and createMultipartKey(..) into different pull requests.

Also, we should refactor the ObjectEndpoint code in master first. Otherwise, merging becomes a pain. Let me work on the refactoring.

szetszwo avatar Jun 29 '22 19:06 szetszwo

Filed https://issues.apache.org/jira/browse/HDDS-6973 for the refactoring.

szetszwo avatar Jun 29 '22 20:06 szetszwo

Hi @szetszwo https://github.com/apache/ozone/pull/3569 has been merged into master, please rebase HDDS-4454, I will modify this PR later, thanks.

guohao-rosicky avatar Jun 30 '22 07:06 guohao-rosicky

Hi @szetszwo #3569 has been merged into master, please rebase HDDS-4454, I will modify this PR later, thanks.

Hi @guohao-rosicky, I just had update HDDS-4454, you can continue this.

captainzmc avatar Jul 04 '22 12:07 captainzmc

@szetszwo should this PR be revisited?

kerneltime avatar Nov 14 '22 17:11 kerneltime