incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

[FEATURE] Support Flink RSS.

Open slfan1989 opened this issue 1 year ago • 9 comments

What changes were proposed in this pull request?

I. Introduction

A. Background

  1. Limitations or issues with the current system
  2. Significance and necessity of the proposed solution

image

  • IntermediateDataSet

An intermediate data set is the data set produced by an operator - either a source or any intermediate operation.

Why are the changes needed?

(Please clarify why the changes are needed. For instance,

  1. If you propose a new API, clarify the use case for a new API.
  2. If you fix a bug, describe the bug.)

Fix: # (issue)

Does this PR introduce any user-facing change?

(Please list the user-facing changes introduced by your change, including

  1. Change in user-facing APIs.
  2. Addition or removal of property keys.)

No.

How was this patch tested?

(Please test your changes, and provide instructions on how to test it:

  1. If you add a feature or fix a bug, add a test to cover your changes.
  2. If you fix a flaky test, repeat it for many times to prove it works.)

slfan1989 avatar Dec 13 '23 03:12 slfan1989

@jerqi @zuston I started to submit the code about Flink rss. I still need to spend some time merging the local code. It will take about 2-3 days to complete the submission.

slfan1989 avatar Dec 13 '23 03:12 slfan1989

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (ecbf2e7) 53.22% compared to head (6013879) 54.11%. Report is 15 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1366      +/-   ##
============================================
+ Coverage     53.22%   54.11%   +0.89%     
- Complexity     2715     2720       +5     
============================================
  Files           418      399      -19     
  Lines         23932    21633    -2299     
  Branches       2043     2051       +8     
============================================
- Hits          12738    11707    -1031     
+ Misses        10408     9208    -1200     
+ Partials        786      718      -68     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Dec 13 '23 03:12 codecov-commenter

@chenglong-bigdata Can you help review the code?

slfan1989 avatar Dec 13 '23 12:12 slfan1989

buffer: https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/

slfan1989 avatar Dec 17 '23 09:12 slfan1989

@chenglong-bigdata Can you help review the code?

ok

chenglong-bigdata avatar Jan 03 '24 01:01 chenglong-bigdata

Is there a design doc for this one? I can help review this feature.

advancedxy avatar Jan 04 '24 02:01 advancedxy

Is there a design doc for this one? I can help review this feature.

Thank you for reviewing the code! The documentation is being improved. I will update it as soon as possible. Unit tests are still being added.

slfan1989 avatar Jan 04 '24 04:01 slfan1989

I will continue to follow up on this PR.

slfan1989 avatar Feb 05 '24 23:02 slfan1989

@chenglong-bigdata We will continue to follow up on this PR.

slfan1989 avatar Feb 28 '24 01:02 slfan1989