alluxio icon indicating copy to clipboard operation
alluxio copied to clipboard

Skip duplicated persist request

Open StephenRi opened this issue 2 years ago • 2 comments

What changes are proposed in this pull request?

Skip the duplicated persist request, if there is a persist request is being processed.

Design:

  1. Use a Set to maintain all the file ids that is being processed
  2. Call Set.add at the entrance of the state machine of persist
  3. Call Set.remove when the final state of the state machine is reached

Why are the changes needed?

  1. Simplify the state machine of Persist
  2. Reduce data uploaded to ufs and save bandwidth

Does this PR introduce any user facing changes?

None

#16511

StephenRi avatar Nov 09 '22 09:11 StephenRi