[Improvement] Quickly delete local or HDFS data at the shuffleId level.

[yl09099]

What changes were proposed in this pull request?

At the shuffleId level, data on the local or HDFS needs to be deleted synchronously. In some scenarios, the deletion time needs to be shortened. You can rename folders and delete them asynchronously.

Why are the changes needed?

Fix: #2083

Does this PR introduce any user-facing change?

No.

How was this patch tested?

UT.

[github-actions[bot]]

Test Results

 2 966 files  ±0   2 966 suites  ±0   6h 27m 53s ⏱️ - 1m 6s  1 096 tests ±0   1 094 ✅ ±0   2 💤 ±0  0 ❌ ±0  13 735 runs  ±0  13 705 ✅ ±0  30 💤 ±0  0 ❌ ±0 

Results for commit 75aaa4f2. ± Comparison against base commit bd7c2cca.

:recycle: This comment has been updated with latest results.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 11.57895% with 168 lines in your changes missing coverage. Please review.

Project coverage is 52.60%. Comparing base (34bf686) to head (9947af7). Report is 11 commits behind head on master.

Files with missing lines	Patch %	Lines
.../handler/impl/HadoopShuffleAsyncDeleteHandler.java	0.00%	47 Missing :warning:
...storage/handler/impl/AsynDeletionEventManager.java	0.00%	45 Missing :warning:
...rage/handler/impl/LocalFileAsyncDeleteHandler.java	0.00%	30 Missing :warning:
...che/uniffle/storage/handler/AsynDeletionEvent.java	0.00%	18 Missing :warning:
...uniffle/storage/factory/ShuffleHandlerFactory.java	0.00%	8 Missing :warning:
...age/request/CreateShuffleDeleteHandlerRequest.java	0.00%	7 Missing :warning:
...pache/uniffle/server/ShuffleServerGrpcService.java	0.00%	3 Missing :warning:
.../org/apache/uniffle/server/ShuffleTaskManager.java	60.00%	2 Missing :warning:
...e/uniffle/server/storage/HadoopStorageManager.java	60.00%	1 Missing and 1 partial :warning:
...he/uniffle/server/storage/LocalStorageManager.java	66.66%	1 Missing and 1 partial :warning:
... and 2 more

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #2084      +/-   ##
============================================
+ Coverage     51.84%   52.60%   +0.75%     
- Complexity     2864     3532     +668     
============================================
  Files           469      534      +65     
  Lines         23879    29317    +5438     
  Branches       1966     2731     +765     
============================================
+ Hits          12380    15421    +3041     
- Misses        10726    12905    +2179     
- Partials        773      991     +218     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[yl09099]

Could you help share which case will need to shorten the deletion time.

During the Stage retry, delete the shuffle data block from the disk or hdfs.

[yl09099]

@zuston Help trigger the error module, I have no local error.

[yl09099]

Overwrite delete logic.

[jerqi]

@zuston Could you help me review this pull request?

[zuston]

@zuston Could you help me review this pull request?

Yes, I will review this in the later 3 days.