celeborn icon indicating copy to clipboard operation
celeborn copied to clipboard
verified publisher icon apache

[CELEBORN-1601] Support revise lost shuffles

Open FMX opened this issue 1 year ago • 4 comments

What changes were proposed in this pull request?

To support revising lost shuffle IDs in a long-running job such as flink batch jobs.

Why are the changes needed?

  1. To support revise lost shuffles.
  2. To add an HTTP endpoint to revise lost shuffles manually.

Does this PR introduce any user-facing change?

NO.

How was this patch tested?

Cluster tests.

FMX avatar Sep 19 '24 08:09 FMX

@FMX, could you also support the corresponding cli command for the HTTP endpoint to revise lost shuffles?

SteNicholas avatar Sep 20 '24 03:09 SteNicholas

@FMX, could you also support the corresponding cli command for the HTTP endpoint to revise lost shuffles?

Sounds good. I'll add the cli command.

FMX avatar Sep 20 '24 07:09 FMX

@FMX, BTW, the HTTP endpoint should introduce the client api to invoke, which could follow README.md to add.

SteNicholas avatar Sep 20 '24 08:09 SteNicholas

@SteNicholas Thanks. I have added the CLI command and the API endpoint. Please review this PR when you have time.

FMX avatar Sep 23 '24 08:09 FMX

This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] avatar Oct 14 '24 08:10 github-actions[bot]

Thanks. Merged to main(v0.6.0).

SteNicholas avatar Oct 21 '24 08:10 SteNicholas