Shuang comments

Results 32 comments of


                                            Shuang

[CELEBORN-1510] Partial task unable to switch to the replica

Merge to main(v060) and branch-0.5(v053) and branch-0.4(v0.4.4)

[CELEBORN-1545] Add Tez plugin skeleton and dag app master

Merge to branch: branch-tez-support

[CELEBORN-1919] Hardsplit batch tracking should be disabled when pushing only a single replica

cc @wangshengjie123

[CELEBORN-1919] Hardsplit batch tracking should be disabled when pushing only a single replica

> In which scenarios will the Batch corresponding to the PushDataRequest with HARD_SPLIT response be appended to the PartitionLocation datafile? @RexXiong Tracking of hardsplit batch can be safely disabled when...

[CELEBORN-1919] Hardsplit batch tracking should be disabled when pushing only a single replica

@FMX @wangshengjie123 Could you help to review this?

[CELEBORN-1553] Using the request base url as swagger server

> Hi @RexXiong > > If no more comments, can we merge it? thanks Thanks for the reminder. Merge to main(v0.6.0)

[CELEBORN-894] Add checksum for shuffle data

After a deep check Flink's implementation, I found that the header/data written by Flink is in big-endian format, while the header written by Spark using Platform(Unsafe) may be in little-endian...

[CELEBORN-1534] Order the API v1 response sequences filed

> it might be costly for sorting shuffle_id and application_id listing API, cc @AngersZhuuuu Perhaps we could add an option to choose between sorted and unsorted results?

[CELEBORN-1855] LifecycleManager return appshuffleId for non barrier stage when fetch fail has been reported

> > @buska88 It would be better to check the validity of a shuffle Id after you get it instead of using a shuffle ID that marked as invalid. >...

[CELEBORN-1855] LifecycleManager return appshuffleId for non barrier stage when fetch fail has been reported

Thanks @buska88 Merge to main(v0.6.0) and also backport to branch-0.5(v0.5.5)