Junfan Zhang comments

Results 434 comments of


                                            Junfan Zhang

fix(spark): add all faulty servers into blacklist while writing for stage recomputing

Could you help review this? @yl09099

fix(spark): add all faulty servers into blacklist while writing for stage recomputing

> Do you add a new ut for this case? I'm still reviewing this part.

[#1711] feat(server): introduce the reconfigurable conf

I drafted this impl, please take a look. @jerqi

[#1711] feat(server): introduce the reconfigurable conf

> I drafted this impl, please take a look. @jerqi gentle ping @jerqi

[#1711] feat(server): introduce the reconfigurable conf

> > I drafted this impl, please take a look. @jerqi > > gentle ping @jerqi ping again

[#1658] fix(server): FileSegmentManagedBuffer#nioByteBuffer read result cacheable

Could you help review this? @rickyma

[#1658] fix(server): FileSegmentManagedBuffer#nioByteBuffer read result cacheable

> Left some comments. I'm OK with this patch. But you need to test it more carefully. Because this is used in gRPC mode. It won't be invoked in Netty...

[#1239] Remote merge on the shuffle server side.

Thanks for proposing this. I have a question that I remember spark will sort the data using the sortExec in dataframe api. That means the sort will not ocurr on...

[#1239] Remote merge on the shuffle server side.

> Does code add here come from Apache Spark? Detailed design could be found on #1239, the initial motivation is to reduce the disk usage of spill, especially on the...

[#1239] Remote merge on the shuffle server side.

> For this particular feature, I think MR/Tez would benefit from that substantially(to be frankly, I had developed a similar shuffle system back ~6-7 years ago). The spark integration is...