Junfan Zhang comments

Results 434 comments of


                                            Junfan Zhang

[FEATURE] support generating larger block size during shuffle map task by spill partial partitions data

+1.

[Umbrella] Remote merge on the shuffle server side.

This optimization also could be applied on Spark. One concern is that whether it will put more pressure on shuffle-server, like IO/cpu ?

[Feature Request] Support push metrics to Prometheus Pushgateway

The similar mechanism is supported by other systems, like Flink: https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/ I think this is a good improvement of observability

[Feature Request] Support push metrics to Prometheus Pushgateway

Let's reopen this?

Failed to load plugin obsidian-annotator

[#1398] fix(mr,tez): Make attempId computable and move it to taskAttemptId in BlockId layout.

cc @zhengchenyu could you help review this ?

[#1398] fix(mr,tez): Make attempId computable and move it to taskAttemptId in BlockId layout.

@qijiale76 Do you want to push this forward?

[Bug] Duplicated shaded-grpc jar cause task fail

Could you help fix this? @lifeSo

[#1222] feature(rust): Create mini rust server to integrate spark tests

> Hi @zuston , could you please take a look whether it is met expectation? Just take a look. Overall LGTM.

[#1554] feat(spark): Fetch dynamic client conf as early as possible

And I think the PR title is not proper for this change, which don't reflect the feature of overwriting the existing client conf.