incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

[Improvement] When committing to a shuffle server, print its IP and port

Open rickyma opened this issue 1 year ago • 0 comments

Code of Conduct

Search before asking

  • [X] I have searched in the issues and found no similar issues.

What would you like to be improved?

Currently, we don't know which shuffle server the client is committing to.

[03:08:14:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO  org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 16201 ms
[03:08:19:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO  org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 21202 ms
[03:08:24:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO  org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 26202 ms
[03:08:29:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO  org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 31202 ms

We can add the shuffle server's IP and port into the log.

How should we improve?

No response

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

rickyma avatar Apr 02 '24 03:04 rickyma