incubator-uniffle
incubator-uniffle copied to clipboard
[Improvement] When committing to a shuffle server, print its IP and port
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Search before asking
- [X] I have searched in the issues and found no similar issues.
What would you like to be improved?
Currently, we don't know which shuffle server the client is committing to.
[03:08:14:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 16201 ms
[03:08:19:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 21202 ms
[03:08:24:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 26202 ms
[03:08:29:067] [Executor task launch worker for task 18899.0 in stage 3.0 (TID 64760)] INFO org.apache.spark.shuffle.writer.RssShuffleWriter.sendCommit:555 - Wait commit to shuffle server for task[75596] cost 31202 ms
We can add the shuffle server's IP and port into the log.
How should we improve?
No response
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!