Junfan Zhang comments

Results 434 comments of


                                            Junfan Zhang

[FEATURE] Quick graceful upgrading for shuffle-server cluster

If someone has passion on this, feel free to contact me.

[FEATURE] Print out the task shuffle write time statistics by table format in log

Feel free to pick this up.

[FEATURE] Print out the task shuffle write time statistics by table format in log

Are you interested on this? @myandpr

[FEATURE] Print out the task shuffle write time statistics by table format in log

> Should we use Spark's metrics instead of log? This may be also valid for MR and Tez. BTW, the task metrics for low shuffle-server don't exist. I haven't seen...

[FEATURE] Print out the task shuffle write time statistics by table format in log

So how to meet the requirement if using metrics ? @jerqi Can you give some ideas if you want

[FEATURE] Print out the task shuffle write time statistics by table format in log

> > So how to meet the requirement if using metrics ? @jerqi Can you give some ideas if you want > > Spark metrics system allows user to add...

[FEATURE] Print out the task shuffle write time statistics by table format in log

> > > > So how to meet the requirement if using metrics ? @jerqi Can you give some ideas if you want > > > > > > >...

[DOCS] shuffle-server with low-memory best practice

Thanks for your share. I will test it in tomorrow.

[DOCS] shuffle-server with low-memory best practice

https://stackoverflow.com/questions/48798024/java-consumes-memory-more-than-xmx-argument Sometimes. the native and off-heap will occupy some memory, this is the java's problem.

[Bug] Occasionally client sent blocks to server time out.

The detailed stacktrace is missing, could you help attach more? BTW, one possible problem is the server GC. But this should be confirmed by more info.