roryqi comments

Results 401 comments of


                                            roryqi

[Improvement] Can we support AE of SPARK2.4

AE of Spark 2.4 isn't a complete feature. We shouldn't support it.

[Improvement] Read shuffle data fail because read index file fail

We recommend users to use StorageType MEMORY_LOCALFILE_HDFS or MEMORY_LOCALFILE. The application won't commit data.

[Improvement] Read shuffle data fail because read index file fail

> > We recommend users to use StorageType MEMORY_LOCALFILE_HDFS or MEMORY_LOCALFILE. The application won't commit data. > > So, should we remove `LOCALFILE`? It's useful for test code. And if...

404 Not Found

We have built the website now. You can click the link again and see it. I will close the issue.

[Improvement] Avoid unnecessary retry in some access checker when using DelegationShuffleManager

@smallzhongfeng What do you think?

[Improvement] Avoid unnecessary retry in some access checker when using DelegationShuffleManager

A little complex, are there similar mechanisms in the other systems? In my opinion, we shouldn't retry when we use candidate checker. when we only use health checker, we can...

[Improvement] Avoid unnecessary retry in some access checker when using DelegationShuffleManager

> If not having this mechanism, how to handle the multiple checkers retry? > > In our internal env, we will use the multiple checkers, including health checker(need to retry)...

[Improvement] Avoid unnecessary retry in some access checker when using DelegationShuffleManager

> So do we need this feature ? cc @jerqi I don't think we need so complex retry mechanism ...

[Improvement] Disallow sendShuffleData if requireBufferId expired

Do you have the solution of the problem?

[Improvement] Disallow sendShuffleData if requireBufferId expired

Could you share your solution? We can discuss first.