roryqi
roryqi
> > The exchange reuse you said is to share the written shuffle data for multiple downstream shuffle readers, right? Yes, it also influence the dynamic allocation. SortShuffleManager need ESS...
You can see https://github.com/apache/incubator-uniffle/blob/master/spark-patches/spark-2.4.6_dynamic_allocation_support.patch
Should we add apache license header to every file?
We have merged https://github.com/apache/incubator-uniffle-website/pull/1 . This pr is duplicated. I'll close this pr.
Will it bring any other problems?
We have a class `WriteBufferManager`. The class will control the memory that we can use. It is strange to cause OOM. I doubt whether it is caused by the issue...
> May be it is, i am not sure. I once found out an application failed due to `requestExecutorMemory` fail. Should we limit memory usage when sending data. I think...
> > > Do you set `spark.rss.data.replica.read=2` > > > > > > Yes > > > As long as the read client gets the metadata from the 2 of...
> I would be happy to review this PR, and you should avoid to fetch redundancy blocks from the another server (because the spark has consumed this blocks). Rss has...
> This will change server's memory storage to add "index" like hdfs This problem will should discuss in another issue, we also should have a simple design doc.