Junfan Zhang comments

Results 434 comments of


                                            Junfan Zhang

Introduce data cleanup mechanism on stage level

Updated.

Introduce data cleanup mechanism on stage level

Changelog of latest commit 1. Extract the config entries of unregistering thread pool size and request timeout sec `spark.rss.client.unregister.request.timeout.sec` and `spark.rss.client.unregister.thread.pool.size`

Introduce data cleanup mechanism on stage level

Gentle ping @jerqi

Introduce data cleanup mechanism on stage level

Thanks for your review @jerqi

[Improvement] Add hdfs path health check to AppBalanceSelectStorageStrategy

> > For object storage, there may be more S3, but for cos, I don't know this very well. Do you have any good suggestions? @jerqi > > Do the...

[Improvement] Limit the speed of memory release when drop pending events

I'm curious about how to find OOM is caused by dropped events.

[Improvement] Limit the speed of memory release when drop pending events

From my understanding, if memory is not enough, it will firstly trigger GC to remove unused objects. This will stop the world. After that, when memory is still not enough,...

[WIP] Introduce the general client conf for mr/spark

Could u help check this proposal for mr/spark client conf? This POC only does the basic implementation. If this is OK, I will go ahead. @jerqi

[Feature] User's resources quota

Maybe this could be solved by implementing a custom AccessChecker to limit the users quota, I have done this.

[Feature] User's resources quota

> At present, the app level is also limited, right? Yes, I introduce a custom access checker to do following operation 1. Do grey-scale. 2. Add the blacklist for some...