sofa-jraft icon indicating copy to clipboard operation
sofa-jraft copied to clipboard

RFC: multi-generation log cache

Open killme2008 opened this issue 2 years ago • 9 comments

RFC: multi-generation log cache

As discussion in #840 , jraft keeps a cache for latest log entries, it's unlimited and may consume too much memory under write-heavy workload or catching up replication when node restarts.

I think we can abstracts a cache interface and provide a multi-generation cache to replace current one:

  • Gen-1: totaly in memory with limited capacity.
  • Gen-2: Off-heap cache.When Gen-1 overflows, move the LRU(least recently used) item into off-heap cache such as OHC.

Main goals:

  • Abstracts log cache interface.
  • Provide a multi-generation cache implementation.

killme2008 avatar Jun 23 '22 03:06 killme2008

@killme2008 hi,I have some Off-heap cache experience before ,and I accept issue #856 ,but the programme wait to discuss,so I want to do this first,and I have time do them together,can you assign me?

sollhui avatar Jul 14 '22 02:07 sollhui

@HHoflittlefish777 Of course, it's yours, feel free to try and discuss with us in slack.

killme2008 avatar Jul 14 '22 02:07 killme2008

@HHoflittlefish777 Of course, it's yours, feel free to try and discuss with us in slack.

Thanks ,I will discuss in slack later

sollhui avatar Jul 14 '22 03:07 sollhui

@killme2008 Hi,I ask some question in slack yesterday,hope to be given a answer if have time.

sollhui avatar Jul 16 '22 09:07 sollhui

@HHoflittlefish777 do not have time to impl this feature, it's open for any developers that are interested in it.

killme2008 avatar Jul 27 '22 02:07 killme2008

@killme2008 @hzh0425 Hello, I'm learning MIT 6.824 Course now, and I've passed the Lab2 in which I implemented Raft. I am not so familiar with JVM. But whatever, I want to have a try. Can you assign me?

RockRockWhite avatar Sep 18 '22 13:09 RockRockWhite

@killme2008 @hzh0425 Hello, I'm learning MIT 6.824 Course now, and I've passed the Lab2 in which I implemented Raft. I am not so familiar with JVM. But whatever, I want to have a try. Can you assign me?

Sure, already assigned to you @RockRockWhite

hzh0425 avatar Sep 19 '22 02:09 hzh0425

@RockRockWhite You can submit a RFC first

hzh0425 avatar Sep 19 '22 03:09 hzh0425

@RockRockWhite May I ask this task is still being implemented?

shihuili1218 avatar Apr 25 '23 10:04 shihuili1218