akka-persistence-cassandra icon indicating copy to clipboard operation
akka-persistence-cassandra copied to clipboard

Published 20 hours ago • verified publisher icon akka

Slow clean tagged events and overload Cassandra by overfetch insufficient data for deleteFromTagView

Open Valocop opened this issue 8 months ago • 5 comments

https://github.com/akka/akka-persistence-cassandra/blob/8006539934ff77bf72b1a8062a44478686112945/core/src/main/scala/akka/persistence/cassandra/reconciler/DeleteTagViewForPersistenceId.scala#L40

The problem is that when we try to clean events by tags, Cassandra Journal runs a stream with current events by tag and over fetch data by fetching event payload for every event. It takes a lot of time to clean tag_views. Better make stream that will be fetch events without payload, these data will be enough for deleteFromTagView.

I faced this problem when cleaning events in tag with a lot of events (millions) and the payload is a big.

Can we fix it, please? Thanks

Valocop avatar May 31 '24 15:05 Valocop