temporal icon indicating copy to clipboard operation
temporal copied to clipboard

Automatically split history event batches when size of reapplied events are too large

Open yycptt opened this issue 11 months ago • 0 comments

Is your feature request related to a problem? Please describe. When doing reset, signals and updates after the reset point will be reapplied (cherry-picked) to the new run. However all those reapplied events are grouped into one batch. Our persistence layer has a validation that basically says each event batch can't be exceeded 4MB size limit (each batch is a separate call to persistence). This means if the size of reapplied events is larger than 4MB, the reset can't be done.

This issue from my understanding only applies to reset, where events from more than one event batches in the base workflow can be picked. The events reapply logic during conflict resolution is triggered by the replication task of a single batch of event, so we won't run into the situation.

Describe the solution you'd like Reset with more than 4MB reapplied events should be supported.

Approach 1:

  • In history builder code, automatically track current event batch size and create a new batch if size limit is exceeded.

Approach 2:

  • Make this specific to reset logic. Track the size of reapplied events and call an method on history build to create a new batch.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

yycptt avatar Mar 05 '24 22:03 yycptt