Deleting Completed / Terminated Workflows Properly
Describe the solution you'd like I am looking to remove completed and terminated workflow history from MySQL persistence.
It would be handy to either set a flag on the workflow definition to flag for deletion on termination or completion, or set a deletion policy at startup to remove all completed workflow entries from their tables [Workflow, Subscription, Event, ExecutionError etc]
Describe alternatives you've considered Currently I have attempted to add a Middleware that runs in the PostWorkflow phase.
During this phase I check the workflow.CompleteTime.HasValue and queue a background job to delete workflow logs if the workflow is completed.
Additional context It seems however that if I run hundreds of workflows at once and they start to complete together, I start to get a lot of errors in QueueConsumer. It appears it is keeping some of the completed workflows still in queue.
Error executing item b6abed40-d14d-43df-ac08-02b6d0f9fbbb - Sequence contains no elements.
System.InvalidOperationException: Sequence contains no elements.
To enqueue a background job to delete pending workflows, I am using HangFire -> BackgroundJob.Enqueue.
I have tried adding a timeout using BackgroundJob.Schedule(..., 10 minutes from now); but I still experience the error.
Meet similar issues when enable deleteComplete feature when using Redis provider:
Error executing item 7aed8d0e-2c86-4ec7-a0dd-caaadb2d4d42 - Value cannot be null. (Parameter 'value')