Elsa 3 WorkflowExecutionLog persistence time out

[Nokecy]

[Nokecy]

PersistActivityExecutionLogMiddleware.cs--------EFCoreActivityExecutionStore.SaveManyAsync Execution timeout @sfmskywalker

[jeanbaptistedalle]

Hi, we're also facing this issue. To add some context, our users created workflows that have to iterate on many datas (around 3000) and then do some HttpRequest for each of them so we have around 400k WorkflowExecutionLogRecords for around 150 Workflow instances. Then, when new workflows trigger, the SavesMany shown in the stacktrace search in the whole table to find missing lines that lead to a timeout as EFCore cut commands that exceed 30 seconds by default.

As a workaround, we modified the timeout, but maybe we could make some optimisation on that by avoid using the BulkUpsert and just use the BulkInsert as LogExecutionRecords are all new records ?

[sfmskywalker]

The latest version uses INSERT rather than UPSERT, so that should improve things. However, it could still be an issue if there are just too many records to insert within a reasonable timeout. Perhaps there should be a way to flush execution log records to the database during workflow execution, instead of flushing everything at the end of a burst of execution.