Performance Degradation in Elsa 3.2 with Workflow Versions and Instance Volume

[paramsachapara]

Performance Improvement Request

Performance Issue Overview

We are experiencing significant performance degradation in workflow execution under two specific scenarios while using Elsa Core and Elsa Designer (v3.2). This affects both development and production environments.

Technical Setup

• Elsa Version: 3.2
• Components Used: Elsa Core, Elsa Designer
• Database: SQL Server
• Workflow Structure:
  • ~60 workflows in total
  • 4–5 main workflows
  • Remaining are sub-workflows and nested sub-sub workflows
  • Frequent use of:
    • HttpEndpoint
    • Scripting
    • Decision
    • sub-workflows as activity

---

Performance Issues

1. High Number of Workflow Versions

• Problem: Workflow execution time increases to ~9 seconds as the number of versions increases due to frequent edits and redeployments.
• Temporary Workaround:
  • Export and re-import workflows to reset version numbers to 1.
  • Execution time improves to ~2–3 seconds.
• Observation: Elsa seems to evaluate all versions or perform extra logic that scales poorly with version count.

2. High Number of Workflow Instances

• Problem: When the WorkflowInstances table has more than ~800 entries, the execution time of main workflows increases to ~9 seconds.
• Temporary Workaround:
  • Deleting or truncating the WorkflowInstances table.
  • Execution time improves again to ~2–3 seconds.
• Impact: This workaround is not sustainable in a production environment where historical workflow instances are important.

Request

Please investigate these performance bottlenecks. Let us know if profiling data or further logs would help identify the root cause. We're happy to assist with more detailed testing if required.

Thank you for your hard work on Elsa – it's a great framework, and we’d love to help improve it further.

[codecov[bot]]

Codecov Report

:x: Patch coverage is 1.72414% with 57 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 29.19%. Comparing base (166d06c) to head (99eebb9). :warning: Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/aiida/tools/archive/create.py	0.00%	57 Missing :warning:

:exclamation: There is a different number of reports uploaded between BASE (166d06c) and HEAD (99eebb9). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (166d06c)	HEAD (99eebb9)
	2	1

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #6946       +/-   ##
===========================================
- Coverage   79.59%   29.19%   -50.39%     
===========================================
  Files         566      566               
  Lines       43540    43510       -30     
===========================================
- Hits        34651    12699    -21952     
- Misses       8889    30811    +21922     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mbercx]

Dogfooding:

Note: I've discovered a different issue while dogfooding, see https://github.com/aiidateam/aiida-core/issues/6982

1. Non-existing temporary directory ✅

> verdi archive create --tmp-dir /nonexistent -G workchain/PBEsol/bader/lumi bader.aiida
Usage: verdi archive create [OPTIONS] [--] OUTPUT_FILE
Try 'verdi archive create --help' for help.

Error: Invalid value for '--tmp-dir': Directory '/nonexistent' does not exist.

2. No temporary directory specified ✅

> verdi archive create -G workchain/PBEsol/scf/gauss -- test.aiida

3. Temporary directory specified ✅

> verdi archive create -G workchain/PBEsol/scf/gauss --tmp-dir ~/tmp -- test.aiida

3. Temporary directory not specified, but different parent folder for archive ✅

> verdi archive create -G workchain/PBEsol/scf/gauss  -- ~/tmp/test.aiida

I checked, and the temporary directories are indeed created in the desired locations. 👌

[danielhollas]

Falls back to output file directory when not specified -> ! This is different from the previous behavior, where the file was always in /tmp ! Thanks @GeigerJ2! Very happy to have this feature, and that the default path is now not in /tmp. I think having the parent of the filename makes a lot more sense.

Just for my curiosity, what were the problems of using /tmp?

[GeigerJ2]

Just for my curiosity, what were the problems of using /tmp?

Running out of disk space on the machine running AiiDA due to creation of the temporary archive file.

[mbercx]

@GeigerJ2 let me know when you want me to review this one again. Unless I am re-requested a review, I typically ignore open PRs that are changing.

[GeigerJ2]

@GeigerJ2 let me know when you want me to review this one again. Unless I am re-requested a review, I typically ignore open PRs that are changing.

Hi @mbercx, thank you for checking on this again. I'd like to finalize the decision if we even keep the --tmp-dir option, then add the necessary changes. Will start the discussion on slack again.