elsa-core icon indicating copy to clipboard operation
elsa-core copied to clipboard

[ENH] Optimize Workflow Import by Storing Raw JSON to Avoid Unnecessary Serialization

Open sfmskywalker opened this issue 8 months ago • 0 comments

Problem Statement

Currently, importing workflows from JSON involves deserializing the JSON into a Workflow object, which is then serialized again into JSON for storage in the database. This process is inefficient and introduces complex deserialization logic, leading to several issues:

  1. Activity Type Resolution: During deserialization, referenced activity types must be resolved from the Activity Registry. If a referenced activity type doesn't exist, a "NotFoundActivity" object is instantiated, retaining the activity state from the JSON file for future deserialization when the activity type is available.

  2. Interdependencies Between Workflows: Workflow definitions can be used as activities. When importing multiple workflow JSON files in bulk, a workflow might reference an activity whose type is based on another workflow definition not yet imported, necessitating the instantiation of a "NotFoundActivity".

  3. Double Import Phase: Due to the above issues, the import phase must be executed twice—first to load all workflows into the system and then to replace any "NotFound" activities with actual activities based on workflow definitions.

Proposed Solution

Instead of deserializing and then serializing workflows during import, I propose directly storing the raw JSON content into the database. This approach has several benefits:

  • Performance Improvement: Eliminates unnecessary serialization, reducing processing time and resource consumption.
  • Simplified Logic: Avoids the need for complex deserialization logic and double import phases, simplifying the import process.
  • Code Cleanliness and Maintainability: Removing the serialization-deserialization cycle enhances code readability and maintainability.

By implementing this solution, we can ensure a more efficient and maintainable workflow import process.

sfmskywalker avatar Jun 08 '24 16:06 sfmskywalker