unstract icon indicating copy to clipboard operation
unstract copied to clipboard

feat: Added support for total files and processing status in execution

Open chandrasekharan-zipstack opened this issue 10 months ago • 1 comments

What

  • Added new field total_files to WF execution model
  • Added successful_files and failed_files fields to execution serializer
  • Changes to calculate total_files before an execution begins
  • MINOR: Checked and changed typing of workflow_execution arg in some functions

NOTE: The total_files value will not be filled for existing records but it can be handled with a data migration if required

Why

  • To provide insight on status of an execution in terms of the total files involved and the progress

How

  • Captured total files involved before an execution begins and persisted it
  • Computing progress by querying for the file execution records by its status

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

  • No, made changes with sane defaults - all additive

Database Migrations

  • 2 Schema migrations
    • For added field and minor db_comment change in WF execution model
    • For related_name change in file execution model

Related Issues or PRs

  • Based on top of #1129

Notes on Testing

  • Tried executing a WF / API deployment locally
  • Computing the values as expected for new records

Screenshots

image

Checklist

I have read and understood the Contribution Guidelines.