unstract icon indicating copy to clipboard operation
unstract copied to clipboard

feat: UN-1983 BE API changes for file centric logging

Open chandrasekharan-zipstack opened this issue 10 months ago • 2 comments

What

  • Updated execution time in WF and file execution models

  • Added data migrations for older records

  • MINOR: Storing enum.value in file execution table

  • From #1131

    • Added new field total_files to WF execution model
    • Added successful_files and failed_files fields to execution serializer
    • Changes to calculate total_files before an execution begins
    • MINOR: Checked and changed typing of workflow_execution arg in some functions
    • NOTE: The total_files value will not be filled for existing records but it can be handled with a data migration if required
  • From #1132

    • Updated status field for below models to use TextChoices correctly
      • WorkflowExecution
      • WorkflowFileExecution
      • FileHistory
    • Updated usage of this enum across backend
    • MINOR: Fixed bug with log_level for latest_log` while listing files
    • Added retrieve API for executions to obtain by execution ID

Why

  • Previously was not calculated, need it for file centric logging enhancements

How

  • Update execution time whenever the status is one of ERROR / COMPLETED / STOPPED

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

  • No, updated a previously empty value

Database Migrations

  • WorkflowFileExecution - data migration to
    • rename statuses correctly
    • update execution time as modified_at - created_at
  • WorkflowExecution - data migration to
    • update execution time as modified_at - created_at

Related Issues or PRs

  • UN-1983
  • Merged PRs into this branch
    • #1131
    • #1132

Notes on Testing

  • Tried executing and noticed these values were updated correctly
  • Migration ran without issues

Screenshots

  1. After migration with updated times and statuses image

  2. From #1131 image

Checklist

I have read and understood the Contribution Guidelines.

filepath function $$\textcolor{#23d18b}{\tt{passed}}$$ SUBTOTAL
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_logs}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_cleanup}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_cleanup\_skip}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_client\_init}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_image\_exists}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_image}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_container\_run\_config}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_container\_run\_config\_without\_mount}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_run\_container}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$ $$\textcolor{#23d18b}{\tt{9}}$$ $$\textcolor{#23d18b}{\tt{9}}$$

github-actions[bot] avatar Feb 19 '25 05:02 github-actions[bot]