astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Cleanup() operator - to clear old temp tables

Open utkarsharma2 opened this issue 1 year ago • 0 comments

Please describe the feature you'd like to see Currently, if the user forgets to add cleanup() operator in the dag, their dag will generate temp tables which will not be cleaned.

Describe the solution you'd like We should have the feature to clear all past temp tables as well.

Additional context This came out of a discussion with @manmeetkaur.

Acceptance Criteria

  • [ ] All checks and tests in the CI should pass
  • [ ] Unit tests (90% code coverage or more, once available)
  • [ ] Integration tests (if the feature relates to a new database or external service)
  • [ ] Example DAG
  • [ ] Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
  • [ ] Exception handling in case of errors
  • [ ] Logging (are we exposing useful information to the user? e.g. source and destination)
  • [ ] Improve the documentation (README, Sphinx, and any other relevant)
  • [ ] How to use Guide for the feature (example)

utkarsharma2 avatar Mar 28 '23 06:03 utkarsharma2