astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Add multiple benchmarking label

Open pankajastro opened this issue 2 years ago • 0 comments

Please describe the feature you'd like to see Currently, our benchmarking job runs for all databases/datasets. We should split it in multiple job based on file size.

Describe the solution you'd like

For - CI run: Subset - 10MB and 1GB files

For the weekly and pre-release runs: Whole spectrum - all files

Acceptance Criteria

  • [ ] All checks and tests in the CI should pass
  • [ ] Unit tests (90% code coverage or more, once available)
  • [ ] Integration tests (if the feature relates to a new database or external service)
  • [ ] Example DAG
  • [ ] Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
  • [ ] Exception handling in case of errors
  • [ ] Logging (are we exposing useful information to the user? e.g. source and destination)
  • [ ] Improve the documentation (README, Sphinx, and any other relevant)
  • [ ] How to use Guide for the feature (example)

pankajastro avatar Oct 04 '22 12:10 pankajastro