astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Populate cols in Table Object

Open utkarsharma2 opened this issue 3 years ago • 0 comments

Please describe the feature you'd like to see We need cols values in operators like merge and append. Currently, we need to manually infer it.

Describe the solution you'd like It would be nice if we can populate Table.columns when passing table objects in load_file operator.

with DAG(...) as dag:
	test_table = Table(...)

   	load_file_task = aql.load_file(File(...), test_table)

Now test table.columns will be populated with columns. Which can be handy to use for operators that need column info like merge.

Are there any alternatives to this feature? Open to suggestions.

Additional context Slack thread - https://astronomer.slack.com/archives/C02B8SPT93K/p1663174979315829

Acceptance Criteria

  • [ ] All checks and tests in the CI should pass
  • [ ] Unit tests (90% code coverage or more, once available)
  • [ ] Integration tests (if the feature relates to a new database or external service)
  • [ ] Example DAG
  • [ ] Docstrings in reStructuredText for each of the methods, classes, functions, and module-level attributes (including Example DAG on how it should be used)
  • [ ] Exception handling in case of errors
  • [ ] Logging (are we exposing useful information to the user? e.g. source and destination)
  • [ ] Improve the documentation (README, Sphinx, and any other relevant)
  • [ ] How to use Guide for the feature (example)

utkarsharma2 avatar Sep 15 '22 14:09 utkarsharma2