astro-sdk
astro-sdk copied to clipboard
Populate cols in Table Object
Please describe the feature you'd like to see We need cols values in operators like merge and append. Currently, we need to manually infer it.
Describe the solution you'd like
It would be nice if we can populate Table.columns when passing table objects in load_file operator.
with DAG(...) as dag:
test_table = Table(...)
load_file_task = aql.load_file(File(...), test_table)
Now test table.columns will be populated with columns. Which can be handy to use for operators that need column info like merge.
Are there any alternatives to this feature? Open to suggestions.
Additional context Slack thread - https://astronomer.slack.com/archives/C02B8SPT93K/p1663174979315829
Acceptance Criteria
- [ ] All checks and tests in the CI should pass
- [ ] Unit tests (90% code coverage or more, once available)
- [ ] Integration tests (if the feature relates to a new database or external service)
- [ ] Example DAG
- [ ] Docstrings in reStructuredText for each of the methods, classes, functions, and module-level attributes (including Example DAG on how it should be used)
- [ ] Exception handling in case of errors
- [ ] Logging (are we exposing useful information to the user? e.g. source and destination)
- [ ] Improve the documentation (README, Sphinx, and any other relevant)
- [ ] How to use Guide for the feature (example)