astro-sdk
astro-sdk copied to clipboard
Option to use qualifiers in table name in Table object
Please describe the feature you'd like to see
I'd like to be able to create a Table
object without having to map DB-specific table qualifiers (e.g. a BQ dataset) to the metadata database
and schema
fields.
Describe the solution you'd like
Option to use the qualifiers as a prefix to the table name, like as I would in a SQL query. e.g. finance.sales_quarterly
Are there any alternatives to this feature? Alternative is how it is currently done with the metadata fields. That does not need to be removed as an option.
Additional context Add any other context about the feature request here.
Acceptance Criteria
- [ ] All checks and tests in the CI should pass
- [ ] Unit tests (90% code coverage or more, once available)
- [ ] Integration tests (if the feature relates to a new database or external service)
- [ ] Example DAG
- [ ] Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
- [ ] Exception handling in case of errors
- [ ] Logging (are we exposing useful information to the user? e.g. source and destination)
- [ ] Improve the documentation (README, Sphinx, and any other relevant)
- [ ] How to use Guide for the feature (example)
I have hacked something while I had some free time, can you try putting the following in your requirements.txt and then pass a fully qualified name in the Table object:
git+https://github.com/astronomer/astro-sdk.git@fully-qualified-table-name#egg=astro-sdk-python[all]&subdirectory=python-sdk
or you don’t use docker run the following:
pip install -U git+https://github.com/astronomer/astro-sdk.git@fully-qualified-table-name#egg=astro-sdk-python[all]&subdirectory=python-sdk
@jeremybeard did you get a chance to try the workaround proposed in the above comment?
@phanikumv I haven't tried it.
@jeremybeard please feel free to open this ticket if you face this again.
@sunank200 Ok, though this wasn't for me personally, it was just a general feature request that I expect others will want.