astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Option to use qualifiers in table name in Table object

Open jeremybeard opened this issue 2 years ago • 1 comments

Please describe the feature you'd like to see I'd like to be able to create a Table object without having to map DB-specific table qualifiers (e.g. a BQ dataset) to the metadata database and schema fields.

Describe the solution you'd like Option to use the qualifiers as a prefix to the table name, like as I would in a SQL query. e.g. finance.sales_quarterly

Are there any alternatives to this feature? Alternative is how it is currently done with the metadata fields. That does not need to be removed as an option.

Additional context Add any other context about the feature request here.

Acceptance Criteria

  • [ ] All checks and tests in the CI should pass
  • [ ] Unit tests (90% code coverage or more, once available)
  • [ ] Integration tests (if the feature relates to a new database or external service)
  • [ ] Example DAG
  • [ ] Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
  • [ ] Exception handling in case of errors
  • [ ] Logging (are we exposing useful information to the user? e.g. source and destination)
  • [ ] Improve the documentation (README, Sphinx, and any other relevant)
  • [ ] How to use Guide for the feature (example)

jeremybeard avatar Oct 05 '22 20:10 jeremybeard

I have hacked something while I had some free time, can you try putting the following in your requirements.txt and then pass a fully qualified name in the Table object:

git+https://github.com/astronomer/astro-sdk.git@fully-qualified-table-name#egg=astro-sdk-python[all]&subdirectory=python-sdk

or you don’t use docker run the following:

pip install -U git+https://github.com/astronomer/astro-sdk.git@fully-qualified-table-name#egg=astro-sdk-python[all]&subdirectory=python-sdk

kaxil avatar Oct 05 '22 21:10 kaxil

@jeremybeard did you get a chance to try the workaround proposed in the above comment?

phanikumv avatar Jan 16 '23 12:01 phanikumv

@phanikumv I haven't tried it.

jeremybeard avatar Jan 17 '23 14:01 jeremybeard

@jeremybeard please feel free to open this ticket if you face this again.

sunank200 avatar Mar 21 '23 13:03 sunank200

@sunank200 Ok, though this wasn't for me personally, it was just a general feature request that I expect others will want.

jeremybeard avatar Mar 21 '23 13:03 jeremybeard