dbt-dry-run
dbt-dry-run copied to clipboard
fix: dbt-external-tables partitions
Description
This update makes changes to how we handle partitions in our project. Here's what's new:
- We've added a partitions list in the ExternalConfig class. It's for keeping track of different sections of our data. By default, this list starts empty.
- There's now a quick way to see partition details in ExternalConfig through partitions_map
- We now consider partitions when we're organizing our data, making sure everything is in the right place.
Fixes #53
Checklist:
- [x] I have run
make verify
and fixed any linting or test errors - [ ] I have added appropriate unit tests or if applicable an integration test
- [ ] OPTIONAL: I have run
make integration
against a Big Query instance
Looks like it will work! Can you add a test to check the dry run schema is as expected when partitions
is used?
The test in here should be a good starting point but you will probably want to use assert_result_has_table
like this to ensure the schema is correctly predicted:
# The test should pass if no InvalidColumnSpecification exception is raised
expected_table = Table(fields=[TableField(name="column1", type="STRING"), ..., ...])
assert_result_has_table(expected_table, result)
Closing as this is stale. Would still accept a fresh PR 👍