astronomer-cosmos
astronomer-cosmos copied to clipboard
Select models which have tests with additional dependencies
Description
Tests such as the relationship test will fail to be selected if we are selecting models with tags and using the manifest loading method.
Related Issue(s)
closes #719
Breaking Change?
Almost all tests will have only one 'depends_on', but when they have two, ie. relationship
, the true parent is the last one.
load_from_dbt_manifest
now considers the primary selectable parent model of a test to be the last item in its 'depends_on' list, rather than the first.
This would break any tests being loaded by tag selection using load_from_dbt_manifest
which somehow depended on a model being the first in that list rather than the last.
Checklist
- [ ] I have made corresponding changes to the documentation (if required)
- [x] I have added tests that prove my fix is effective or that my feature works
Deploy Preview for amazing-pothos-a3bca0 ready!
Name | Link |
---|---|
Latest commit | 213620fccdf33fe05375f65cde17bf350eb1f20e |
Latest deploy log | https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/657af27701122100082a33fc |
Deploy Preview | https://deploy-preview-751--amazing-pothos-a3bca0.netlify.app |
Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify site configuration.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Comparison is base (
82e8db9
) 93.27% compared to head (6c4bfe4
) 93.28%.
Additional details and impacted files
@@ Coverage Diff @@
## main #751 +/- ##
==========================================
+ Coverage 93.27% 93.28% +0.01%
==========================================
Files 55 55
Lines 2499 2503 +4
==========================================
+ Hits 2331 2335 +4
Misses 168 168
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
@adammarples as you're working on this, I'd like to bring to your attention a case you're missing, and which we're running into recently:
File "/home/airflow/.local/lib/python3.11/site-packages/cosmos/dbt/selector.py", line 298, in _should_include_node
node.tags = getattr(self.nodes.get(node.depends_on[0]), "tags", [])
~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
Although not a dbt best practice, it's entirely possible to have a test with 0 dependencies, or model dependencies (test on sources)
Hi @adammarples , this PR - and the discussion - was beneficial. Thank you. And thanks, @jbandoro, for giving an alternative solution to the problem.
Cosmos can improve its implementation towards tests with multiple parents - we should make it consistent with dbt. @adammarples, you make a great start - would you be interested in further contributing to this - or would you rather close this PR and let someone else take over the work?
@ms32035 I didn't realise that was possible! Although it relates to the original problem, I logged a separate ticket: #782. Would you be interested in contributing?
Closing this PR in favour of the ticket: https://github.com/astronomer/astronomer-cosmos/issues/978