dbt-redshift
dbt-redshift copied to clipboard
enable crossdb column metadata with ra3 nodes
resolves #736
Problem
R3-node users cannot fetch column metadata for tables & views outside of the current database.
Solution
Similar to @jiezhen-chen's approach in #428: use svv_all_columns instead of information_schema."columns".
While we're here, I have a hunch that @aiguofer's https://github.com/dbt-labs/dbt-core/pull/2754 and it's usage of pg_catalog.svv_external_columns can be dropped. Still to be verified.
Questions
- Does this fix things for RA3 node cross-db scenarios?
- Does it break things for non-RA3 scenarios?
- What is the performance impact of such a change?
- Is it correct to assume that
svv_all_columnsincludes the results ofsvv_external_columns? - To ameliorate 2 and 3, should this
svv_all_columnsonly be used if the relation-to-be-fetched'sdatabasedoes not match the one to which dbt is currently connected? (similar: https://github.com/dbt-labs/dbt-redshift/issues/179#issuecomment-1332460656)
Checklist
- [x] I have read the contributing guide and understand what's expected of me
- [ ] I have run this code in development and it appears to resolve the stated issue
- [ ] This PR includes tests, or tests are not required/relevant for this PR
- [ ] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
- Is it correct to assume that svv_all_columns includes the results of svv_external_columns?
I can confirm that svv_all_columns contains column information of external schemas as well.
This PR has been marked as Stale because it has been open with no activity as of late. If you would like the PR to remain open, please comment on the PR or else it will be closed in 7 days.
Although we are closing this PR as stale, it can still be reopened to continue development. Just add a comment to notify the maintainers.