dbt-core icon indicating copy to clipboard operation
dbt-core copied to clipboard

Support materialized views in `get_columns_in_relation`

Open dbeatty10 opened this issue 1 year ago • 1 comments

resolves #9419

[!IMPORTANT]
The contents of this PR should be compared with https://github.com/dbt-labs/dbt-core/pull/7459 since they both touch on the same code and could easily overwrite each other. Ideally, both of them will functional tests so that one doesn't undo the other by accident.

Problem

As described in https://github.com/dbt-labs/dbt-core/issues/9419#issuecomment-1907067697, the information_schema.columns view in postgres includes these:

  • o[r]dinary table, [v]iew, [f]oreign table, [p]artitioned table

But it does not include:

  • [m]aterialized view

Solution

Two things:

  1. https://github.com/dbt-labs/dbt-core/pull/9433/commits/531446ecb1679c2827e39047a3d3b9079f79f56b Add the full definition of the information_schema.columns view into dbt-postgres.
  2. https://github.com/dbt-labs/dbt-core/pull/9433/commits/afaca6316e4cbf4051a7cf365245f50dc7c964dc Add 'm'::"char" to the definition so that MVs are included.

Definition of the information_schema.columns view

To get the full text definition of this view, run the following:

select definition 
from pg_catalog.pg_views 
where schemaname = 'information_schema' 
  and viewname = 'columns'

The output is a fairly gnarly query as seen in https://github.com/dbt-labs/dbt-core/pull/9433/commits/531446ecb1679c2827e39047a3d3b9079f79f56b

To do

After a quick search, I couldn't find any relevant tests for adapter.get_columns_in_relation, so we'll probably want some that cover all the different types of relations:

  • ephemeral
  • table
  • view
  • materialized view

The tests would basically just need to:

  1. Define a relation with a column or two.
  2. Build it.
  3. Make sure that those columns are returned by adapter.get_columns_in_relation.

I'm assuming that ephemeral will either raise an exception or return nothing, but we'll want to add applicable tests either way.

Checklist

  • [x] I have read the contributing guide and understand what's expected of me
  • [x] I have run this code in development and it appears to resolve the stated issue
  • [ ] :point_left: This PR includes tests, or tests are not required/relevant for this PR
  • [x] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc)

dbeatty10 avatar Jan 24 '24 02:01 dbeatty10

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (2f2e0ce) 87.85% compared to head (01de40f) 87.83%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9433      +/-   ##
==========================================
- Coverage   87.85%   87.83%   -0.03%     
==========================================
  Files         150      150              
  Lines       21915    21915              
==========================================
- Hits        19253    19248       -5     
- Misses       2662     2667       +5     
Flag Coverage Δ
integration 85.28% <ø> (-0.03%) :arrow_down:
unit 62.39% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jan 24 '24 02:01 codecov[bot]