dlt-meta icon indicating copy to clipboard operation
dlt-meta copied to clipboard

Allow to specify source database instead of defaulting to bronze database in __get_silver_dataflow_spec_dataframe

Open rtdtwo opened this issue 1 year ago • 2 comments

Issue

The FAQs describes that any data source can be used for silver tables. However, the code always tries to find the bronze database from the onboarding rows, and there's no explicit mention of a source database other than a bronze database.

Fix

Change the code to accomplish the following:

  • Use bronze_database_{env} if the field is found in onboarding rows.
  • If not, find and use source_details.

rtdtwo avatar Jan 22 '24 15:01 rtdtwo

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (2a93dd9) 90.28% compared to head (f4fbd2b) 89.95%.

Files Patch % Lines
src/onboard_dataflowspec.py 40.00% 1 Missing and 2 partials :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #34      +/-   ##
==========================================
- Coverage   90.28%   89.95%   -0.34%     
==========================================
  Files           8        8              
  Lines         803      806       +3     
  Branches      149      151       +2     
==========================================
  Hits          725      725              
- Misses         31       32       +1     
- Partials       47       49       +2     
Flag Coverage Δ
unittests 89.95% <40.00%> (-0.34%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jan 22 '24 23:01 codecov[bot]

@rtdtwo, Needs to add unit tests for added new lines as per above code coverage report.

ravi-databricks avatar Jan 22 '24 23:01 ravi-databricks

Closing this PR since its very old and author did not followup on review changes.

ravi-databricks avatar Dec 04 '24 20:12 ravi-databricks