Ravi Gawai
Ravi Gawai
@pewoInspari can you share onboarding.json and dqe for bronze tables? There are integration tests which incorporates scenario, if you take a look at this [dqe](https://github.com/databrickslabs/dlt-meta/blob/95867fd4d012c49db637a8759014fb603dff51f1/integration_tests/conf/cloudfiles-onboarding.template#L33C1-L34C1) which is mapped to [bronze_data_quality_expectations](https://github.com/databrickslabs/dlt-meta/blob/95867fd4d012c49db637a8759014fb603dff51f1/integration_tests/conf/dqe/customers/bronze_data_quality_expectations.json)....
@pewoInspari please correct your `bronze_data_quality_expectations.json `as below after removing square brackets `[]`, Rerun onboarding. After running onboarding please check your bronze dataflowspec table for `dataQualityExpectations` column if it contains below...
Once Direct publishing mode becomes PuPr then can be merged to release branch.
Use CLI to test this feature. We have added broze_silver layer option which will launch single dlt with bronze/silver layers.
Introducing new layer option as **bronze_silver** for invoking dataflowpipeline **invoke_dlt_pipeline** function. This feature will only work if customer create DLT pipelines using **Direct publishing mode feature which is currently in...
Merged this issue to feature/0.0.9 branch for release
You should see linage for silver uc table pointing to bronze. Inside DLT-META we are calling DLT APIs so should work same way if anyone would do notebook based sql...
I created branch [Issue_94](https://github.com/databrickslabs/dlt-meta/tree/issue_94) to chain bronze/silver into single DLT As of now you need to use **Direct publishing mode** which is in **Preview** channel Here is how dlt-meta config...
It shows for silver tables pointing to downstream pipelines and for bronze there is nothing upstream. If you click on linage graph you would see streaming table.
@JunHongPP If you use apply_changes in the silver layer, expectations should work! In the medallion architecture, the bronze layer is append-only, while the silver layer supports Type 1 and Type...