dlt
dlt copied to clipboard
Add extra_credentials to clickhouse destination configuration
Description
Adds a new configuration option to the clickhouse destination named extra_credentials which will map to the extra_credentials argument to the s3 function. This allows dlt to use role based authentication in alignment with the clickhouse docs
Related Issues
- Fixes #2887
Additional Context
Currently using this change in our staging and production environments.
Deploy Preview for dlt-hub-docs ready!
| Name | Link |
|---|---|
| Latest commit | fc653ba7de6a965aa7a29434619a57854203da88 |
| Latest deploy log | https://app.netlify.com/projects/dlt-hub-docs/deploys/687a8337506a5a000856614d |
| Deploy Preview | https://deploy-preview-2888--dlt-hub-docs.netlify.app |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify project configuration.
Hey, thanks for this PR :) I have one note about the naming of the setting and two requests:
* Please add a simple test where you check the building of these strings depending on different settings * Please update the clickhouse docs page and include information about this new setting
Thank you for the review and for your work on the project.
I have made the requested changes which includes tests and doc updates.
Regarding the unit tests. I refactored the run method a little to make it easier to test this specific change without running a pipeline.
I gave the unit tests a good shot, but honestly got a little confused with the test project structure specifically in the loads directory so if you have advise or guidance as far as what I can do differently, I would appreciate to hear it.