dbt-data-reliability icon indicating copy to clipboard operation
dbt-data-reliability copied to clipboard

[ELE-47] feature: allow glue adapter

Open Jeremynadal33 opened this issue 1 year ago • 2 comments

Partially resolves ELE-47

Description

This adds the needed macros to run both the dbt-data-reliability logging process in a dbt-glue set up and the macros needed for by the Elementary CLI.

It uses the Delta file format by default and therefore needs a glue environment that can use Delta. I add a new issue for other file formats.

However, since we use Delta Lake file format, we need to use a specific Glue Session :

Can anyone tell me how I could add disclaimers about it ?

FYI : I am sure this PR needs testing but I don't know how to do them, I would be grateful for advises on that 🙏

Tests that were run locally

Jeremynadal33 avatar Oct 01 '24 11:10 Jeremynadal33

Hi @Jeremynadal33 !

I haven't had the chance to review the PR yet, but I wanted to comment on testing. Please see the instructions here - you should see here how to run our integration tests with py.test.

Please let me know if you have any questions.

Cheers, Itamar

haritamar avatar Oct 02 '24 09:10 haritamar

Hi there, Thanks a lot for your reactivity! I actually saw the test suit after submitting the PR... Haha. We are currently testing it and will get back to it as soon as we are done !

Thanks again

Jeremynadal33 avatar Oct 02 '24 09:10 Jeremynadal33

closing due to inactivity. @Jeremynadal33 feel free to reach out if this is still of interest to you

arbiv avatar Nov 11 '25 08:11 arbiv