dbt-loom icon indicating copy to clipboard operation
dbt-loom copied to clipboard

Support pulling in multiple manifests from single bucket

Open akromish opened this issue 11 months ago • 6 comments

Currently, dbt-loom supports pulling in a manifest from cloud storage using bucket name + object name.

However, for organizations with n number of dbt-core projects that need to peer with each other, adding an entry to each repo gets difficult. I propose that in the s3 and gcp clients, we add a method that allows for specifying just the bucket name. From there, dbt loom will iterate through all the manifests in the bucket and add them to the project.

I could take a first stab at implementing s3 version.

Edit: Would actually prefer trying this in artifiactory first if this is something we want to do. Can implement single and muli-manifest json pull from artifiactory

akromish avatar Feb 28 '24 19:02 akromish