[ADAP-702] [Feature] Allow user to specify location directly (instead of concatenating `location_root` and `alias`)

[timvw]

trafficstars

Is this your first time submitting a feature request?

• [X] I have read the expectations for open source contributors
• [X] I have searched the existing issues, and I could not find an existing issue for this feature
• [X] I am requesting a straightforward extension of existing dbt-spark functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Currently we have a location_clause macro in place which calculates the location where the data should be stored as {{ location_root }}/{{ identifier }}.

It would be nice to be able to specify location directly (independent of location_root and identifier). Eg: we have naming conventions where the tables are name blah_blah and locations are s3://bucket/blah-blah.

Describe alternatives you've considered

We could not implement this and push this to user-space

Who will this benefit?

Anyone that wants to specify a location that is not the result of concatenating location_root and identifier.

Are you interested in contributing this feature?

Yes

Anything else?

[dbeatty10]

Thanks for raising this idea @timvw 💡 !

Would you be willing to share the code for your location_clause macro in addition to examples using it? And also an example of how you are proposing to specify the location directly instead?

That way, we can take a look at your proposal vs. the alternative you are using currently.

[timvw]

The existing implementation of location_clause and the difference with my proposal can be found here: https://github.com/dbt-labs/dbt-spark/pull/835/files

Usage is straightforward, simply add a location parameter to your model config:

models:
  - name: model_a
    config:
      format: delta
      location: 's3://bucket/sourcesystem/conformed/model-a'
...

Without this feature we would be forced to rename our model to model-a..

[dbeatty10]

Thanks for this example and explanation @timvw !

What do you think @dataders? Do you see this as tying into https://github.com/dbt-labs/dbt-core/discussions/8617 at all?

[github-actions[bot]]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.