cicd-templates icon indicating copy to clipboard operation
cicd-templates copied to clipboard

[DISCUSSION] How should this ideally interact with Databricks "Repos for Git integration"?

Open pgorsira opened this issue 4 years ago • 0 comments

Databricks Repos for Git Integration seems like a relatively new way to manage / deploy code in a Databricks environment. I understand that cicd-templates uses dbx which (this part may be wrong) packages up jobs into .whl files to be installed in Databricks environments.

From the "Repos for Git Integration" docs, one part says:

If you are using %run commands to make Python or R functions defined in a notebook available to another notebook, or are installing custom .whl files on a cluster, consider including those custom modules in a Databricks repo.

This seems like a different approach vs. the dbx approach. Just wondering if there are any guidelines here on what's appropriate when, how these approaches overlap / complement each other, and if the future of this project / dbx involves a deeper integration with the "Repos" approach.

pgorsira avatar Nov 01 '21 23:11 pgorsira