dbx deploy does not support --pip-install-extras

[pspeter]

trafficstars

Since the --requirement-file option was deprecated, there is now no supported way to supply additional library dependencies to dbx deploy. However, these are crucial for jobs like integration tests, where you usually create an --assets-only job with additional dependencies like pytest that are not needed in production jobs.

Your Environment

• dbx version used: 0.7.4
• Databricks Runtime version: 11.1

[renardeinside]

hi @pspeter , this is a very valid question, we're currently considering various options on how to implement this.

[SamRodkey]

Any updates on this? Ran into this today when trying to deploy and launch the tutorial python quickstart integration tests. When I do this it can't find pytest and doesn't run...

[renardeinside]

hi @SamRodkey
we were discussing design of this feature in this issue - https://github.com/databrickslabs/dbx/issues/483

Some outcomes are there, but I need time to add an implementation.

[danricbede]

Any hope of this materialising? For integration testing the alternative is using a single user (we're on unity catalog) all purpose cluster and dbx execute, or hardcoding our dev requirements into the main package requirements 😬