Installing private git repo

[benjaminbluhm]

trafficstars

Hi,

Suppose I have a private python package in some git repo that I need to use in my workflow. Is it possible to add this as a dependency to conda_dependecies.yml?

I tried adding this to the pip section of conda_dependecies.yml as follows:

git+ssh://[email protected]/user/myrepo.git

I also added an install ssh key task to diabetes_regression-ci.yml as described here. However, when I run the pipeline the image build failes which I believe is due to an authentication error.

Extract from log file azureml-logs/20_image_build_log.txt:

Collecting git+https://github.com/user/myrepo.git (from -r /azureml-environment-setup/condaenv.m4b5awlb.requirements.txt (line 6)) Cloning https://github.com/user/myrepo.git to /tmp/pip-req-build-xeimm1d_ [91m Running command git clone -q https://github.com/user/myrepo.git /tmp/pip-req-build-xeimm1d_ [0m[91m fatal: could not read Username for 'https://github.com': No such device or address [0m[91mERROR: Command errored out with exit status 128: git clone -q https://github.com/user/myrepo.git /tmp/pip-req-build-xeimm1d_ Check the logs for full command output.

Any help on how to install a private git repo would be much appreciated.

[j-so]

Can you expand on exactly what part of the workflow uses the private git repo? Is this for the training pipeline? Or a part of the Azure DevOps pipeline that does not use AzureML?

[benjaminbluhm]

This is for the training pipeline. The repo contains a custom model class with a fit and predict method. Therefore I need to import the module in different stages of the training pipeline. Let me know if you need more details, thanks

[epa095]

@benjaminbluhm We also have a custom model class like you, and described a workaround in #280. It would still be good to be able to import custom pip packages though, since there are definitely cases where the cleanest thing is to separate stuff out in a new library so it can be used by several of your internal projects. Today the easiest is to push your boss to open source it and put it on pypi ;-)

[benjaminbluhm]

@epa095 Thanks for pointing out the workaround described in #280. Not sure if I can convince my boss to open source the repo ;-) I will take a look at your proposed solution and see if I can make it work. It would be very helpful to add an example of this workaround to the docs.