Support private PyPI repositories

[ab-10]

This issue is related to the following conversation from fast.ai forum: https://forums.fast.ai/t/nbdev-for-private-pypi-repositories-and-poetry-package-management/100523

Goal of the issue

1. To gauge the interest in nbdev supporting private PyPi repos (e.g. GCP Artifact Registry).
2. To discuss the best way to integrate private PyPi repo support into nbdev.

If there's interest, I'd be happy to make the contribution.

Problem

nbdev does not support dependencies from private PyPi repositories.

Workaround

Changing the following line in nbdev CI workflow:

test -f setup.py && pip install -e ".[dev]"

to:

test -f setup.py && pip install --extra-index-url https://$PYPIUSER:$PYPIPASSWORD@$PYPISERVER -e ".[dev]"

fixes the issue.

[UpstatePedro]

+1 for interest - this would be a big win for being able to use this for work!

[Elijas]

Option 1. Doing it without nbdev

How to publish a package to a private pypi server:

test -f setup.py && rm -rf dist build *.egg-info .eggs
python setup.py sdist bdist_wheel
twine upload --repository-url <URL> --username <PYPI_USERNAME> --password <PYPI_PASSWORD> dist/*

How to then install the package from a private pypi server:

pip install --extra-index-url "http://<PYPI_USERNAME>:<PYPI_PASSWORD>@123.134.56.7:8080" private_package

Change the placeholders:

• <URL>: The URL of your private PyPI repository.
• <PYPI_USERNAME>: The username for accessing your private PyPI repository.
• <PYPI_PASSWORD>: The password for accessing your private PyPI repository.
• 123.134.56.7: The placeholder IP address of your private PyPI repository.
  • Note: it's better to use a domain like my-pip-server.com instead of an IP address so that you can set up an SSL connection, otherwise your credentials are passed in plain text over the internet. Instructions how to do that are available here and here.
• 8080: The placeholder port number for your private PyPI repository.
• private_package: The private package you want to install.

Option 2. Using nbdev to do it

nbdev itself doesn't do anything different, as seen in their def release_pypi function in https://github.com/fastai/nbdev/blob/3f0266328c2537a35487767288bc4283308f7048/nbs/api/18_release.ipynb#L748

In fact, you probably can just create a ~/.pypirc file like

[distutils]
index-servers =
    private

[private]
repository: https://pypi.example.com
username: user
password: PUT_PASSWORD_HERE

and just pass release_pypi(repository="private") and you will be able to upload to your private PyPI server.

Existing threads related to this topic

• https://forums.fast.ai/t/nbdev-for-private-pypi-repositories-and-poetry-package-management/100523/4
• https://github.com/fastai/nbdev/issues/1304