pycross lockfile gen fails to translate private sources + auth correctly from poetry lockfile

[ashwin1dd]

Versions

rules_pycross = 0.6.1 bazel = 7.4.0

Problem

I have the following snippet in my Poetry toml file that defines a private source

[[tool.poetry.source]]
name = "artifactory"
url = "https://<private-url>/api/pypi/pypi-local/simple/"
priority = "supplemental"

I am able to add dependencies from this source to poetry successfully in poetry.lock:

[[package]]
name = "<artifact-name>"
version = "1.0.1"
description = ""
optional = false
python-versions = "*"
groups = ["main"]
files = [
    {file = "<artifact-name>-1.0.1.tar.gz", hash = "sha256:<sha>"},
]

[package.dependencies]
googleapis-common-protos = ">=1.5.3"
grpcio = ">=1.13.0"
grpcio-tools = ">=1.13.0"

[package.source]
type = "legacy"
url = "https://<private-url>/api/pypi/pypi-local/simple"
reference = "artifactory"

However, the source information is lost when translating into the pycross lockfile (.bzl)

    maybe(
        pypi_file,
        name = "my_poetry_lock_file_sdist_<artifact-name>_1.0.1",
        package_name = "<artifact-name>",
        package_version = "1.0.1",
        filename = "<artifact-name>-1.0.1.tar.gz",
        sha256 = "<sha>",
    )

Since no index is specified above, rules-pycross defaults to pypi.org and my build fails since it cannot find my artifact on pypi.org Ideally the translator should be able to use the right package source and appropriately apply basic http auth to fetch the package. https://python-poetry.org/docs/repositories/#configuring-credentials

Is there a workaround or something I could do differently to make this work?

Thank you!

[njlr]

Is this a duplicate of https://github.com/jvolkman/rules_pycross/issues/117 ?

[ashwin1dd]

No, I am not getting an error on lock generation. I am getting an error during my project build and I can see that the lockfile content is incorrect.

[jvolkman]

There are a few ways to go about this.

1. if your artifactory server mirrors pypi for public packages, you can instruct pycross to use a different pypi base URL. See the pypi_index attr here: https://github.com/jvolkman/rules_pycross/blob/main/docs/ext_lock_repos.md
2. set always_build to True and provide a build_target that returns your .whl. Note that you don't actually need to "build" your wheel in this target; it just has to return a valid wheel file. See https://github.com/jvolkman/rules_pycross/blob/main/docs/ext_lock_import.md#package

[blockjon-dd]

I'm trying the above guidance for option 2, but I just cannot figure out a way to make it work. I'm still in my first year of Bazel, so I might be missing something obvious.

I'm using pycross+poetry. Is there an example reference implantation that I can refer to that shows how I could get a python package from our private artifactory?

[dingfengshaoDD]

Hi @jvolkman, for option 1, is it possible to support multiple pip index? Thanks!