Install a python requirement with an --extra-index-url argument

[mocaccano]

What is the problem or limitation you are having?

Hi, I have a project which requires PyTorch CPU version, not the default GPU version. Normally, it's possible to install this in one of two ways using pip:

pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cpu

or

pip install -r requirements.txt

where the requirements look like:

--extra-index-url https://download.pytorch.org/whl/cpu
torch==2.0.1
torchvision==0.15.2

Describe the solution you'd like

How do I translate this to the Briefcase environment? This is a question rather than a solution, but I'm new to Beeware and don't see any obvious way of including --extra-index-url in the requires section of the pyproject.toml file.

Describe alternatives you've considered

Thanks!

Additional context

No response

[mhsmith]

It would be useful to allow the pyproject.toml file to reference a requirements file, but we don't currently support that (#1275). Meanwhile, you should be able to set the option using an environment variable PIP_EXTRA_INDEX_URL, as shown here.

[freakboy3742]

Agreed that this is something we should formally support.

In addition to the PIP_EXTRA_INDEX_URL workaround, you could download the wheel manually, and add a direct reference to the wheel in your requires list.

Another workaround - although one that is a lot more fragile, and I wouldn't really want to rely on long term - would be to add ["--extra-index-url", "https:///...." to your requires list. By a quirk of implementation, it should work on macOS, iOS and Cocoa. On Android and Linux, adding "--extra-index-url https:///...." might work. This is obviously very fragile, but as a workaround, it might do the job.

[mocaccano]

Thank you for your responses. I tried adding "--extra-index-url https://...." to requires but it didn't work. I am on Ubuntu. On the other hand PIP_EXTRA_INDEX_URL works very well, so I'm using this for now. Manual wheel could work, but haven't tried it so far.

[rmartin16]

I tried adding "--extra-index-url https://...." to requires but it didn't work. I am on Ubuntu.

To add to the fragility of this workaround, you can't have the space separating the option and its value...an equal sign must be used. This should be true for any pip option you want to use:

 requires = [
     "--extra-index-url=https://download.pytorch.org/whl/cpu",
     "torch==2.0.1",
     "toga-gtk~=0.3.1",
 ]

[wyrd-0]

Agreed that this is something we should formally support.

In addition to the PIP_EXTRA_INDEX_URL workaround, you could download the wheel manually, and add a direct reference to the wheel in your requires list.

Another workaround - although one that is a lot more fragile, and I wouldn't really want to rely on long term - would be to add ["--extra-index-url", "https:///...." to your requires list. By a quirk of implementation, it should work on macOS, iOS and Cocoa. On Android and Linux, adding "--extra-index-url https:///...." might work. This is obviously very fragile, but as a workaround, it might do the job.

New here, but I just got through the tutorial and I'm looking to contribute. Wanted some clarity about what the most desirable solution would be:

1. extend briefcase to enable passing in a requirements.txt file OR
2. extend briefcase to enable passing in multiple --extra-index-url parameters and passing them along to pip OR
3. add [extra-index-urls] to pyproject.toml and incorporate that into updates

[mhsmith]

The most flexible way would be to allow arbitrary pip options, like Chaquopy does. @freakboy3742 @rmartin16: any thoughts?

[freakboy3742]

That's definitely an option; my primary hesitation is the possibility that at some point in the future, we might provide an install option other than pip (the most likely alternative being Conda; but in theory, there could be others), as well as (potentially) lock file formats that are bound to specific tools (e.g., poetry).

Baking pip options directly into the config could back us into a corner, whereas "repository url" and/or "extra repository URL" is less likely to be a problem.

[wyrd-0]

That's definitely an option; my primary hesitation is the possibility that at some point in the future, we might provide an install option other than pip (the most likely alternative being Conda; but in theory, there could be others), as well as (potentially) lock file formats that are bound to specific tools (e.g., poetry).

Baking pip options directly into the config could back us into a corner, whereas "repository url" and/or "extra repository URL" is less likely to be a problem.

@freakboy3742, how about I then add repository_url and extra_repository_url under [tool.briefcase.app.{project_name}], which will be ignored if left empty, and otherwise passed to the package installer?

[freakboy3742]

Sounds like a good approach to me; with the caveat that the extra needs to be a list, not just a single URL (since you can specify multiple extra URLs)

[rfletchr]

It would be fantastic if we could simply opt to disable briefcases package resolution and use an existing system like setup tools (boring tech FTW).

from setuptools import setup

setup(
    name='somepackage',
    install_requires=[
        'somedep'
    ],
    dependency_links=[
        'https://pypi.example.org/pypi/somedep/'
    ]
    # ...
)

pip install .
briefcase dev

I mean why re-invent the whl;)

[freakboy3742]

@rfletchr Briefcase will honor PEP621 metadata (now formally documented here). That means dependencies will be honoured as a [project] key; but dependency_links isn't a PEP621 key.

Beyond that, setuptools doesn't have sufficient flexibility to define the packaging needs of a bundled Python app, so it's not possible to fully specify a Briefcase configuration purely with setuptools.

[freakboy3742]

Adding to the feature list - we probably also want to support --find-links with a local_wheel_path setting so that we can point at a directory of local wheels.