installer Support for alternative installation schemes

From using this in Linux, it seems like the installation scheme used here is always "posix_prefix." I think it would be helpful to have the option to use "posix_user" too!

Apr 24 '22 19:04 greyltc

This also plays into vendor layout customizations, they could add their layouts to Python and then select them with this option.

Apr 24 '22 19:04 FFY00

This should already be possible via the API, which takes a mapping containing the installation scheme’s resolved paths: https://installer.readthedocs.io/en/latest/api/destinations/#installer.destinations.SchemeDictionaryDestination

Could you be a bit more specific about what you are trying to do, and what you’re requesting here?

Apr 24 '22 23:04 pradyunsg

I’m just guessing but maybe the goal is to install packages to the user site with the CLI. Currently python -m installer --destdir ~/.local does not do the “right thing”.

Apr 25 '22 04:04 uranusjr

Yes, that is not the goal of destdir. I think this would look something like python -m installer --scheme posix_user.

Apr 25 '22 04:04 FFY00

I’m more inclined to think installer’s CLI shouldn’t get into alternative schemes at all, and should probably remove --destdir altogether. But if y’all do want to do this, it’d be best to rethink how context var override should work (what --destdir currently do but only partially).

Right now only base can be overridden (via --destdir), but if you support changing the scheme, people will eventually want at least userbase and platbase, and eventually you’d get into the same situation of pip where people want to use the CLI for all sorts of things like populating an environment on Windows running Python 3.10 to use on a Repsberry Pi with Python 3.6.

Apr 25 '22 04:04 uranusjr

Base is never overridden by --destdir as they do conceptually different things.

e.g.

'purelib': '{base}/lib/python{py_version_short}/site-packages'

Specifically, base for a system installation is /usr, so that fills out as

/usr/lib/python3.10/site-packages

destdir=./foo prepends to that, so you end up with

./foo/usr/lib/python3.10/site-packages

And then you pack up the ./foo directory into a system image and install it as an overlay onto / so that the final product gets files in /usr/lib/python3.10/site-packages

Currently python -m installer --destdir ~/.local does not do the “right thing”.

This is a hard command-line API violation, based on the incorrect notion that destdir is intended to override {base}. It doesn't do the right thing, because as an API violation it produces garbled output.

if you support changing the scheme, people will eventually want at least userbase and platbase, and eventually you’d get into the same situation of pip where people want to use the CLI for all sorts of things like populating an environment on Windows running Python 3.10 to use on a Repsberry Pi with Python 3.6.

This is just a standard slippery slope. I fully believe in the ability of this project's maintainers to make rules and stick with them, and people already wanted to do all those things you fear, but their suggestions were rejected.

Accepting mostly-unrelated features should not necessitate backing down on previous stances taken. And the use case of this ticket is pretty clearly defined. It allows the CLI to ask sysconfig for one of a radio list of installation layouts. The two example cases given are:

implementing the equivalent of pip install --user,
implementing "install to wherever sysconfig says that debian packages put their python modules, instead of where users put them"

A radio list of schemes known to sysconfig is a pretty significant difference from "next thing you know, people will be asking to support arbitrary format-string definition of the install environment".

Apr 25 '22 06:04 eli-schwartz

I’d like to request folks to hold off going this direction in the discussion, based off of a guess.

I specifically asked for details, to better understand why OP wants this. Let’s wait for OP (or someone else who wants this functionality) to elaborate why they want it.

Apr 25 '22 07:04 pradyunsg

I've been trying to learn a bit more about python's packaging internals and I found myself reading https://docs.python.org/3/library/sysconfig.html?highlight=installation%20scheme#installation-paths which describes the several different instillation schemes python supports on equal terms. I was just trying to use installer as an educational tool to test these different schemes in different environments and platforms and I found I had no obvious way of asking installer to use a specific scheme. I didn't see an obvious reason why the scheme shouldn't be something I could select (such as via --scheme nt_user or --scheme osx_framework_user for example) and so I made this feature request.

Apr 30 '22 12:04 greyltc

The obvious way is to use the Python API I guess (which is the primary intention to the project). I’m not saying a CLI flag as you proposed isn’t useful though!

May 03 '22 23:05 uranusjr

Other use-case for this is allowing vendor to install to their custom schemes, something that our proposal to standardize vendor patching (https://gist.github.com/jaraco/b48e1acd05ecd3e54ddf0f04f91065c3) relies on. I think this is a very significant use-case, and is simple enough to support.

May 04 '22 00:05 FFY00

If someone wants to file a PR implementing such a flag, please feel welcome to do so!

May 04 '22 07:05 pradyunsg

Cross referencing https://github.com/pypa/installer/issues/145, which takes this a step further -- let's bring over all the logic that pip has currently, for locating files, and move it into this library.

Nov 25 '22 15:11 pradyunsg