pypi-support
pypi-support copied to clipboard
pypa
PEP 541 Request: `install`
Project to be claimed
PROJECT_NAME: https://pypi.org/project/install
Your PyPI username
USER_NAME: https://pypi.org/user/FFY00
Reasons for the request
Name squatting.
Maintenance or replacement?
Replacement. Upstream: https://github.com/FFY00/python-install
Contact and additional research
I contacted the owner, he confirmed he was name squatting. His main argument was that he is trying to prevent the abuse of the namespace to distribute malware, however I have a valid use for it.
The full details:
I lurked google and the owner's github page for the project but I couldn't find it, I contacted him on the 28th of May:
Hey Eugene,
Sorry to contact you this way. As you may know, Python packaging is changing, as defined in PEP517[1]. Because of this Linux distributions will have to adapt the build and install workflows. The use of
setup.py installis also being removed, forcing us to change the workflow to build and install wheels. It looks like the PEP517 and setuptools compatible workflow will be using build[2], to build wheels, and install[3], to install the wheels, at least these seem like the most promising tools at the moment. I am actually the author of install[3] and I would like to release it in PyPI under the 'install' namespace, but as you might remember, it is already taken by your install[4] project. I search for the project page for a bit but couldn't find anything. Is the project still active? I wanted ask you if you would consider giving up that namespace?To release it you can simply remove the package or or transfer it to me (FFY00).
If you have any questions, please let me know. Any help with this would be greatly appreciated :)
[1] https://www.python.org/dev/peps/pep-0517/ [2] https://github.com/FFY00/python-build [3] https://github.com/FFY00/python-install
Cheers, Filipe Laíns
He reached out on the 10th of June:
Hello Filipe,
Thank you for reaching out to me. PEP-517 sounds exciting, and the work you're doing on it is great. However, at this time I do not wish to transfer the install project. Due to typographical errors that occur, the usage of the pip package, 'install', is dangerous. I believe package installation should be delegated to pip solely.
Best, Eugene
I replied the same day:
Hi Eugene,
I have written some thoughts about pip vs distribution package management here[1], please have a look. The sum of it is that just doesn't work well when binaries start appearing.
If you are just talking about using pip to install the wheels, instead of this new
installpackage, the sort answer is that it is not compatible with several workflows, the main on being Linux distributions. Our problem is that we need to bootstrap pip, how do we install pip if we need pip to install? We need to manually install all dependencies. The pip dependency tree just for build/install requirements has 35 packages, we would need to manually install every single one of them to be able to use pip in our workflow. This has been extensively discussed here[2] and here[3]. The solution to create aninstallmodule had been made in collaboration with PyPA (Python Packaging Authority) members and is the correct way to move forward for such workflows. Workflows that want reproducible install, ideal if you are deploying, at this moment, also require the use of such tool, as pip does not support them[4].Due to typographical errors that occur, the usage of the pip package, 'install', is dangerous.
Could you elaborate on this? I do not fully understand.
[1] https://github.com/analogdevicesinc/libiio/issues/545#issuecomment-640053834 [2] https://github.com/pypa/setuptools/pull/2080 [3] https://github.com/pypa/setuptools/issues/2088 [4] https://github.com/pypa/pip/issues/5648
Cheers, Filipe Laíns
And he finally replied today (14th of June):
Hi Filipe,
As far as I know, pip is included with python these days. No package manager is perfect, but I do not believe creating additional build/installation tools is the solution.
The typographical errors that I mention have to do with the issue known as 'typosquatting'. I specifically have
installregistered, so that somebody else does not register it and distribute malware when somebody typos 'pip install install otherpkg'. A few instances of this threat have been written about here: https://nakedsecurity.sophos.com/2017/09/19/pypi-python-repository-hit-by-typosquatting-sneak-attack/ and https://lwn.net/Articles/694830/.Best, Eugene
I sent my final reply and decided to open this request since the owner confirmed the name squatting and doesn't seem to be collaborative:
Hi Eugene,
On Sat, 2020-06-13 at 19:01 -0400, Eugene Kolo wrote:
Hi Filipe,
As far as I know, pip is included with python these days. No package manager is perfect, but I do not believe creating additional build/installation tools is the solution.
It isn't, it's a totally separate project, hence the need to bootstrap the dependencies. You can believe whatever you whish, I provided you enough information for you to understand the problem in depth. If you think you have a better way to deal with our issue, feel free to let me know, I would love to hear it.
The typographical errors that I mention have to do with the issue known as 'typosquatting'. I specifically have
installregistered, so that somebody else does not register it and distribute malware when somebody typos 'pip install install otherpkg'. A few instances of this threat have been written about here: https://nakedsecurity.sophos.com/2017/09/19/pypi-python-repository-hit-by-typosquatting-sneak-attack/ and https://lwn.net/Articles/694830/.While I appreciate you registering the name to prevent malware to be distributed, 'install' is still a valid namespace and right now there is a legitimate need to use it and I am requesting it. Currently your use of the namespace is invalid, you should free it when requested.
Cheers, Filipe Laíns
I acknowledge I might have been a bit adamant in my last reply. I am a bit frustrated, especially since given the owner's comments on how he believes my project is wrong, even after I provided links to discussions which explore the issue I am trying to solve in depth. This is not an excuse, I should have worded it in a more friendly way.
Hi @FFY00, there's a few elements at play here.
Firstly, at the moment of this writing, install has a new release with some functionality, therefore it cannot be considered invalid or namesquatting. While I appreciate you taking the time to contact the owner us moderators are obliged to make contact ourselves. In that spirit I'm tagging GitHub user @eugenekolo.
Secondly, if he does confirm he refuses the transfer, the PEP is clear about that:
Under no circumstances will a name be reassigned against the wishes of a reachable owner.
Thirdly, in those emails you've posted the owner does make a case that install is a dangerous project name and I agree, however, in these cases PyPI admins place the name in a blocklist, user namesquatting is not encouraged.
Finally, you argue that your project is a better fit to the name install as it implements PEP 517 and mention "collaboration with PyPA":
The solution to create an install module had been made in collaboration with PyPA (Python Packaging Authority) members and is the correct way to move forward for such workflows
However, there's no sign of involvement of the PyPA in your project. I would encourage you to reach out to the PyPA (I suggest the official Python forum) and discuss the project with them, including any potential names and subsquent actions.
I would very much like for other @pypa/pypi-moderators to comment on this. Personally, I think the best course forward is to place install in the block list and for @FFY00 to discuss his project elsewhere with the PyPA.
Firstly, at the moment of this writing,
installhas a new release with some functionality, therefore it cannot be considered invalid or namesquatting.
This change was made after I created the request. The owner admitted to namesquatting and refused to collaborate, and as such, I do not believe this change was made in good faith. I have been very patient and cordial when dealing with this, up to the penultimate email, I have explained why the project is needed and why other alternatives are not usable, @eugenekolo simply dismisses it as "I don't agree" providing no arguments. I do not think this action is correct.
The package had no downloads until today, which now has 2. https://pypistats.org/packages/install
And after I checked out the contents of the package, I cannot help to feel like this is a personal attack.
#!/usr/bin/python
import subprocess
import sys
import shlex
import os
import tempfile
import urllib.request
def _get_pip():
fd, path = tempfile.mkstemp('_get-pip.py')
urllib.request.urlretrieve("https://bootstrap.pypa.io/get-pip.py", path)
subprocess.check_call([sys.executable, path])
def _check_pip():
try:
subprocess.check_call([sys.executable, '-m', 'pip'], stdout=subprocess.DEVNULL)
return True
except subprocess.CalledProcessError as exc:
return False
def install(pkg, use_pep517=None, requirements=None, options=None):
"""Install packages dynamically in your code
Args:
pkg: Name of the package as a string, you can also use version specifiers like requests==1.2.3
use_pep517: Optionally set to True or False to force --use-pep517/--no-use-pep517
options: Arbitary list of options to pass to pip for installation
"""
if not _check_pip(): _get_pip()
cmd = [sys.executable, '-m', 'pip', 'install']
if options and isinstance(options, list):
cmd.extend(options)
if use_pep517 is True:
cmd.append('--use-pep517')
elif use_pep517 is False:
cmd.append('--no-use-pep517')
pkg = shlex.quote(pkg)
cmd.append(pkg)
subprocess.check_call(cmd)
The package downloads pip and installs packages with it. I am perplexed seeing someone would engage in such behavior. I do not see any reason for him to do this other than spite.
"I believe package installation should be delegated to pip solely."
"No package manager is perfect, but I do not believe creating additional build/installation tools is the solution."
I can't......
About the PyPA involvement, I talked with @pganssle (sorry for tagging you like this, please let me know if you wish me to not do that in the future) and the outcome of the conversation was that we could have a standalone tool to build wheels and one to install them. For the install step, https://github.com/pradyunsg/installer exists but there was a discussion on whether to have a CLI, and if so, what was the scope. In the initial discussion there was clear that there were several people against some of my requirements for the CLI, and so it was better to provide a suitable CLI for my needs in a separate project, hence https://github.com/FFY00/python-install. For the build step, I created https://github.com/FFY00/python-build which is now in the process of being transferred to PyPA.
Thanks for reaching out to me @yeraydiazdiaz.
I refuse the transfer. I am okay with either the package name being blocked, or for me to continue maintaining ownership of the package. The current functionality of the package is being used in a few projects of mine and may be useful to others too.
@eugenekolo that doesn't make much sense. If we were to place the project name in the blocklist you wouldn't be able to use that code with that name thus breaking your own usage of the project.
Regardless you uploaded some code after the request was raised and now you're claiming you're actively using the code.
PEP 541 does not completely cover every possible claim case so we rely on the good will of the community to work together to resolve these issues.
If I may, I would like to advocate against placing the name on the blocklist. The install package had not registered downloads until today, which tells me that the issue with people mistakenly calling pip install install something is not that much of an issue. It is still possible though, but if we have a project there worst-case scenario people will install that project. And finally, there is a real need for the namespace.
I just wanted to state this before any further discussion. I will now refrain from making any other comments (until I am directly addressed) and let you proceed with the conflict resolution. Thank you :)
@yeraydiazdiaz Sorry for the misunderstanding. The code I've placed into the package is code I am using on other projects. It was added to the install package and will continue to be maintained there. Thanks for looking into this.
@eugenekolo You registered the project in 2016 with no functionality and then you pushed a release with some code after the request was made in an attempt to keep the project name.
I understand you probably had plans for that project name in 2016 but the fact is you have not used it in four years. By pushing some code now, immediately after the PEP 541 request, and giving some vague reasons for it you're not showing a lot support for a process that's driven by volunteers such as myself.
From my point of view as a moderator install should be transferred to @FFY00 because it was name squatting. Other @pypa/pypi-moderators and admins can make a final decision on this matter, I will not comment further.
What is the argument for putting it onto the blocked list, particularly if Filipe is going to put a legitimate package there?
It's possible someone will typo it and it will accidentally install the extra package, but that's true of many things. That's really only dangerous if something malicious lives there.
I don't really think it matters very much, I think Filipe's package could easily be named wheel-installer or something (it's more likely to live in distro packaging scripts where "short and sweet" doesn't matter), but I'm not convinced that this is an actual security issue.
I am just not sure what the argument is for
I don't really think it matters very much, I think Filipe's package could easily be named
wheel-installeror something
It is currently only a wheel installer, but I wanted to keep the possibility of other types in the future. Much like https://github.com/FFY00/python-build. It is not crazy to think that in the future other package types could arise.
Anyway, even though I would really like for it to be called install, it is not a big blocker.
Similarly, I don't really understand the argument. I feel like this is being a little too conservative. We are preemptively trying to solve an issue that has no real data backing it up -- by this I mean no data supports that people are mistakenly running pip install install something or similar. Even if they were, I struggle to categorize it as a security issue.
puts on pip maintainer hat
I would encourage you to reach out to the PyPA (I suggest the official Python forum) and discuss the project with them
@FFY00 had also discussed this with me (and I'm a PyPA member, but don't represent all of PyPA). We'd talked about his project and https://github.com/pradyunsg/installer. The latter is certainly a project that I will be putting forward for being accepted into PyPA once it's "ready". He also expressed interest in doing so, if installer doesn't add a proper CLI for the Linux-distro use case, which I'm 100% onboard for.
puts on PyPI moderator hat
I'm definitely a +1 for putting this on the prohibited project name list.
When @FFY00 and I discussed about installers for Python packages, I'd advised him that install on PyPI was definitely namesquatted and a name transfer via a PEP 541 request would be reasonable step since he actually wanted to use the project. I'd also hinted that I wasn't 100% sure about the name -- I felt then and still do that pip install install certainly feels like a like typo than something you'd intentionally do, and should've been prohibited on PyPI already. :)
What is the argument for putting it onto the blocked list
Typos like pip install install <list of packages>. I'm aware it's a sample size of 1, but yea, I have certainly done made this typo before, while debugging pip's behavior. And yes, I've looked at this package before. (my workflow usually involves copy pasting a lot of pip commands while my brain is mushy because I can't understand what pip is doing :P)
Typos like
pip install install <list of packages>
Yeah, I get that it is something you might install accidentally, but I might also mindlessly type anything in there. The only problem with "typo squatting" like this is if someone puts something malicious there (or has the potential to put something malicious there), and accidental installs lead to harmful outcomes.
This is just a situation where someone might accidentally install more packages than they wanted to, which is not really a problem and happens constantly.
Assuming that install never gets compromised, ofc.
On Tue, Jun 16, 2020, 1:10 PM Paul Ganssle [email protected] wrote:
Typos like pip install install
Yeah, I get that it is something you might install accidentally, but I might also mindlessly type anything in there. The only problem with "typo squatting" like this is if someone puts something malicious there (or has the potential to put something malicious there), and accidental installs lead to harmful outcomes.
This is just a situation where someone might accidentally install more packages than they wanted to, which is not really a problem and happens constantly.
— You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub https://github.com/pypa/pypi-support/issues/451#issuecomment-644986691, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB5I42MQ6WPEACHPIFRQJ3RW7GVFANCNFSM4N5GAF6A .
Assuming that
installnever gets compromised, ofc.
Sure, but if the threat model is "package compromise", then it's no different whether it is deliberately installed or accidentally installed. In fact, things that are primarily (or even very frequently) accidentally installed are probably a particularly bad target for compromise, because that doesn't happen terribly often compared to deliberate installs and when someone accidentally installs something they're much more likely to scrutinize it de novo than if they deliberately installed it.
Like I said, I don't really care that much, and I don't even really think install is a terribly good name for a package, but so long as it is being put to non-malicious use, I don't think there's a security issue with a package being named that.
Hello, I'd like to bring up a few facts and statements to this conversation. Unfortunately, FFY00 has blocked me on GitHub, and that means I cannot respond to any threads created by them with my main account, so please excuse this secondary account.
- I have added legitimate code to the
installpackage that I meant to 3 years ago. It seems that I messed up at some point, and the commit never went through. I believe that explains the 0 installations over the period of time. - At the time of this comment (and for the past 3 days), the
installpackage is currently receiving approximately 3000 installations a day. I do not know how many of those find the package that I've created useful, or how many are typos. - At least one user (other than myself) is interested in the
installpackage and has asked if I will add Python 2.7 support. - The
installpackage was registered in 2016, before the draft (2017) and finalization (2018) of PEP-541. At the time of creation (and until last week), I was not aware of any the limitations or rules that PEP-541 brings to the table. I wish to comply with PEP-541, and moving forward I am much more aware. - I am a reachable maintainer of this package and do not wish to transfer it.
I believe that in accordance with PEP-541, I am in the right to continue maintaining this project as I am a reachable and legitimate maintainer of a project that is receiving active installations.
Unfortunately, FFY00 has blocked me on GitHub, and that means I cannot respond to any threads created by them with my main account, so please excuse this secondary account.
My bad, this was to prevent you to interact with my personal repositories, as I do not think the way you handled this was correct. I did not think it would prevent you from commenting on this issue. I have temporarily removed it until this gets resolved.
I have added legitimate code to the
installpackage that I meant to 3 years ago. It seems that I messed up at some point, and the commit never went through. I believe that explains the 0 installations over the period of time.
I would really like to believe this but it does not track with our email communication. After I contacted you, you did not pushed anything, that only happened after I opened this request.
Furthermore, what you said in our emails goes directly against this statement.
I specifically have install registered, so that somebody else does not register it and distribute malware when somebody typos 'pip install install otherpkg'.
Respectfully, all this seems like damage control...
I always try to assume best intentions, unless I have proof against it. In this case, I find it difficult. But if you are truly being genuine, I apologize for all the drama/trouble. This would have been simply resolved by you replying "Hi Filipe, I do have plans to publish something under that namespace.".
I believe that in accordance with PEP-541, I am in the right to continue maintaining this project as I am a reachable and legitimate maintainer of a project that is receiving active installations.
Well, I actually think this specific case is not defined in PEP 541. This was name squatting, you only pushed something after this request was opened.
I think we have enough information from involved parties for the time being, and it looks like things could get unnecessarily heated. I'm going to lock this to maintainers/admins for the time being until we can get a decision made.
As long as this is a valid and non-malicious project, I'm also not really seeing a reason to put this on the prohibited project name list.
Looking at the source for https://pypi.org/project/install/ (see below) it seems to be both, basically just a thin importable wrapper around python -m pip install.
I'm wondering if there's a potential for collaboration here if both parties are willing. Based on the discussions in https://discuss.python.org/t/moving-python-build-to-pypa/4390/ about @FFY00's build project, https://github.com/FFY00/python-install may need to grow the ability to install arbitrary packages from PyPI (unless https://github.com/pradyunsg/installer ends up being the tool used instead).
I imagine it would require an importable API as well, so https://pypi.org/project/install/ could have roughly the existing behavior (importable API to invoke pip install, as well as the behavior of https://github.com/FFY00/python-install).
Ultimately if we can't find a compromise, I agree with @pganssle that https://github.com/FFY00/python-install doesn't really need a short, memorable name as it's not likely to be widely used by end-users. In addition, PEP 541 is quite clear that a name should not be reassigned against the wishes of a reachable owner.
PS: I couldn't find a source repository for https://pypi.org/project/install/, so I extracted the source from the latest distribution and included it below to make it easier to understand the goal of this project.
#!/usr/bin/python
import subprocess
import sys
import shlex
import os
import tempfile
import urllib.request
def _get_pip():
fd, path = tempfile.mkstemp('_get-pip.py')
urllib.request.urlretrieve("https://bootstrap.pypa.io/get-pip.py", path)
subprocess.check_call([sys.executable, path])
def _check_pip():
try:
subprocess.check_call([sys.executable, '-m', 'pip'], stdout=subprocess.DEVNULL)
return True
except subprocess.CalledProcessError as exc:
return False
def install(pkg, use_pep517=None, requirements=None, pip_options=None, install_options=None):
"""Install packages dynamically in your code
Args:
pkg: Name of the package or requirements.txt file as a string, you can also use version specifiers like requests==1.2.3
use_pep517: Optional boolean to force --use-pep517/--no-use-pep517
requirements: Optional boolean if a requirements.txt was specified
pip_options: Optional arbitary list of global options to pass to pip
install_options: Optional arbitary list of install options to pass to pip install
"""
if not _check_pip(): _get_pip()
cmd = [sys.executable, '-m', 'pip']
if pip_options:
if isinstance(pip_options, list):
options = [shlex.quote(option) for option in pip_options]
cmd.extend(options)
else:
raise TypeError('pip_options passed to install must be a list')
cmd.append('install')
if install_options:
if isinstance(install_options, list):
options = [shlex.quote(option) for option in install_options]
cmd.extend(options)
else:
raise TypeError('install_options passed to install must be a list')
if use_pep517 is True:
cmd.append('--use-pep517')
elif use_pep517 is False:
cmd.append('--no-use-pep517')
if requirements:
cmd.append('-r')
pkg = shlex.quote(pkg)
cmd.append(pkg)
subprocess.check_call(cmd)
(I'll give this a little more time to cool off and for other admins to add their thoughts before unlocking.)
I'm going to motion based on @di's research that this project be placed on the prohibited project list, and that the existing project is removed. Especially given that there have been a few reports regarding the package name to the security list. It voices a number of similar concerns around confusion and potential for abuse.
Given that, I propose a 30 day window beginning with admin approval to provide @eugenekolo time to release the project with a new name before the existing one is removed.
Wow that was a long 30 days.
I'm going to unlock this issue temporarily to allow for any public objections before proceeding to prohibit the project.
FWIW (and tangential to the ownership conflict here), using extremely generic names like «build», «install», «installer» (or «cryptography» or things like «comments» for a django-comments app — end tangent) met some opposition when the projects were created but not enough to convince their authors.
As expected, there have been many cases of people being confused (of instructions like «pip install build», the awkwardness of referencing «pypa/build» for communication, issues with Python trying to import from a local build directory because of implicit namespace packages…)
It’s probably too late to convince the authors to change the tools’ names, but hopefully future projects will avoid all of these issues with more unique names («requirementslib» is good, «packaging» is borderline)!
According to pepy.tech, this project has 732,566 downloads in the last month: https://www.pepy.tech/projects/install
I have downloaded the latest version (1.3.5) to inspect the code.
The code has following functionality:
- check if package is already installed. This is done with
importlib.import_module(pkg) - download pip, if not installed
- build the
pipcommand which is later called bysubprocess.check_call(cmd)
To use the module, you need to import the install package.
This should be a working example:
from install import install
install("requests")
It does not return any status codes or an output, but raises an exception if the command exits with a non zero value.
The only problem is, that install tries to load the package as a module. This can fail because the packagename must not a valid module.
The same can be achieved by following code. This example also checks if the package is installed with importlib.metadata and works in cases where the packagename is not the same as the module name.
import subprocess
from importlib.metadata import metadata, PackageNotFoundError
try:
metadata('requests')
return
excpet PackageNotFoundError:
pass
subprocess.check_call(["pip", "install", "requests"])
It's interesting that the project has such a high number of downloades.
I have also done some wrong installations like pip install install requests because of copy paste errors or other mistakes. This behavior was also mentioned in https://github.com/pypa/pip/issues/12433
Installing "install" without notice is an error which can lead to an unwanted package installation. As long as the "install" package has no dependencies to other packages the installation of "install" should not have negative side effects. This might change if "install" required some dependencies with specific version numbers. This can lead to vulnerable software or introduce hard to find bugs because of incompatible packages.
Some of you may argue installing unwanted packages is a common mistake in software development, but in this has the potential for a hard to find bug. You are expecting an "install" argument when using pip. So the second "install" might be overseen or mistaken added to an other argument list.
In my (personal) opinion I expect that pip stops with an error, mentioning the second "install" argument.
Alternative:
Perhaps "pip" can provide a list of "package" names which conflicts with common pip arguments. In such cases pip will stop execution and asks the user to proceed or not:
Warning!
A second "install" argument was provided.
This will install the package "install".
Perhaps this was a mistake.
Do you want to continue installing the "install" package? [y/N]
To avoid such warninigs an additional argument to pip can be introduced to bypass this check and allow scripted installations:
pip --allow-pip-args-as-package-name install install
Not an objection, but: I strongly support adding this to the prohibited names list! IMO doing so does not preclude use for an official tool in the future, but does close a current unintended user error pathway.
I'm not directly involved here other than this comment, but I agree that install feels like a good candidate to put on the prohibited package names list based on the above discussion.
Petr and I recommend the @pypi/warehouse-admins add this project to the prohibited names list given previous discussion. Ee has given the project plenty of time to object, and it seems everybody's fine with the name being taken away.
Disclaimer: We are providing support to the PyPI Administrators to validate this request and make a recommendation on the outcome and actions to be taken. Final determination will be made by the PyPI Administrators when our process is complete.
everybody's fine with the name being taken away
This includes the current owner, per https://github.com/pypi/support/issues/451#issuecomment-643770420
