cpython
cpython copied to clipboard
python
'venv --clear' should prompt user before nuking entire directory
| BPO | 46326 |
|---|---|
| Nosy | @vsajip, @eryksun, @alimpfard |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2022-01-10.09:01:46.744>
labels = ['type-bug']
title = "'venv --clear' should prompt user before nuking entire directory"
updated_at = <Date 2022-01-13.15:29:41.064>
user = 'https://github.com/alimpfard'
bugs.python.org fields:
activity = <Date 2022-01-13.15:29:41.064>
actor = 'ned.deily'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2022-01-10.09:01:46.744>
creator = 'alimpfard'
dependencies = []
files = []
hgrepos = []
issue_num = 46326
keywords = []
message_count = 6.0
messages = ['410192', '410223', '410228', '410229', '410230', '410282']
nosy_count = 3.0
nosy_names = ['vinay.sajip', 'eryksun', 'alimpfard']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46326'
versions = []
virtualenv --clear is extremely eager to delete the passed directory without
any sort of confirmation, leading to possible data-loss
(e.g. with a mistyped command, or a misunderstanding of what it actually does).
Simply deleting an entire directory tree with a command that's extremely
prone to misunderstanding should not be virtualenv's job, but as it has
decided to make this its job, then it should take proper precautions to avoid
unintentional rm -fr some-directory by at least asking for confirmation.
The previous behaviour can of course, stay behind a --force flag for uses in
CI or similar, but this should not be the default behaviour.
related: https://github.com/pypa/virtualenv/issues/1890 mentions that "[it is not] good practice to put non-virtual environment-related files into a virtual environment", which is sensible, and so virtualenv should then simply refuse to be instantiated in a non-empty directory.
On a more subjetive note, virtualenv --clear <dir> sounds like a way to purge the virtual environment files from a directory (yes, I understand what the help says, but that hardly changes anything), so perhaps a more verbose flag like "--clear-contents" would be more appropriate, but I digress.
It appears you are describing behavior in the third-party virtualenv project; if so, you should report the problem to that project (https://pypi.org/project/virtualenv/). If similar behavior is also an issue with the Python standard library venv module (which is a derivative of virtualenv), please update this issue accordingly. Thanks!
https://docs.python.org/3/library/venv.html
Ned, the CLI of venv has a --clear option, which invokes EnvBuilder [1] with clear=True. It's implemented by EnvBuilder.clear_directory(). Ali wants this method to prompt for confirmation before deleting the directory contents and also proposes the addition of a new --force option to skip the prompt.
--- [1] https://docs.python.org/3/library/venv.html#api
Ned, the CLI of venv has a
--clearoption, which invokes EnvBuilder [1] with clear=True.
Thanks, I saw that. However, the issue as written does not mention venv at all, only virtualenv. I wanted the OP to verify that it does apply to venv and, if the problem exists both places, to update this issue and to open an issue against virtualenv.
The implementation of --clear in virtualenv apparently was changed to align with venv. On the linked virtualenv issue, Bernát Gábor suggested creating a bpo issue for venv to determine how the issue should be resolved for virtualenv.
If similar behavior is also an issue with the Python standard library venv module (which is a derivative of virtualenv), please update this issue accordingly
Yes, my bad, I should have mentioned that this applies to venv, I will update the issue title.
Not sure if I can modify my earlier comment however (I don't see any way to).
I think venvs should be seen as disposable, and easily recreatable using e.g. pip install -r requirements.txt. What's the reason why this can't be done on the odd occasion when a venv is cleared out unintentionally? The help for this option is pretty unambiguous.
was messing with venv, and now my (fortunately termux) $HOME is gone.
The help for the venv cli says:
positional arguments:
ENV_DIR A directory to create the environment in.
which means it should be perfectly fine to pass . or any existing directory as ENV_DIR, and same for --clear as well. You can say it's the user's (my) fault of not understanding what venv does, but also seems the help also doesn't say what it does clearly.
That's one of the evilest things I can imagine.
--clear Delete the contents of the environment directory if it already exists, before environment creation.
Does the description of this damn option says that it will wipe my source code? YES
Is it my fault that I just deleted an hour of my work? certainly
This can, of course, be understood literally, but for the love of god, please make it more clear and intuitive, because it may well be interpreted as clearing contents previously created by the venv itself, and not everything, including user files.
So please, either add a data loss warning in the description or a user-acknowledged warning before executing the command as mentioned earlier.
Plus, the expected behavior imho should be to remove only files created by the venv itself, like lib/python3.11/site-packages/, because - I assume - other users, like me, may want to simply "refresh" the venv for a fresh start. I see no justification for deleting the entire directory. Should someone want it, can do so before running -m venv.
It is not AFAIK considered good practice to put your source code in a venv directory. From the Python documentation on venv:
A virtual environment is (amongst other things):
- Used to contain a specific Python interpreter and software libraries and binaries which are needed to support a project (library or application). These are by default isolated from software in other virtual environments and Python interpreters and libraries installed in the operating system.
- Contained in a directory, conventionally either named venv or .venv in the project directory, or under a container directory for lots of virtual environments, such as ~/.virtualenvs.
- Not checked into source control systems such as Git.
- Considered as disposable – it should be simple to delete and recreate it from scratch. You don’t place any project code in the environment
- Not considered as movable or copyable – you just recreate the same environment in the target location.
Please note the second-from-last point. Which part of that documentation is unclear? Or have I misunderstood something about your post, @c0m4r?
Which part of me admitting that it was totally my fault you didn't understand?
I use it differently for convenience and because I can. It's not like someone will come to me and slap me in the face because whooo I don't follow instructions. At most, I will wipe some part of my work, as I did, but I will forget about it 5 minutes later since I'm doing my best to backup or push any changes frequently; no big deal. Why is it convenient for me you ask? Because switching back and forth to the venv every single time is annoying, and I do it a lot, often on a fresh OS where I can't simply script it. Therefore, typing f.e. bin/python is quicker than .venv/bin/python. Does typing a few more characters seem like a hassle? No. Can I just switch to the venv and type even fewer characters? Yes. But it's convenient for most use cases, not every single one.
Not a good practice? On every single electronic device, there's a freaking warning sign and a message telling me that I'm going to die the moment I open it up, and it's not considered to be a good practice to do so. Yet, I'm still alive. There's a reason why this issue exists in the first place. It is because people sometimes don't follow instructions, and I'm taught to take that into account when designing anything, making stuff failsafe or at least prompting people for confirmation when I'm about to ruin their day, even though they didn't follow instructions. Like these warnings put on the electronics. That's why I am only suggesting, like the OP that it's not at least an intuitive and sufficient way to inform someone that you're about to wipe their data, and the option itself is counter-intuitive. Not mentioning that part:
Simply deleting an entire directory tree with a command that's extremely prone to misunderstanding should not be virtualenv's job
Over and out
Which part of me admitting that it was totally my fault you didn't understand?
The part where you're complaining here.
Therefore, typing f.e. bin/python is quicker than .venv/bin/python
Consider investing in an auto venv activator like Yelp/aactivator, and making your venvs disposable again.
Bit of a late comment, but allow me to clarify some things and respond to some comments.
What's the reason why this can't be done on the odd occasion when a venv is cleared out unintentionally?
This is not at all about invoking venv --clear with the intent of refreshing and/or recreating a virtual env, imagine this very real scenario (unfortunately not too uncommon with tools, see 1 for an example)
# Invoked by {mistake, some tool, some _tool_, etc etc}
$ python -m venv ~
# `python -m venv --help` has clear, surely that just clears the env - PEBKAC of course
$ python -m venv --clear ~
# RIP.
Of course, none of this is directly venv's fault (except maybe the part where you can make a venv in a nonempty dir, but you can't get rid of it) - the cli help is...minimally...clear on what the option does.
[not] good practice to put your source code in a venv directory.
Certainly, and I have no problem with that, I imagine anyone doing so intentionally is doing it at their own discretion.
Of course, I understand that the number of people facing this issue is not nearly large enough to warrant introducing a breaking change (even if I personally don't agree with this option' behaviour), so perhaps it's better to add an option to actually purge venv files, and add that to the cli. win-win.
I actually don't mind a flag like, say --only, which only deletes the bin, lib and pyvenv.cfg files, and then only deletes the entire folder if it is empty.
Ideally that should have been the behaviour in --clear since the beginning, not very sure if adding an opt-in flag for safety will really help anyone though.
adding an opt-in flag for safety
Yes, adding an opt-in complementary flag is unlikely to help anyone.
I can't say for sure, but I think someone only skimming the help output/page is more likely to notice two flags (e.g. --clear and --purge) for seemingly the same purpose, and thus actually pay attention to the description of those flags.