Virtualenv gets deleted when I enable Poetry plugin

[intgr]

Describe the bug

Whenever I start IDEA with the Poetry plugin enabled, it starts indexing the virtualenv and within seconds the whole $VIRTUAL_ENV directory gets completely removed. No new virtualenv is created in its place. This only occurs in one project though.

1. IDEA complains "Invalid Python interpreter selected for the module".
2. I restore the virtualenv from backup
3. When I click on the "Configure Python Interpreter" on IDEA's notification, immediately it starts deleting files, which takes a few seconds until completely removed.
4. Go to 1

This started occurring in within the last week, I performed a 'brew update' and update to IDEA plugins. Until then I had been using the Poetry plugin without issues.

Disabling the Poetry plugin solves this issue.

I am not sure how to debug this further, nothing relevant appears in idea.log file. Is it possible to log all commands that the Poetry plugin invokes?

Environments (please complete the following information):

• IDE: IntelliJ IDEA 2020.3.1 (Ultimate Edition)
• OS: macOS 10.15.7
• Poetry Version 1.1.4
• Plugin version 1.0.1

[intgr]

I noticed that Poetry has some built-in "broken virtualenv detection", which I can trigger manually by deleting bin/python. But I don't know why the Poetry plugin would be triggering this behavior.

% rm $VIRTUAL_ENV/bin/python
% poetry shell
The virtual environment found in /.../Library/Caches/pypoetry/virtualenvs/idea-bugs-H1yE5bTC-py3.9 seems to be broken.
Recreating virtualenv idea-bugs-H1yE5bTC-py3.9 in /.../Library/Caches/pypoetry/virtualenvs/idea-bugs-H1yE5bTC-py3.9
Virtual environment already activated: /.../Library/Caches/pypoetry/virtualenvs/idea-bugs-H1yE5bTC-py3.9

Since I also have poetry installed within the virtualenv, it may fail to re-create itself after deleting.

[intgr]

I also tried deleting and re-creating the IDEA Python plugin interpreter configuration, but the problem persists.

[koxudaxi]

@intgr I'm sorry for the late reply.

I am not sure how to debug this further, nothing relevant appears in idea.log file. Is it possible to log all commands that the Poetry plugin invokes?

No. We can't save logs of the commands. We may add a feature to get logs.

I tested it on my environment close to your environment. But, I couldn't get the same error.

My environment IDE: IntelliJ IDEA 2020.3.1 (Ultimate Edition) OS: macOS 10.15.7 Poetry Version 1.1.4 Plugin version 1.0.2

Since I also have poetry installed within the virtualenv, it may fail to re-create itself after deleting.

I don't recommend to install poetry within the virtualenv. I think pipx is good tools to install poetry. https://python-poetry.org/docs/#installing-with-pipx

[intgr]

I don't recommend to install poetry within the virtualenv.

Yeah I would very much like to get rid of it, but it's currently a necessary workaround for us until this PR goes anywhere: https://github.com/python-poetry/poetry/pull/3349

[joelb123]

I can confirm this is an issue for me. It makes the plugin totally unusable. I install a virtualenv, and within seconds it gets auto-deleted: ``OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

• WINDOW DECORATIONS RELOADED
• 2021-01-12 10:03:02,932 [ 412] WARN - llij.ide.plugins.PluginManager - Resource bundle redefinition for plugin * 'com.jetbrains.pycharm.pro.customization'. Old value: messages.ActionsBundle, new value: messages.PyBundle *
• 2021-01-12 10:03:14,959 [ 12439] WARN - .skeletons.PySkeletonRefresher - Some skeletons failed to generate
• 2021-01-12 10:03:14,960 [ 12440] WARN - .skeletons.PySkeletonRefresher - Bio.PDB.QCPSuperimposer.qcprot
• 2021-01-12 10:03:14,960 [ 12440] WARN - .skeletons.PySkeletonRefresher - llvmlite.binding.libllvmlite
• 2021-01-12 10:03:14,960 [ 12440] WARN - .skeletons.PySkeletonRefresher - numba.np.ufunc.omppool
• 2021-01-12 10:03:14,960 [ 12440] WARN - .skeletons.PySkeletonRefresher - scipy.optimize.lsap
• 2021-01-12 10:11:53,098 [ 530578] WARN - tions.ex.ConfigurableCardPanel - auto-dispose 'Plugins' id=preferences.pluginManager
• 2021-01-12 10:11:53,100 [ 530580] WARN - tions.ex.ConfigurableCardPanel - auto-dispose 'Python Interpreter' id=com.jetbrains.python.configuration.PyActiveSdkModuleConfigurable
• 2021-01-12 10:11:53,495 [ 530975] WARN - com.intellij.util.xmlb.Binding - no accessors for org.jetbrains.idea.perforce.perforce.ConnectionId

I believe it's the auto-dispose of the python interpreter that's doing the crime here.

Environment: Poetry Version 1.1.4 installed in external environment Plugin Version 1.0.2 I used 1.0.1 extensively and it did not do this under Pycharm 3.2.0 Pycharm 3.2.2 Gentoo Linux kernel 5.10.6 OpenJDK 64-bit server built by JetBrains

[joelb123]

I reinstalled plugin version 1.0.1 and verified that the auto-dispose of the python interpreter does not happen with the plugin enabled.

[wgordon17]

This is biting me now too!

PyCharm 2020.3.4 (Professional Edition)
Build #PY-203.7717.65, built on March 16, 2021
Runtime version: 11.0.10+8-b1145.96 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 10.16
GC: ParNew, ConcurrentMarkSweep
Memory: 4416M
Cores: 12
Non-Bundled Plugins: org.intellij.plugins.postcss, com.khmelyuk.multirun, com.paperetto.dash, fr.mary.olivier.aw-watcher, ideanginx9, net.seesharpsoft.intellij.plugins.csv, mobi.hsz.idea.gitignore, org.toml.lang, NodeJS, com.intellij.plugins.html.instantEditing, me.kapien.vue-router-helper, org.jetbrains.plugins.vue, com.jetbrains.intellij.datalore, ru.adelf.idea.dotenv, ru.meanmail.plugin.requirements

$ poetry debug info

Poetry
Version: 1.1.5
Python:  3.9.1

Virtualenv
Python:         3.9.1
Implementation: CPython
Path:           NA

System
Platform: darwin
OS:       posix
Python:   /Users/wgordon/.pyenv/versions/3.9.1

Any advice or recommendation would be greatly appreciated!

[wgordon17]

I seem to be able to hit this consistently by removing the python interpreter from PyCharm (I've confirmed that removing the interpreter from PyCharm doesn't touch the directory on disk), and then re-adding it. The deletion appears to occur once I hit "OK" and during the "Loading Packages L..." phase that I see along the bottom of the PyCharm window.

Going from 1.0.6, all the way down to 1.0.0, I was still able to consistently reproduce this issue. I've also confirmed that this seems to just be affected by the Poetry plugin. If I add the same interpreter as a "Virtualenv Environment", I don't see the same behavior.

[koxudaxi]

@wgordon17 Thank you for show me the detail. I can't reproduce the problem 🤔

Going from 1.0.6, all the way down to 1.0.0, I was still able to consistently reproduce this issue.

I just checked the histories of changes. But, you said the changes are not related :(

[wgordon17]

Weird! Is there any way to enable debugging or audit logging in PyCharm? I'm happy to provide any additional information if necessary!

[koxudaxi]

Weird! Is there any way to enable debugging or audit logging in PyCharm? I'm happy to provide any additional information if necessary!

Thank you.

The simple answer is nothing :( Of course, you can run a plugin from a source with a debugger in IDEA. But, this way is not easy.

Would you tell me how do you reproduce it?

My environment Poetry: 1.1.5 PyCharm: 2020.3.4 Python: 3.9.2 macOS: 10.15.7

[joelb123]

This might be a complete red herring, but the other problem that was driving me crazy was JCEF crashing with the ReStructuredText plugin enabled due to problems accessing the GPU. I solved that by using the system java rather than bundled; I believe that may have solved the this problem, but I'm not in a situation to test this atm.

[koxudaxi]

Thank you for the comment. The viewpoint may be good. Because @wgordon17 said, the problem happened in 1.0.0. The version has the same similar implementation as Pipenv integration in PyCharm. It means it does not include the additional features for only poetry.

[koxudaxi]

@sproshev Could you give us some advice on the problem?

In the latest version, I had added some features to checking poetry executable. The feature is sync function that blocks PyCharm. I was afraid the feature may cause the problem. But, The problem happened older version too which doesn't have the feature.

[wgordon17]

I hope this helps https://www.youtube.com/watch?v=Xt5WEfaYzrA

I'm really not sure where the issue is ¯\_(ツ)_/¯ At first, I was totally clueless, so then I used opensnoop to track it down. I could clearly see python3.9 creating things (presumably poetry install), and then I would see a single -1 from pycharm, which I can only assume means "delete".

Once I realized it was PyCharm I updated to 2020.3.4. And finally, at my whits end, I stumbled across this issue. I honestly don't recall this being an issue in the past, but then again, how often does someone switch out there Project Interpreter. It definitely seems to be an issue with older versions of your Poetry plugin, but whether it's an interaction with newer versions of PyCharm...who the heck knows ¯\_(ツ)_/¯

I'm at least back to productive again using the "Virtualenv Environment" interpreter, but I definitely miss your plugin's "magic" ;)

[koxudaxi]

@wgordon17 Thank you for uploading the video. Today, I tried to reproduce your problem on my machine. But, I can't see it. Also, I have never seen the error. But, I don't give up to fix the problem.

[sproshev]

@koxudaxi we have a similar issue with pipenv https://youtrack.jetbrains.com/issue/PY-36526 but I'm not sure if it is actually related.

[koxudaxi]

@sproshev Thank you for the link. I will check the issue.

[svenski]

The behaviour described in here started happening for me today, out of the blue. A poetry install from the command line resets the environment but as soon as I open pycharm, it get's deleted. A workaround that has worked so far is to re-create the poetry env from within pycharm. It survived a re-start.

Peotry Pycharm Plugin version: 1.1.4

Poetry version 1.1.6

Pycharm Version PyCharm 2021.1.2 (Professional Edition) Build #PY-211.7442.45, built on June 1, 2021

Runtime version: 11.0.11+9-b1341.57 amd64 VM: Dynamic Code Evolution 64-Bit Server VM by JetBrains s.r.o. Linux 5.8.0-59-generic GC: G1 Young Generation, G1 Old Generation Memory: 2048M Cores: 4 Non-Bundled Plugins: IdeaVIM (0.68), Key Promoter X (2021.1.1), net.seesharpsoft.intellij.plugins.csv (2.17.0), com.koxudaxi.poetry (1.1.4), com.koxudaxi.pydantic (0.3.3) Current Desktop: Regolith:GNOME-Flashback:GNOME

[phillipuniverse]

Also ran into this today. Thanks to @svenski this workaround worked!

A workaround that has worked so far is to re-create the poetry env from within pycharm. It survived a re-start.

Allowing Pycharm to create the Poetry venv allowed me to utilize poetry from both Pycharm and the command line (e.g. poetry env info).

PyCharm 2021.2.2 (Professional Edition)
Build #PY-212.5284.44, built on September 14, 2021
Licensed to Shipwell / Phillip Verheyden
Subscription is active until February 20, 2022.
Runtime version: 11.0.12+7-b1504.28 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 11.5.2
GC: ParNew, ConcurrentMarkSweep
Memory: 4012M
Cores: 12
Non-Bundled Plugins: org.sonarlint.idea (5.3.0.36775), com.github.hinaser.gfm-advanced (0.0.8), com.github.redfoos.logstash-intellij-plugin (0.65-EAP), org.intellij.plugins.hcl (0.7.14), name.kropp.intellij.makefile (212.4746.52), ski.chrzanow.foldableprojectview (1.1.1), org.asciidoctor.intellij.asciidoc (0.33.19), mobi.hsz.idea.gitignore (4.2.0), com.intellij.swagger (212.5080.16), com.dmarcotte.handlebars (212.4746.57), org.toml.lang (0.2.155.4114-212), org.rust.lang (0.4.155.4114-212), com.koxudaxi.poetry (1.1.5-212), com.jetbrains.intellij.datalore (0.1.13-212), ru.meanmail.plugin.requirements (2021.4.1-212), com.leinardi.pycharm.pylint (0.12.2), ru.adelf.idea.dotenv (2021.3.0.212), com.koxudaxi.pydantic (0.3.8), com.dropbox.plugins.mypy_plugin (0.3.2), aws.toolkit (1.32-212)