checkPython fails on macOS

[ansman]

I'm trying to gtr started with this project but checkPython keeps failing. This seems to be because it assumes that the binary will be located at .gradle/python/bin/python but virtualenv puts it in .gradle/python/usr/local/bin/python.

I'm using python 3.10.6 and gradle-use-python-plugin version 3.0

> Task :checkPython
Using python 3.10.6 from /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.10 (python3)
Using pip 22.2.2 from /usr/local/lib/python3.10/site-packages/pip (python 3.10)
Using virtualenv /usr/local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
virtualenv 20.4.2 from /Users/nicklas/Library/Python/3.10/lib/python/site-packages/virtualenv/__init__.py (in '.gradle/python')
[python] python3 -m virtualenv .gradle/python
         /usr/local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
           warnings.warn(
         created virtual environment CPython3.10.6.final.0-64 in 340ms
           creator CPython3Posix(dest=/Users/nicklas/Repositories/personal/eagger/.gradle/python, clear=False, no_vcs_ignore=False, global=False)
           seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/nicklas/Library/Application Support/virtualenv)
             added seed packages: pip==21.0.1, setuptools==52.0.0, wheel==0.37.1
           activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

> Task :checkPython FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkPython'.
> Python not found: .gradle/python/bin/python. This must be a bug of virtualenv support, please report it (https://github.com/xvik/gradle-use-python-plugin/issues). You can disable virtualenv usage with 'python.scope = USER'.

[xvik]

I havn't seen environments created like this (.gradle/python/usr/local/bin/python) before and didn't find any related issues by quick googling.

I suspect virtulenv was installed by plugin (20.4.2), but plugin intentionally use older virtualenv for a wider range of supported python configurations. You can try to update virtualenv with pip3 install -U virtualenv. After that current environment must be cleared with cleanPython task and then, you can try to create it again.

If virtualenv upgrade would help, you can configure higher version in plugin python.virtualenvVersion = '20.16.7' to avoid problems in the future.

If this would not help, then you can only use user scope (python.scope = USER) instead of virtualenv or use docker (if you have docker installed): python.docker.use = true (in this case virtualenv would be created, but using python inside docker).

Please tell me about your results. I will try to investigate this too, but usually such cases are hard to reproduce and I need some clues

[ansman]

I did bump up virtualenv in the buildscript files but it did not work. Upgrading virtualenv manually using pip3 install virtualenv --upgrade did work though!

[xvik]

python.virtualenvVersion is taken into account only if there is no virtualenv installed - plugin does not re-install it (assumming that it might be installed manually and with exact version user need; auto installation in plugin is only for case when no virtualenv installed at all). I was suggesting to update this setting only to make sure good virtualenv version would be installed next time on some fresh environment.

Glad that it works eventually. Compatibility is a hard topic for python. I will update the default virtualenv version with the next release (even though it will drop some old pythons support)

[mfriess2]

I'm having the same problem and tried it out like this:

python {
	virtualenvVersion = '20.23.1'
}

mkdocs {
	sourcesDir = ''
}

But it won't work (I even deleted the .gradle folder)! Already went through the documentations but can't find what I'm doing wrong. And there seems to be no example which combines both topics :-(

Did anyone get it working? To override the virtualenv version?

[xvik]

@mfriess2 please post gradle output here.

Original issue was solved by manual virtualenv update pip3 install virtualenv --upgrade - might help in your case too. Also, if you have docker installed, you can use docker instead of local python.

These are just possible workarounds. Wihtout seeing log I can't suggest the exact reason.

[mfriess2]

Thanks for your help!

Here is my output:

./gradlew clean mkdocsServe
Starting a Gradle Daemon, 3 stopped Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.

> Task :checkPython
Using python 3.11.4 from /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11 (python3)
Using pip 23.0.1 from /opt/homebrew/lib/python3.11/site-packages/pip (python 3.11)
Using virtualenv /opt/homebrew/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
virtualenv 20.4.2 from /Users/<UNAME>/Library/Python/3.11/lib/python/site-packages/virtualenv/__init__.py (in '.gradle/python')
[python] python3 -m virtualenv .gradle/python
         /opt/homebrew/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
           warnings.warn(
         created virtual environment CPython3.11.4.final.0-64 in 3452ms
           creator CPython3Posix(dest=/Users/<UNAME>/<PRJ_DIR>/.gradle/python, clear=False, no_vcs_ignore=False, global=False)
           seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/<UNAME>/Library/Application Support/virtualenv)
             added seed packages: pip==23.1.2, setuptools==67.8.0, wheel==0.40.0
           activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

> Task :checkPython FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkPython'.
> Python not found: .gradle/python/bin/python. This must be a bug of virtualenv support, please report it (https://github.com/xvik/gradle-use-python-plugin/issues). You can disable virtualenv usage with 'python.scope = USER'.

...

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

See https://docs.gradle.org/7.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 12s

[xvik]

Thank you for the log! Indeed, the same as the original issue. Then you can fix it now only with manual virtualenv upgrade:

python3 -m pip install virtualenv --upgrade

After that, remove .gradle/python folder in project (or using cleanPython task) and, with more recent virtualenv, environment should be created correctly.

python.virtualenvVersion = '20.23.1' does not help because virtualenv is installed only if there is no installed virtualenv already. So changing this setting after plugin execution does not change anything. But if you'll remove virtualenv manually (pip3 uninstall virtualenv) then plugin would attempt to install it again on next execution (this time using specified version).

I will update the default virtualenv version in the next plugin version to avoid such situations. Ideally, I should switch to venv instead of virtualenv (because venv is pre-installed in python now), but can't promise this very soon.

[mfriess2]

Okay, I think I have it working now!

First, I installed it manually like you proposed with pip3 install virtualenv --upgrade. Then I wanted to test it once more without that and resetted everything:

rm -rfi .gradle
pip3 uninstall virtualenv

After that my build.gradle worked:

python {
	virtualenvVersion = '20.23.1'
}

mkdocs {
	sourcesDir = ''
}

Thanks for your help and commitment!

[zubri]

I faced similar issues after migrating to M2, where the installation of python is located in /opt/homebrew/opt/[email protected]

And virtualenv creates the bin and lib within .gradle/python/opt/homebrew instead of just .gradle/python/

I could not find a proper solution, so my workaround was just removing the invalid parent opt/homebrew directory

[xvik]

The problem should be in too old virtualenv version, installed by plugin (which improperly creates project environment). Manual update should fix the problem in general: pip3 install virtualenv --upgrade.

I would rise up default virtualenv version in the plugin or even add venv support (to simplify setup)

[zubri]

Thanks for the follow up. In my case pip3 install virtualenv --upgrade did not solve it. And I'd rather not put a fixed version in build.gradle as suggested above, to avoid impacting other environments but my local.

[xvik]

4.0.0 released with venv support by default, which should workaround this problem entirely. All old environments, created with virtualenv would still work normally (no need to re-create them).

Venv support could be disabled with python.useVenv = false and plugin would fall back to virtualenv usage (virtualenv would also be used if venv not found).

Virtualenv version, installed by default was also increased so problem should not appear anymore without venv too.