pygradle
pygradle copied to clipboard
linkedin
Newer versions of the sdist plugin do not resolve pytz properly
When using com.linkedin.python-sdist > 0.4.9 the build always fails inside of installBuildRequirements with
Processing /home/vagrant/.gradle/caches/modules-2/files-2.1/pypi/Sphinx/1.4.1/d18b856710b22ae9740147e21754ca5b851af9b2/Sphinx-1.4.1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-1phE1A-build/setup.py", line 79, in <module>
from babel.messages.pofile import read_po
File "/home/vagrant/dev/xxx/build/venv/local/lib/python2.7/site-packages/babel/messages/__init__.py", line 12, in <module>
from babel.messages.catalog import *
File "/home/vagrant/dev/xxx/build/venv/local/lib/python2.7/site-packages/babel/messages/catalog.py", line 23, in <module>
from babel.dates import format_datetime
File "/home/vagrant/dev/xxx/build/venv/local/lib/python2.7/site-packages/babel/dates.py", line 28, in <module>
from babel.util import UTC, LOCALTZ
File "/home/vagrant/dev/xxx/build/venv/local/lib/python2.7/site-packages/babel/util.py", line 282, in <module>
UTC = _pytz.utc
AttributeError: 'module' object has no attribute 'utc'
The latest versions pin Sphinx at 1.4.9 and your build is installing 1.4.1 according to the log above. Are you perhaps pinning it at that version?
Did you try running a clean and then rebuild? We do not see this in our builds.
This is a clean build, nothing is pinned. This takes place inside of installBuildRequirements, so it is just doing the requirements for pygradle. My project does not need python, and does not have docs. In fact, I have tasks.packageDocs.enabled = false. Kind of odd it still wants to pull down Sphinx. Gradle version is 3.4.
Again, how are you getting Sphinx-1.4.1 with the latest pygradle (0.6.14)? We pin it at 1.4.9, so it's not clear how are you getting 1.4.1. Can you provide some steps to try and reproduce this? We cannot reproduce on our end.
Regarding tasks.packageDocs.enabled = false, that only disables the task for packaging docs. (You may want to disable building docs too.) The use of Sphinx is disabled with such settings. However, Sphinx is still a dependency in the build configuration and gets downloaded because of that. It would have to be excluded from the build configuration to remove it from dependencies.
Steps to reproduce:
- Clone the pygradle project.
- cd to the example-project folder.
- Update the version number on sdist to 0.6.12 (which is the latest)
- Follow README.
I pushed 0.6.14 to plugins.gradle.org. Thanks. We'll investigate and let you know.
I'm having this same issue as well.
I've tried a range of versions from 0.5.7 through 0.7.2 tried using Gradle 4.0 and 4.6 all combinations have the same issue.
Reverting back to 0.4.9 on Gradle 3.5 solves it for me.
I had a reason to build another internal wheel at my company so I took another run at this. This is the dependency tree before I specified anything.
$ ./gradlew dependencies
> Task :dependencies
------------------------------------------------------------
Root project
------------------------------------------------------------
archives - Configuration for archive artifacts.
No dependencies
build
+--- pypi:flake8:2.6.2
| +--- pypi:pyflakes:0.8.1
| +--- pypi:pycodestyle:2.0.0
| \--- pypi:mccabe:0.2.1
+--- pypi:Sphinx:1.4.9
| +--- pypi:six:1.5.0 -> 1.11.0
| +--- pypi:Jinja2:2.3
| +--- pypi:Pygments:2.0
| +--- pypi:docutils:0.11
| +--- pypi:snowballstemmer:1.1.0
| +--- pypi:Babel:1.3
| | \--- pypi:pytz:2004b.2
| +--- pypi:alabaster:0.7.1
| +--- pypi:imagesize:0.7.1
| \--- pypi:colorama:0.3.5
\--- pypi:pex:1.2.13
+--- pypi:setuptools:20.3 -> 33.1.1
\--- pypi:wheel:0.26.0 -> 0.29.0
\--- pypi:argparse:1.4.0
default - Configuration for default artifacts.
No dependencies
pydocs
No dependencies
pygradleBootstrap
\--- pypi:virtualenv:15.1.0
python
No dependencies
setupRequires
+--- pypi:setuptools:33.1.1
+--- pypi:wheel:0.29.0
| \--- pypi:argparse:1.4.0
+--- pypi:pip:9.0.3
\--- pypi:setuptools-git:1.2
test
+--- pypi:pytest:3.1.2
| +--- pypi:py:1.4.33
| +--- pypi:setuptools:36.0.1 -> 33.1.1
| +--- pypi:argparse:1.4.0
| \--- pypi:colorama:0.3.9
+--- pypi:pytest-cov:2.5.1
| +--- pypi:pytest:2.6.0 -> 3.1.2 (*)
| \--- pypi:coverage:3.7.1
\--- pypi:pytest-xdist:1.17.1
+--- pypi:execnet:1.1
+--- pypi:pytest:2.7.0 -> 3.1.2 (*)
\--- pypi:py:1.4.22 -> 1.4.33
venv
No dependencies
wheel
No dependencies
(*) - dependencies omitted (listed previously)
The troublesome pypi:pytz:2004b.2 is already in there. So I started looking at setup.py files. Sphix has a dependancy 'babel>=1.3,!=2.0', there is no version of Babel between 1.3 and 2.0 so this may as well be babel==1.3 Babel 1.3 has a dependancy of pytz>=0a, (which is still there right up to Babel 2.6.0).
After reverting back to my previously mentioned 'safe place' of 0.4.9 on Gradle 3.5 this is the dependencies output.
$ ./gradlew dependencies
Build number is 20180608085444
:dependencies
------------------------------------------------------------
Root project
------------------------------------------------------------
archives - Configuration for archive artifacts.
No dependencies
build
+--- pypi:flake8:2.5.4
| +--- pypi:pyflakes:0.8.1
| +--- pypi:pep8:1.5.7
| \--- pypi:mccabe:0.2.1
\--- pypi:Sphinx:1.4.1
+--- pypi:six:1.4.0 -> 1.10.0
+--- pypi:Jinja2:2.3
+--- pypi:Pygments:2.0
+--- pypi:docutils:0.11
+--- pypi:snowballstemmer:1.1.0
+--- pypi:Babel:1.3
| \--- pypi:pytz:2004b.2
+--- pypi:alabaster:0.7.1
+--- pypi:imagesize:0.7.1
\--- pypi:colorama:0.3.5
default - Configuration for default artifacts.
No dependencies
pydocs
No dependencies
pygradleBootstrap
\--- pypi:virtualenv:15.0.1
python
No dependencies
setupRequires
+--- pypi:wheel:0.26.0
| \--- pypi:argparse:1.4.0
+--- pypi:setuptools:19.1.1
+--- pypi:pip:7.1.2
+--- pypi:setuptools-git:1.1
\--- pypi:pbr:1.8.0
test
+--- pypi:pytest:2.9.1
| +--- pypi:py:1.4.29
| +--- pypi:colorama:0.3.9
| \--- pypi:argparse:1.4.0
+--- pypi:pytest-cov:2.2.1
| +--- pypi:pytest:2.6.0 -> 2.9.1 (*)
| \--- pypi:coverage:3.7.1
\--- pypi:pytest-xdist:1.14
venv
No dependencies
wheel
No dependencies
(*) - dependencies omitted (listed previously)
I tried putting in replacements for pytz to bring it up to date, but it didn't actually change what version was installed.
For whatever it's worth, even in my 'safe place' config, if I attempt to auto generate docs, (make a docs directory) I hit this problem as well.
Created a pull request over on Babel to bump their pytz version requirement https://github.com/python-babel/babel/pull/592
Huh. What sort of dependency resolver is in use here? I don't see why pytz>=0a would end up choosing a 2004 version.
That said, the Babel PR is fine, but I think there's a bug in the resolver here too.
@akx Gradle is the resolver in use here. For the record, I don't think the pytz change is the root of the problem, it was just kind of muddying the waters for me. And given how easy of a fix it is, I thought it was worth the time to send in the PR.
I'm starting to lean towards an issue with our internal ivy repo we're resolving against as the root of the problem. The reason being, if I include with the pygradle demo repo in addition our own repo the problem disappears. Still digging though, hoping to find something clear soon.
What I was experiencing was an implementation problem on our side.
Internally we have a task built on top of pygradle/pivy-importer/src/main/groovy/com/linkedin/python/importer/ImporterCLI.java that publishes to a local repo.
But we weren't passing in the 'latest' flag, so it was going out and finding the oldest possible version of pytz.
https://github.com/linkedin/pygradle/blob/dfb28e25e914336a9999e02e0f585ccc03548c61/pivy-importer/src/main/groovy/com/linkedin/python/importer/distribution/SourceDistPackage.groovy#L148
Fixing that seems to have resolved the issue for me. Not sure if it's the same root problem as @bmellish had