pycontribs
pyrax via pypi breaks duplicity
Hello pyrax maintainers!
It looks like recent versions of pyrax in pypi break integration with duplicity 0.6.24-1.
jenkins@fff8561e79b2:/$ /usr/bin/duplicity --no-encryption restore cf+http://my_awesome_files_bucket tmp
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1509, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1503, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1336, in main
action = commandline.ProcessCommandLine(sys.argv[1:])
File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 1059, in ProcessCommandLine
backup, local_pathname = set_backend(args[0], args[1])
File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 952, in set_backend
globals.backend = backend.get_backend(bend)
File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 163, in get_backend
return _backends[pu.scheme](pu)
File "/usr/lib/python2.7/dist-packages/duplicity/backends/_cf_pyrax.py", line 67, in __init__
% (e.__class__.__name__, str(e)),
TypeError: __str__ returned non-string (type Error)
jenkins@fff8561e79b2:/$ pip freeze | sort
Babel==2.2.0
bzr==2.7.0.dev1
cffi==0.8.6
chardet==2.3.0
cliff==1.15.0
cmd2==0.6.8
colorama==0.3.2
configobj==5.0.6
cryptography==0.6.1
debtcollector==1.2.0
dh-virtualenv==0.11
docutils==0.12
duplicity==0.6.24
ecdsa==0.11
funcsigs==0.4
html5lib==0.999
ip-associations-python-novaclient-ext==0.1
iso8601==0.1.11
jenkins-job-builder==1.4.0
jenkins-jobs-slack==0.3.2
jenkins-view-builder==0.5.0
Jinja2==2.7.3
keyring==8.2
keystoneauth1==2.2.0
lockfile==0.8
lxml==3.4.0
MarkupSafe==0.23
mercurial==3.1.2
mock==1.0.1
monotonic==0.6
msgpack-python==0.4.7
multi-key-dict==2.0.3
ndg-httpsclient==0.3.2
netaddr==0.7.18
netifaces==0.10.4
oauthlib==0.6.3
ordereddict==1.1
os-diskconfig-python-novaclient-ext==0.1.3
oslo.config==3.4.0
oslo.i18n==3.2.0
oslo.serialization==2.2.0
oslo.utils==3.4.0
os-networksv2-python-novaclient-ext==0.25
os-virtual-interfacesv2-python-novaclient-ext==0.19
paramiko==1.15.1
pbr==1.8.1
Pillow==2.6.1
ply==3.4
prettytable==0.7.2
pyasn1==0.1.7
pycparser==2.10
pycrypto==2.6.1
Pygments==2.0.1
PyJWT==0.2.1
pyOpenSSL==0.14
pyparsing==2.0.7
pyrax==1.9.7
python-jenkins==0.4.12
python-keystoneclient==2.1.1
python-novaclient==2.27.0
pytz==2015.7
PyYAML==3.11
rackspace-auth-openstack==1.3
rackspace-novaclient==1.5
rax-default-network-flags-python-novaclient-ext==0.3.2
rax-scheduled-images-python-novaclient-ext==0.3.1
requests==2.4.3
roman==2.0.0
simplejson==3.8.1
six==1.8.0
Sphinx==1.2.3
stevedore==1.10.0
unicodecsv==0.14.1
urllib3==1.9.1
virtualenv==1.11.6
wheel==0.24.0
wrapt==1.10.6
This manifests whether pyrax versions 1.9.6 and 1.9.7 are installed via pypi. However if I pip install off HEAD in master (currently 8b55db5), things are a-ok.
jenkins@c4df005fcfcd:/$ pip freeze | sort
Babel==2.2.0
bzr==2.7.0.dev1
cffi==0.8.6
chardet==2.3.0
cliff==1.15.0
cmd2==0.6.8
colorama==0.3.2
configobj==5.0.6
cryptography==0.6.1
debtcollector==1.2.0
dh-virtualenv==0.11
docutils==0.12
duplicity==0.6.24
ecdsa==0.11
funcsigs==0.4
html5lib==0.999
ip-associations-python-novaclient-ext==0.1
iso8601==0.1.11
jenkins-job-builder==1.4.0
jenkins-jobs-slack==0.3.2
jenkins-view-builder==0.5.0
Jinja2==2.7.3
keyring==8.2
keystoneauth1==2.2.0
lockfile==0.8
lxml==3.4.0
MarkupSafe==0.23
mercurial==3.1.2
mock==1.3.0
monotonic==0.6
msgpack-python==0.4.7
multi-key-dict==2.0.3
ndg-httpsclient==0.3.2
netaddr==0.7.18
netifaces==0.10.4
oauthlib==0.6.3
ordereddict==1.1
os-diskconfig-python-novaclient-ext==0.1.3
oslo.config==3.4.0
oslo.i18n==3.2.0
oslo.serialization==2.2.0
oslo.utils==3.4.0
os-networksv2-python-novaclient-ext==0.25
os-virtual-interfacesv2-python-novaclient-ext==0.19
paramiko==1.15.1
pbr==1.8.1
Pillow==2.6.1
ply==3.4
prettytable==0.7.2
pyasn1==0.1.7
pycparser==2.10
pycrypto==2.6.1
Pygments==2.0.1
PyJWT==0.2.1
pyOpenSSL==0.14
pyparsing==2.0.7
pyrax==1.9.6
python-jenkins==0.4.12
python-keystoneclient==2.1.2
python-novaclient==2.27.0
pytz==2015.7
PyYAML==3.11
rackspace-auth-openstack==1.3
rackspace-novaclient==1.5
rax-default-network-flags-python-novaclient-ext==0.3.2
rax-scheduled-images-python-novaclient-ext==0.3.1
requests==2.9.1
roman==2.0.0
simplejson==3.8.1
six==1.10.0
Sphinx==1.2.3
stevedore==1.10.0
unicodecsv==0.14.1
urllib3==1.9.1
virtualenv==1.11.6
wheel==0.24.0
wrapt==1.10.6
Package diffs between a pypi install vs git+https install:
$ diff /tmp/{1.9.7,master}
29c29
< mock==1.0.1
---
> mock==1.3.0
57c57
< pyrax==1.9.7
---
> pyrax==1.9.6
59c59
< python-keystoneclient==2.1.1
---
> python-keystoneclient==2.1.2
67c67
< requests==2.4.3
---
> requests==2.9.1
70c70
< six==1.8.0
---
> six==1.10.0
I tried comparing master to the v1.9.6 tag in GitHub but somethings fishy showing no differences. Also there doesnt appear to be a v1.9.7 tag in GitHub.
master, v1.9.6 and v1.9.7 are all the same at the moment. There was a packaging issue causing us to bump to 1.9.7 and re-release.
Based on all of those being the same, I am not sure why there would be any difference. Their setup.py files all indicate the exact same dependencies.
Those dependencies are listed as:
"python-novaclient==2.27.0",
"rackspace-novaclient",
"keyring",
"requests>=2.2.1,<3",
"six>=1.9.0,<2",
The only thing I can see there is that your install of 1.9.7 is somehow installing six==1.8.0 which does not satisfy the pyrax dependency.
We run into this in a Jenkins Docker container. The relevant pip snippets from the Dockerfile are as follows:
# Upgrade Python tools
RUN pip install pip==7.1.2
RUN pip install setuptools==18.2
RUN cp /usr/local/bin/pip /usr/bin/pip
RUN cp /usr/local/bin/pip /usr/bin/pip2
...
RUN pip install jenkins-job-builder jenkins-view-builder jenkins-jobs-slack
RUN pip install --upgrade git+https://github.com/rackspace/pyrax.git
That allows us to get duplicity working.
We have run into issues in the past with pbr (which is a dep of the upstream openstack modules) causing deps to be installed that do not satisfy the pyrax specifications. This may be due to pyrax not being reliant on pbr.
In most cases, a second pip install --upgrade pyrax will usually resolve any dependency inconsistencies.