duplicity icon indicating copy to clipboard operation
duplicity copied to clipboard

S3 location in Frankfurt must use

Open ManuelB opened this issue 9 years ago • 2 comments

When using the frankfurt location of S3 it is not working because version 4 of the signature algorithm has to be used. The following work arounds must be applied:

Update boto and duplicity to most recent versions.

Make sure boto uses sigv4

$ cat .boto 
[s3]
use-sigv4 = True

Afterwards you will receive this:

$  /usr/bin/duplicity collection-status --include=/home/manuel/.cache/deja-dup/metadata --exclude=/home/manuel/.local/share/Trash --exclude=/home/manuel/.xsession-errors --exclude=/home/manuel/.thumbnails --exclude=/home/manuel/.cache/deja-dup --exclude=/home/manuel/.cache --include=/home/manuel --exclude=/sys --exclude=/run --exclude=/proc --exclude=/var/tmp --exclude=/tmp --exclude=** --s3-use-new-style s3+http://s3-eu-central-1.amazonaws.com/manuel-backup-xps-13 --no-encryption --verbosity=9 --gpg-options=--no-use-agent --archive-dir=/home/manuel/.cache/deja-dup --tempdir=/tmp --s3-european-buckets
...
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1523, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1517, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1355, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 1093, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 209, in get_backend_object
    return factory(pu)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 161, in __init__
    self.resetConnection()
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 183, in resetConnection
    self.conn = get_connection(self.scheme, self.parsed_url, self.storage_uri)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 95, in get_connection
    conn = storage_uri.connect(is_secure=(not globals.s3_unencrypted_connection))
  File "/usr/local/lib/python2.7/dist-packages/boto/storage_uri.py", line 117, in connect
    **connection_args)
  File "/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py", line 196, in __init__
    "When using SigV4, you must specify a 'host' parameter."
HostRequiredError: BotoClientError: When using SigV4, you must specify a 'host' parameter.

Then you have to patch:

root@manuel-XPS-13-9343:/usr/lib/python2.7/dist-packages/duplicity/backends# diff -u _boto_single.py.orig _boto_single.py
--- _boto_single.py.orig    2015-06-21 18:14:21.510061758 +0200
+++ _boto_single.py 2015-06-21 18:09:22.054067121 +0200
@@ -21,6 +21,7 @@

 import os
 import time
+import socket

 import duplicity.backend
 from duplicity import globals
@@ -92,7 +93,7 @@

     if not parsed_url.hostname:
         # Use the default host.
-        conn = storage_uri.connect(is_secure=(not globals.s3_unencrypted_connection))
+        conn = storage_uri.connect(host=socket.gethostname(),is_secure=(not globals.s3_unencrypted_connection))
     else:
         assert scheme == 's3'
         conn = storage_uri.connect(host=parsed_url.hostname,

ManuelB avatar Jun 21 '15 16:06 ManuelB

All the above won't solve the problem in the end I used a Bucket in Ireland instead of Frankfurt

ManuelB avatar Jun 21 '15 16:06 ManuelB

Frankfurt buckets still don't work. When will this be fixed?

aggsol avatar Feb 19 '16 22:02 aggsol