Support Other Than Amazon Hosts ?

[diginfo]

Just tried to use digital ocean's spaces but it failed:

s4cmd ls
[Runtime Exception] An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.
Traceback (most recent call last):
  File "/usr/local/bin/s4cmd.py", line 1928, in main
    CommandHandler(opt).run(args)
  File "/usr/local/bin/s4cmd.py", line 1557, in run
    CommandHandler.__dict__[cmd + '_handler'](self, args)
  File "/usr/local/bin/s4cmd.py", line 129, in wrapper
    ret = func(*args, **kargs)
  File "/usr/local/bin/s4cmd.py", line 1628, in ls_handler
    self.pretty_print(self.s3handler().list_buckets())
  File "/usr/local/bin/s4cmd.py", line 129, in wrapper
    ret = func(*args, **kargs)
  File "/usr/local/bin/s4cmd.py", line 694, in list_buckets
    for bucket in self.s3.list_buckets().get('Buckets') or []:
  File "/usr/local/bin/s4cmd.py", line 402, in wrapped_method
    ret = getattr(self.client, method)(*args, **merged_kargs)
  File "/usr/local/lib/python3.7/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.

Are other hosts supported ?

[supergeeky]

Take a look at the --endpoint-url option. It lets me use s4cmd with an on premises S3 device.

[skunkworker]

@diginfo If anyone comes here from google, here is how you can use digitalocean spaces. My ~/.s3cfg was configured fine for s3cmd but s4cmd didn't pick up the correct host.

replace sfo2.digitaloceanspaces.com with your digital ocean spaces host.

s4cmd --endpoint-url sfo2.digitaloceanspaces.com sync s3://bucket1/ s3://bucket2/

[turicas]

Duplicate of #158.

[moreinhardt]

Unfortunately specifying --endpoint-url like suggested by @skunkworker does not work for me :confused:

[Runtime Exception] Unable to connect to s3: Invalid endpoint: ams3.digitaloceanspaces.com
Traceback (most recent call last):
  File "/usr/bin/s4cmd", line 687, in connect
    self.s3 = BotoClient(self.opt)
  File "/usr/bin/s4cmd", line 387, in __init__
    self.client = self.boto3.client('s3', endpoint_url=opt.endpoint_url)
  File "/usr/lib/python3/dist-packages/boto3/__init__.py", line 91, in client
    return _get_default_session().client(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/boto3/session.py", line 258, in client
    return self._session.create_client(
  File "/usr/lib/python3/dist-packages/botocore/session.py", line 831, in create_client
    client = client_creator.create_client(
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 83, in create_client
    client_args = self._get_client_args(
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 285, in _get_client_args
    return args_creator.get_client_args(
  File "/usr/lib/python3/dist-packages/botocore/args.py", line 99, in get_client_args
    endpoint = endpoint_creator.create_endpoint(
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 286, in create_endpoint
    raise ValueError("Invalid endpoint: %s" % endpoint_url)
ValueError: Invalid endpoint: ams3.digitaloceanspaces.com

[davidparks21]

Unfortunately specifying --endpoint-url like suggested by @skunkworker does not work for me confused

I think you just need https://ams3.digitaloceanspaces.com, I guess you missed the https:// part.