Restore with AWS_ENDPOINT (not relying on config)

[vanschelven]

The litestream restore command, when invoked directly with a s3://bucket/path URL and no config file, cannot access custom S3-compatible storage (e.g., Hetzner) because there is no way to specify the endpoint. This makes it impossible to restore from non-AWS S3 without defining a replica in a config file.

Reproduction

AWS_ACCESS_KEY_ID=... \
AWS_SECRET_ACCESS_KEY=... \
AWS_ENDPOINT=https://fsn1.your-objectstorage.com \
litestream restore -o /tmp/db.sqlite3 -if-db-not-exists s3://mybucket/mypath

This fails with:

cannot lookup bucket region: InvalidAccessKeyId

Expected behavior

If AWS_ENDPOINT is set in the environment, it should be used for litestream restore even without a config file.

Alternatively, litestream restore should allow specifying the endpoint explicitly via a CLI flag such as -endpoint, or infer from the hostname that the target is not AWS and avoid calling GetBucketLocation.

Why this matters

Backups are often needed under stress — for example, when the original system is lost. In that case, the configuration file might be gone too. Even if I do my best not to lose it, requiring a config file weakens defense in depth. It also adds friction when testing or inspecting backups on new machines.

Scope

This applies to any S3-compatible provider. I happen to be using Hetzner, but I imagine the same applies to all providers that don’t follow AWS region conventions.

Workaround

Define a minimal litestream.yml with the custom endpoint set on the replica:

dbs:
  - path: /dummy
    replicas:
      - name: hetzner
        url: s3://mybucket/mypath
        endpoint: https://fsn1.your-objectstorage.com

Then run:

litestream restore -config litestream.yml -replica hetzner -o /tmp/db.sqlite3 -if-db-not-exists /dummy

But this shouldn’t be necessary when all required details are already on the command line.

[github-actions[bot]]

This issue has been inactive for 90 days and will be automatically closed in 30 days if there is no further activity. If this issue is still relevant, please add a comment to keep it open. Thank you for your contribution!

[vanschelven]

stalebot go away

[cloud8421]

@vanschelven sorry for interjecting, I have been struggling to get replication to Hetzner S3 working, would you have a working example you can share? Thanks a lot.

[vanschelven]

It's been a while but my config is simply a version of this (with the above-mentioned notes about restoration)

access-key-id: ...
secret-access-key: ...

dbs:
 - path: /home/...
   replicas:
     - url: s3://.../...
       endpoint: https://fsn1.your-objectstorage.com
       retention: ...h