Digital Ocean Spaces api error SignatureDoesNotMatch

[jiqiren]

access-key-id: redacted
secret-access-key: redacted

dbs:
  - path: /gotosocial/sqlite.db
    replica:
      url: s3://redacted.sfo3.digitaloceanspaces.com/db

Bug Description

Connects fine to local sqlite db but write to Digital Ocean Spaces fails with Signature issues.

time=2025-12-13T01:38:00.891Z level=INFO msg=litestream version=v0.5.3 level=""
time=2025-12-13T01:38:00.892Z level=INFO msg="initialized db" path=/gotosocial/sqlite.db
time=2025-12-13T01:38:00.893Z level=INFO msg="replicating to" type=s3 sync-interval=1s bucket=dustyninja-backup path=db region=sfo3 endpoint=https://sfo3.digitaloceanspaces.com
time=2025-12-13T01:38:00.893Z level=INFO msg="starting L0 retention monitor" interval=15s retention=5m0s
time=2025-12-13T01:38:00.897Z level=INFO msg="starting compaction monitor" level=1 interval=30s
time=2025-12-13T01:38:00.897Z level=INFO msg="starting compaction monitor" level=2 interval=5m0s
time=2025-12-13T01:38:00.897Z level=INFO msg="starting compaction monitor" level=3 interval=1h0m0s
time=2025-12-13T01:38:00.897Z level=INFO msg="starting compaction monitor" level=9 interval=24h0m0s
time=2025-12-13T01:38:30.887Z level=INFO msg="signal received, litestream shutting down"
time=2025-12-13T01:38:30.967Z level=ERROR msg="monitor error" db=sqlite.db replica=s3 error="no position, waiting for data"
time=2025-12-13T01:41:56.191Z level=INFO msg=litestream version=v0.5.3 level=debug
time=2025-12-13T01:41:56.195Z level=INFO msg="initialized db" path=/gotosocial/sqlite.db
time=2025-12-13T01:41:56.195Z level=INFO msg="replicating to" type=s3 sync-interval=1s bucket=dustyninja-backup path=db region=sfo3 endpoint=https://sfo3.digitaloceanspaces.com
time=2025-12-13T01:41:56.196Z level=INFO msg="starting L0 retention monitor" interval=15s retention=5m0s
time=2025-12-13T01:41:56.196Z level=INFO msg="starting compaction monitor" level=1 interval=30s
time=2025-12-13T01:41:56.196Z level=DEBUG msg="db not ready, skipping" level=1 path=/gotosocial/sqlite.db
time=2025-12-13T01:41:56.196Z level=INFO msg="starting compaction monitor" level=2 interval=5m0s
time=2025-12-13T01:41:56.196Z level=DEBUG msg="db not ready, skipping" level=2 path=/gotosocial/sqlite.db
time=2025-12-13T01:41:56.196Z level=INFO msg="starting compaction monitor" level=3 interval=1h0m0s
time=2025-12-13T01:41:56.197Z level=DEBUG msg="db not ready, skipping" level=3 path=/gotosocial/sqlite.db
time=2025-12-13T01:41:56.197Z level=INFO msg="starting compaction monitor" level=9 interval=24h0m0s
time=2025-12-13T01:41:56.197Z level=DEBUG msg="db not ready, skipping" level=9 path=/gotosocial/sqlite.db
time=2025-12-13T01:41:57.334Z level=DEBUG msg=sync db=sqlite.db txid=0000000000000001 offset=32 snap=true
time=2025-12-13T01:42:11.207Z level=DEBUG msg="enforcing l0 retention" db=sqlite.db retention=5m0s
time=2025-12-13T01:42:24.961Z level=DEBUG msg="db sync" db=sqlite.db status=ok
time=2025-12-13T01:42:26.199Z level=DEBUG msg="enforcing l0 retention" db=sqlite.db retention=5m0s
time=2025-12-13T01:42:39.233Z level=DEBUG msg=verify saltMatch=true prevWALOffset=1277232
time=2025-12-13T01:42:41.201Z level=DEBUG msg="enforcing l0 retention" db=sqlite.db retention=5m0s
time=2025-12-13T01:42:52.545Z level=DEBUG msg=verify.2 lastPageMatch=true
time=2025-12-13T01:42:56.198Z level=DEBUG msg="enforcing l0 retention" db=sqlite.db retention=5m0s
time=2025-12-13T01:43:06.235Z level=DEBUG msg=sync db=sqlite.db txid=0000000000000002 offset=1281352 chkpt=true
time=2025-12-13T01:43:06.275Z level=DEBUG msg="db sync" db=sqlite.db status=ok
time=2025-12-13T01:43:06.314Z level=DEBUG msg=checkpoint db=sqlite.db mode=PASSIVE result=0,1695,1695
time=2025-12-13T01:43:06.352Z level=DEBUG msg=verify saltMatch=false prevWALOffset=6979312
time=2025-12-13T01:43:06.364Z level=DEBUG msg=verify.2 lastPageMatch=true
time=2025-12-13T01:43:06.375Z level=DEBUG msg=sync db=sqlite.db txid=0000000000000003 offset=32 chkpt=true
time=2025-12-13T01:43:06.378Z level=DEBUG msg="db sync" db=sqlite.db status=ok
time=2025-12-13T01:43:06.380Z level=DEBUG msg=verify saltMatch=true prevWALOffset=32
time=2025-12-13T01:43:06.381Z level=DEBUG msg=sync db=sqlite.db txid=0000000000000004 offset=4152
time=2025-12-13T01:43:06.381Z level=DEBUG msg=verify saltMatch=true prevWALOffset=32
time=2025-12-13T01:43:06.382Z level=DEBUG msg=sync db=sqlite.db txid=0000000000000004 offset=4152 chkpt=true
time=2025-12-13T01:43:06.384Z level=DEBUG msg=checkpoint db=sqlite.db mode=PASSIVE result=0,1,1
time=2025-12-13T01:43:06.386Z level=DEBUG msg=verify saltMatch=false prevWALOffset=32
time=2025-12-13T01:43:06.387Z level=DEBUG msg=sync db=sqlite.db txid=0000000000000004 offset=4152 chkpt=true snap=true reason="wal header salt reset, snapshotting"
time=2025-12-13T01:43:06.422Z level=DEBUG msg="replica sync" db=sqlite.db replica=s3 txid=0000000000000003
time=2025-12-13T01:43:06.519Z level=DEBUG msg="no compaction" level=1 path=/gotosocial/sqlite.db
time=2025-12-13T01:43:06.613Z level=ERROR msg="monitor error" db=sqlite.db replica=s3 error="write ltx file: s3: upload to db/0000/0000000000000001-0000000000000001.ltx: upload multipart failed, upload id: 2~KQSv4kC57uibbEPFLoycIYFyGSmDSky, cause: operation error S3: UploadPart, https response error StatusCode: 403, RequestID: 54bb5be5-d558-291953972, HostID: , api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method."
time=2025-12-13T01:43:11.214Z level=DEBUG msg="enforcing l0 retention" db=sqlite.db retention=5m0s
time=2025-12-13T01:43:26.210Z level=DEBUG msg="enforcing l0 retention" db=sqlite.db retention=5m0s

Environment

docker: root@sfo3-01:~# docker image ls | grep lite litestream/litestream 0.5 f8c8fd6a60b3 32 hours ago 44.5MB

litestream.yml is at beginning of this post.

Litestream version:

v0.5.3

Operating system & version: Debian 13 trixie

Installation method: Docker

Storage backend: S3

Steps to Reproduce

Follow guide to use S3 on Digital Ocean Spaces: https://litestream.io/guides/digitalocean/

Expected behavior: replicate to Digital Ocean Spaces

Actual behavior: Some kind of signature error when trying to upload files from litestream.

Configuration

access-key-id: redacted
secret-access-key: redacted

dbs:
  - path: /gotosocial/sqlite.db
    replica:
      url: s3://redacted.sfo3.digitaloceanspaces.com/db

Logs

Log output

time=2025-12-13T01:43:06.613Z level=ERROR msg="monitor error" db=sqlite.db replica=s3 error="write ltx file: s3: upload to db/0000/0000000000000001-0000000000000001.ltx: upload multipart failed, upload id: 2~KQSv4kC57uibbEPFLoycIYFyGSmDSky, cause: operation error S3: UploadPart, https response error StatusCode: 403, RequestID: 54bb5be5-d558-291953972, HostID: , api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method."

I use s3cmd with exactly the same key/secret from same host to upload copies of sqlite file to S3. I do this once per day via cron.

[jiqiren]

forgot to add for the purpose of this report I changed my config to turn on debug logging:

access-key-id: redacted
secret-access-key: redacted

logging:
  level: debug # options: debug | info | warn | error

dbs:
  - path: /gotosocial/sqlite.db
    replica:
      url: s3://dustyninja-backup.sfo3.digitaloceanspaces.com/db

Already included my bucket name above so leaving it here now (it's fine - bucket is empty).

[jiqiren]

I was able to get working using v0.3.13. I tried v0.5.3 with exactly same config and still get error above.

logging:
  level: info # options: debug | info | warn | error

dbs:
  - path: /gotosocial/sqlite.db
    replicas:
      - type: s3
        bucket: dustyninja-backup
        path: sqlite-litestream
        endpoint: sfo3.digitaloceanspaces.com
        region: sfo3
        access-key-id: KEYID
        secret-access-key: SECRETKEY

[mzmcgrat]

I encountered the same issue when I upgraded to the latest version of Litestream. I am using a self-hosted instance of Minio as my S3 endpoint, and adding the new config option sign-payload: true seems to have solved the issue for me.

[plttn]

Thanks for listing the config option! Backblaze B2 now also requires sign-payload: true.

[mzmcgrat]

Maybe the documentation/examples for the various s3-compatible endpoints, and especially this table: https://litestream.io/reference/config/#s3-compatible-provider-requirements should be updated, since apparently both B2 and Minio require sign-payload: true now.

[MohamedBassem]

I wonder if this should get called out in the release notes. Because litestream was working fine without sign-payload: true on 0.5.2 and only after upgrading to 0.5.3 did it start failing (and it seems like a new config option anyway).

[jiqiren]

Thanks for all the comments everyone! I never saw the option sign-payload: true at all to know about it. This was my first experience / attempt to use litestream and it was pretty frustrating. Will be switching over to v0.5.3 and add in sign-payload: true.

[kalbasit]

I got hit with this as well using my MinIO instance; Adding sign-payload: true fixed my issue.

[mprokopiev]

Following the #895, setting sign-payload to true doesn't with Filebase:

# cat /etc/litestream.yml
logging:
  level: TRACE
dbs:
  - path: ${NTFY_AUTH_FILE}
    replica:
      url: s3://litestream/ntfyv5
      endpoint: https://s3.filebase.com
      sign-payload: true
# litestream replicate -exec "ntfy serve"
....
time=2025-12-14T20:03:08.646+02:00 level=ERROR msg="monitor error" db=user.db replica=s3 error="write ltx file: s3: upload to ntfyv5/0000/0000000000000001-0000000000000001.ltx: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 0a3cb7a127c2944767cbf0153fdb3c9f, HostID: ZmlsZWJhc2UtNmI5Y2M2OTY5Yy03Nnc4cQ==, api error AccessDenied: Access Denied"

It isn't creds issue as 0.3.13 binary works with the same creds.

[corylanou]

PR #899 addresses this issue with a comprehensive fix for S3-compatible providers:

1. Auto-detection: Automatically detects DigitalOcean Spaces and applies sign-payload: true by default
2. aws-chunked fix: Disables aws-chunked encoding for all custom endpoints (not just unsigned payloads)
3. Provider defaults: Auto-applies appropriate settings for DigitalOcean, Backblaze B2, Filebase, Scaleway, and MinIO

Once merged, DigitalOcean Spaces should work without any manual configuration. The sign-payload: true workaround will no longer be needed.

[MohamedBassem]

Thanks for the fix @corylanou, just want to call out that cloudflare's R2 also required the flag. Might be useful to auto detect it as well.

[corylanou]

@MohamedBassem Cloudflare R2 auto-detection has been added in PR #899. R2 endpoints (*.r2.cloudflarestorage.com) will now automatically have SignPayload=true applied.

Thanks for the suggestion!