'no snapshot available'

[yapishu]

Howdy! I've used Litestream with great success for a personal project and was trying to work it into a new project. I have Litestream running in Docker, reading a WAL-mode DB in a volume shared with another container. However, the first time I tried turning it on, it began spamming the following error:

/db/db.db(s3): monitor error: cannot determine replica position: no snapshot available: generation=60bedb457196f69d
/db/db.db(s3): snapshot written 60bedb457196f69d/00000003
/db/db.db(s3): monitor error: cannot determine replica position: no snapshot available: generation=60bedb457196f69d
/db/db.db(s3): snapshot written 60bedb457196f69d/00000003
/db/db.db(s3): monitor error: cannot determine replica position: no snapshot available: generation=60bedb457196f69d
/db/db.db(s3): snapshot written 60bedb457196f69d/00000003

The DB file it's replicating had already existed for some time before I tried adding Litestream to the mix. Restarting Litestream did not fix it, though it did create a new snapshot dir on S3.

The weird thing is that it seemed to be writing successfully to the bucket, though it pretty quickly dumped ~9mb of data despite the DB being ~250kb, and only two or three small transactions taking place while I had it running. I had it hooked up to a free-tier Backblaze bucket and it hit the daily transaction limit within ten minutes as well (2500 S3 API calls).

I'm not really sure what's going on here -- it can definitely read the DB file, it can authenticate with Backblaze. Googling the error message didn't give me any leads either. Any chance someone can parse this issue better than I?

[benbjohnson]

@yapishu Do you have more than one server running Litestream and pointing to the same S3 bucket/path? Multiple instances can conflict when they go to clean up the old WAL files and they can delete each other's snapshots.

Also, what version are you running? I'm not sure why it would cause 9MB of data being uploaded for only 2-3 transactions on a small database. Can you provide your litestream.yml file as well?

[yapishu]

@benbjohnson No, just the one copy, running in a container via compose. I created this bucket solely for this purpose, nothing else knows about it. I tried setting up a new Vultr S3 bucket and got the same error (had not previously used Vultr S3 at all).

I am running the latest version of the Docker image. Here's my config:

access-key-id: ${LITESTREAM_ACCESS_KEY_ID}
secret-access-key: ${LITESTREAM_SECRET_ACCESS_KEY}

dbs:
  - path: /db/db.db
    replicas:
      - type: s3
        bucket: <redacted>
        path: /
        endpoint: ewr1.vultrobjects.com
        force-path-style: true

[yapishu]

Possibly there's something weird about my DB? FWIW, it's managed by some python r/w functions, and has WAL-mode enabled explicitly, but nothing crazy going on -- it's just a normal collection of tables. Since the error persists across S3 hosts I reckon it has to be something client-side, but my config is pretty bog-standard.

[benbjohnson]

Possibly there's something weird about my DB?

I haven't seen what you're experiencing before. If you have a small script to reproduce the issue then I can try to take a look.

[yapishu]

i changed the s3 provider to DO and it stopped spamming errors

[defp]

i changed the s3 provider to DO and it stopped spamming errors

How to changed? DO is DigitalOcean?

[yapishu]

@defp I used digitalocean.com Spaces as my S3 provider, that's the only thing I changed

[defp]

I have the same problem, also use s3 provider and aliyun oss config

dbs:
  - path: ./fruits.db
    replicas:
      - type: s3
        bucket: nocode-gitlab-ci
        path: /litestream/2/fruits.db  
        region: oss-cn-hangzhou
        endpoint: https://oss-cn-hangzhou.aliyuncs.com
        force-path-style: false

[defp]

Maybe the provider s3 api is not compatible

[cycade]

@defp Try removing leading slash from replicas.path?