client write: stat backup/generations/xxx.wal.lz4.tmp: no such file or directory

[dwebfan]

Thanks for the great useful library. While I'm trying the local copy, I found one strange issue:

1. modify https://github.com/benbjohnson/litestream-library-example and use local FS instead of S3.
2. start the example, and visit the webpage localhost:8080. I can see the nice web page.
3. delete all generated files including orig.db, orig.db-wal, orig.db-shm, backup directory, and hidden directory .test.db-litestream
4. restart the example, and visit the web page again. Sometime webpage will has failure such as

client write: stat backup/generations/ead3b03b39a914b3/wal/00000000_00004080.wal.lz4.tmp: no such file or directory

Also it happens if I did these steps:

1. delete orig.db
2. restart the example, it can restore successful and web page will continue counting
3. stop the example, and delete backup directory which has the snapshots and wals
4. restart the example, and visit web page again. Above error also shows up sometimes.

Let me know if you need more information. Thanks again for making the library

[benbjohnson]

hey @dwebfan, I'm sorry for the late reply. I thought I had replied much earlier but it doesn't look like it posted. Do you have the code that you're using that switches to use the local file system instead of S3?

[dwebfan]

@benbjohnson here is the diff https://github.com/benbjohnson/litestream-library-example/compare/main...dwebfan:main.

This time, even if I start the example at the first time, it has the failure.

After the failure, I refresh web page, it has the right result, but visit count seems already bump up to 5 even if I only click 2 times.

Below is console output. OS is MAC. Golang version 1.16.

$ ./litestream-library-example -type file -dsn orig.db -bucket test
no generation found, creating new database
listening on :8080
2021/08/23 23:34:05 orig.db: sync: new generation "f249fbbdece8354d", no generation exists
2021/08/23 23:34:05 orig.db(file): snapshot written f249fbbdece8354d/00000000
2021/08/23 23:34:30 new transaction: pre=f249fbbdece8354d/00000000:24752 post=f249fbbdece8354d/00000000:28872 elapsed=12.99109ms
2021/08/23 23:34:30 new transaction: pre=f249fbbdece8354d/00000000:28872 post=f249fbbdece8354d/00000000:32992 elapsed=18.008301ms
2021/08/23 23:35:26 new transaction: pre=f249fbbdece8354d/00000001:4152 post=f249fbbdece8354d/00000001:8272 elapsed=23.758204ms
2021/08/23 23:35:53 new transaction: pre=f249fbbdece8354d/00000001:8272 post=f249fbbdece8354d/00000001:12392 elapsed=10.508717ms
2021/08/23 23:35:54 new transaction: pre=f249fbbdece8354d/00000001:12392 post=f249fbbdece8354d/00000001:16512 elapsed=18.32489ms
2021/08/23 23:35:54 new transaction: pre=f249fbbdece8354d/00000001:16512 post=f249fbbdece8354d/00000001:20632 elapsed=16.246002ms
2021/08/23 23:35:54 new transaction: pre=f249fbbdece8354d/00000001:20632 post=f249fbbdece8354d/00000001:24752 elapsed=17.745326ms
2021/08/23 23:35:54 new transaction: pre=f249fbbdece8354d/00000001:24752 post=f249fbbdece8354d/00000001:28872 elapsed=13.689926ms
^C
$ ls
LICENSE                    go.mod                     litestream-library-example orig.db                    orig.db-wal                vendor
README.md                  go.sum                     main.go                    orig.db-shm                test
$ tree test/
test/
└── generations
    └── f249fbbdece8354d
        ├── snapshots
        │   └── 00000000.snapshot.lz4
        └── wal
            ├── 00000000_00000000.wal.lz4
            ├── 00000000_00004080.wal.lz4
            ├── 00000000_00005098.wal.lz4
            ├── 00000000_000060b0.wal.lz4
            ├── 00000000_000070c8.wal.lz4
            ├── 00000001_00000000.wal.lz4
            ├── 00000001_00001038.wal.lz4
            ├── 00000001_00002050.wal.lz4
            ├── 00000001_00003068.wal.lz4
            ├── 00000001_00004080.wal.lz4
            ├── 00000001_00005098.wal.lz4
            └── 00000001_000060b0.wal.lz4

4 directories, 13 files

[hifi]

Hi, is this still reproducible with the latest git head?