Piotr Tabor

Please try to run `bboltctl check` on your DB file: https://github.com/etcd-io/bbolt/blob/f6be82302843a215152f5a1daf652c1ee5503f85/cmd/bbolt/main.go#L111

Bbolt does a lot of random reads, but it's fetching pages of size of 4KB -> 64KB (depending on os config/arch). As most cloud infrastructures and even local drives work...

> I think just a max page size should be sufficient to solve that. There is also no reason for it to grow exponentially. @lavalamp : max page size is...

The whole patch to apply in your code is here: https://github.com/Hargne/jest-html-reporter/compare/master...ptabor:20230901-checsum-instead-of-base64?expand=1#diff-9c4c6148b34d502eafc592468318c68e6b3383698e94c869700517af02b885a2 It's just using sha of the directory name instead of the full dictionary name. To reproduce the issue try...

Moby seems to use raw bbolt: https://github.com/moby/moby/blob/e068c38618ff94e3be9e572c568be3bed396c263/vendor.mod#L89 It's interesting how this happened. I have no other idea than 7 rolled back transactions and 8th successful in scope of the same...

Tx.Check is diagnostic method that checks consistency in given point of time using some set of rules. So far it focuses on relationship between pages: - Whether all pages references...

BTW: https://github.com/etcd-io/bbolt/pull/225 is expanding the Checks to cover also a logical errors (unexpected key order).

That's interesting. 1. Out of curiosity - you might try to log the size of the free-pages list that gets written with each transaction: https://github.com/etcd-io/bbolt/blob/505fc0f7af3c9bba93a80fc33918c90c1b0517ad/tx.go#L238 The list is not differential,...

60GB / 200MBit/s = 60GB / 25MB/s = 40min In etcd we are not going above 8GBs and either-way doing full scan on startup to compute hash. Please consider: ```...

In this spreadsheet I tried to model cost of a single key edit with NoFreelistSync=False: https://docs.google.com/spreadsheets/d/1O_wexgv1vM9GKZNCRjfQZ8U_bRmX_CLc-YSsBsVxDLg/edit#gid=0 With assumption of: dbSize=60GB avgKVsize=1000 bytes avgKeySize=16 bytes the optimal page size seems to...