dfuse-eosio icon indicating copy to clipboard operation
dfuse-eosio copied to clipboard

fluxdb/shardinject should print more frequent progress reports

Open matthewdarwin opened this issue 4 years ago • 1 comments

When loading statedb shards into the kv store, you only get a report every time a new file is loaded. If the files are pretty big, then you don't get an update very often (could be several hours between messages).

(statedb) processing shard file (fluxdb/shardinject.go:78){"filename": "0076000000-0077000000"}

On the other hand, trxdb loader is very verbose about what it is doing printing a message every 5 seconds.

(trxdb-loader) 5sec AVG INSERT RATE (trxdb-loader/loader.go:423){"block_num": 8831774, "last_tick_block": 8831448, "block_rate_last_5s": 65.2, "block_rate_last_500s": 71.09}

Thus this request is that statedb-enable-reproc-injector-mode = true should generate messages at least once per minute as to what block it is on. Additional stats like trxdb-loader are nice, but not necessary.

matthewdarwin avatar Sep 18 '20 19:09 matthewdarwin

can also use this to see progress:

dfuseeos tools check statedb-reproc-injector

matthewdarwin avatar Oct 22 '20 21:10 matthewdarwin