Jano Suchal

I've also tried to run the fetch command by hand, it downloaded only the one file, no prefetch files created.

@x4m actually there is not that folder, all i see is the `running` folder. Everything is empty. ``` root@restore-test:~# wal-g --version wal-g version v0.2.19 8dba4f4 2020.11.30_09:18:01 PostgreSQL ```

``` root@restore-test:~# WALG_LOG_LEVEL=DEVEL wal-g --config .walg-recovery.json wal-fetch 000000020002313A00000070 /tmp/RECOVERYXLOG DEBUG: 2021/03/01 07:59:15.989094 LC_MEASUREMENT=en_US.UTF-8 DEBUG: 2021/03/01 07:59:15.989097 SSH_CONNECTION=188.121.182.171 51192 204.48.29.49 22 DEBUG: 2021/03/01 07:59:15.989100 LESSCLOSE=/usr/bin/lesspipe %s %s DEBUG: 2021/03/01 07:59:15.989102 LC_PAPER=en_US.UTF-8...

What is weird is this, i've tried running by hand the prefetch and it works totaly as expected. ``` INFO: 2021/03/01 08:16:05.171820 WAL-prefetch file: 000000020002313A000000AB INFO: 2021/03/01 08:16:05.181920 WAL-prefetch file:...

Ok, I have a new hypothesis. I think that the WAL apply is just too fast. I am running a recovery now and changing the env var WALG_DOWNLOAD_CONCURRENCY. When I...

This is definitely it. I've tried tu run an insane prefetch (with 10k concurrency) to a dir that was used by restore by hand and suddenly i see a huge...

I don't know. What completely baffles me is that the wal-prefetch here is prefetching old segments concurrently, see the segment for fetch and prefetch. ``` 31789 postgres 20 0 33.032g...

I've finally came to test this on real data. Here is how this looks. The first peak is getting base backup, after that you see wal-g 0.20.x version, then i...

yes you are right, i am not sure if its waiting for wal fetch is there an easy way to find out?

If anyone wants a drop-in fix. This worked for us. Drop this monkey patch somewhere (e.g. `initializers` in rails) It basically works by scoping the inner query by the scope...