neon
neon copied to clipboard
failed to run pgbench in docker setup
I'm running pgbench -s5 -i in docker setup as it is in our console repo.
pgbench output:
NOTICE: table "pgbench_accounts" does not exist, skipping
NOTICE: table "pgbench_branches" does not exist, skipping
NOTICE: table "pgbench_history" does not exist, skipping
NOTICE: table "pgbench_tellers" does not exist, skipping
creating tables...
generating data (client-side)...
500000 of 500000 tuples (100%) done (elapsed 0.28 s, remaining 0.00 s)
vacuuming...
pgbench: fatal: query failed: ERROR: could not read block 0 in rel 1663/16385/16399.2 from page server at lsn 0/08D8C7A8
CONTEXT: while scanning block 0 of relation "public.pgbench_branches"
pgbench: query was: vacuum analyze pgbench_branches
In pageserver logs:
INFO Created database 1663/16385, copied 1014 blocks in 243 rels at 0/169F110, location: pageserver::restore_local_repo, pageserver/src/restore_local_repo.rs:646
INFO switched segment 0/1FFEB00 to 0/2002660, pageserver/src/walreceiver.rs:257
INFO switched segment 0/2FFFDF8 to 0/301D870, pageserver/src/walreceiver.rs:257
ERRO get_page_at_lsn: Timed out while waiting for WAL record at LSN 0/8D89210 to arrive, pageserver/src/page_service.rs:377
INFO switched segment 0/3FFF090 to 0/4004C40, pageserver/src/walreceiver.rs:257
INFO switched segment 0/4FFE860 to 0/5005EF8, pageserver/src/walreceiver.rs:257
ERRO get_page_at_lsn: Timed out while waiting for WAL record at LSN 0/8D89210 to arrive, pageserver/src/page_service.rs:377
INFO switched segment 0/5FFFAF8 to 0/6003658, pageserver/src/walreceiver.rs:257
INFO switched segment 0/6FFED28 to 0/7000D88, pageserver/src/walreceiver.rs:257
ERRO get_page_at_lsn: Timed out while waiting for WAL record at LSN 0/8D89210 to arrive, pageserver/src/page_service.rs:377
INFO switched segment 0/7FFFF80 to 0/8001FE0, pageserver/src/walreceiver.rs:257
WARN drop_segment called on non-existent relish 1663/16385/16389 at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16389_fsm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16389_vm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16386 at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16386_fsm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16386_vm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16395 at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16395_fsm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16395_vm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16392 at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16392_fsm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
WARN drop_segment called on non-existent relish 1663/16385/16392_vm at 0/8D8A6D0, pageserver/src/layered_repository.rs:820
ERRO get_page_at_lsn: Timed out while waiting for WAL record at LSN 0/8D8C7A8 to arrive, pageserver/src/page_service.rs:377
Timeouts are probably fine, because I had runs with timeouts but without this warnings
@lubennikovaav is it #502 or something new?
Warning was fixed some time ago, wait lsn timeout error should be fixed in #630
@LizardWizzard let's check that again since backpressure was merged/fixed? #803
@LizardWizzard could this be closed ? or do we need it for anything else ?
Lets close this in favor of https://github.com/neondatabase/neon/issues/1587.
Current issue was discovered when I was running whole setup in docker on mac os, so it was quite slow. I no longer have this setup at hand, so I think it makes sense to add additional test for backpressure with CPU limits applied for pageserver process