Test failure: integration test membership/transferPendingTransaction

[MathieuBordere]

see https://github.com/canonical/dqlite/actions/runs/3640210436/jobs/6144579153

membership/transferPendingTransaction                       
  disk_mode=0                                               [ ERROR ]
LIBDQLITE 1670426213715849606 VfsInit:2024 vfs init
LIBDQLITE 1670426213715977709 raftProxyInit:242 raft proxy init
LIBDQLITE 1670426213716077211 fsm__init:711 fsm init
LIBDQLITE 1670426213748460188 impl_init:45 impl init
LIBDQLITE 1670426213748602191 dqlite_node_start:716 dqlite node start
LIBDQLITE 1670426213751115451 impl_listen:55 impl listen
LIBDQLITE 1670426213751208754 clientInit:17 init client fd 15
LIBDQLITE 1670426213751281755 VfsInit:2024 vfs init
LIBDQLITE 1670426213751341857 conn__start:290 conn start
LIBDQLITE 1670426213751377558 gateway__init:18 gateway init
LIBDQLITE 1670426213751425459 raftProxyInit:242 raft proxy init
LIBDQLITE 1670426213751445959 fsm__init:711 fsm init
LIBDQLITE 1670426213788379145 impl_init:45 impl init
LIBDQLITE 1670426213788441846 dqlite_node_start:716 dqlite node start
LIBDQLITE 1670426213788733553 impl_listen:55 impl listen
LIBDQLITE 1670426213788789055 clientInit:17 init client fd 23
LIBDQLITE 1670426213788818155 VfsInit:2024 vfs init
LIBDQLITE 1670426213788847156 raftProxyInit:242 raft proxy init
LIBDQLITE 1670426213788870656 conn__start:290 conn start
LIBDQLITE 1670426213788883957 gateway__init:18 gateway init
LIBDQLITE 1670426213788902757 fsm__init:711 fsm init
LIBDQLITE 1670426213820434913 impl_init:45 impl init
LIBDQLITE 1670426213820500515 dqlite_node_start:716 dqlite node start
LIBDQLITE 1670426213820850423 impl_listen:55 impl listen
LIBDQLITE 1670426213820908925 clientInit:17 init client fd 31
LIBDQLITE 1670426213820958026 clientSendHandshake:52 client send handshake fd 15
LIBDQLITE 1670426213820988226 conn__start:290 conn start
LIBDQLITE 1670426213821002327 gateway__init:18 gateway init
LIBDQLITE 1670426213821037828 clientSendAdd:340 client send add fd 15 id 2 address @2
LIBDQLITE 1670426213821089729 gateway__handle:1280 gateway handle
LIBDQLITE 1670426213821108529 handle_add:883 handle add
LIBDQLITE 1670426213821672243 clientRecvEmpty:379 client recv empty fd 15
LIBDQLITE 1670426213823241380 raftChangeCb:866 raft change cb status 0
LIBDQLITE 1670426213823324282 clientSendAssign:350 client send assign fd 15 id 2 role 1
LIBDQLITE 1670426213823504987 clientRecvEmpty:379 client recv empty fd 15
LIBDQLITE 1670426213823355283 gateway__resume:1327 gateway resume - finished
LIBDQLITE 1670426213823533787 gateway__handle:1280 gateway handle
LIBDQLITE 1670426213823544288 handle_assign:916 handle assign
LIBDQLITE 1670426213823623790 impl_connect:169 impl connect id:2 address:@2
LIBDQLITE 1670426213823754693 connect_work_cb:63 connect work cb
LIBDQLITE 1670426213823810694 conn__start:290 conn start
LIBDQLITE 1670426213823817794 gateway__init:18 gateway init
LIBDQLITE 1670426213823881696 raft_connect:111 raft_connect
LIBDQLITE 1670426213823889396 raftProxyAccept:271 raft proxy accept
LIBDQLITE 1670426213823955798 connect_after_work_cb:139 connect after work cb status 0
LIBDQLITE 1670426213824569712 impl_connect:169 impl connect id:1 address:@1
LIBDQLITE 1670426213824606513 connect_work_cb:63 connect work cb
LIBDQLITE 1670426213824656114 conn__start:290 conn start
LIBDQLITE 1670426213849924420 leader__barrier:446 leader barrier
LIBDQLITE 1670426213849927320 leader__barrier:449 not needed
LIBDQLITE 1670426213849930120 prepareBarrierCb:279 prepare barrier cb status:0
LIBDQLITE 1670426213849963521 gateway__resume:1327 gateway resume - finished
LIBDQLITE 1670426213849970621 clientRecvStmt:187 client recv stmt fd 15 stmt_id 3
LIBDQLITE 1670426213849976421 clientSendQuery:225 client send query fd 15 stmt_id 3
LIBDQLITE 1670426213849991922 clientRecvRows:245 client recv rows fd 15
LIBDQLITE 1670426213850001022 gateway__handle:1280 gateway handle
LIBDQLITE 1670426213850008122 handle_query:516 handle query
LIBDQLITE 1670426213850014622 leader__barrier:446 leader barrier
LIBDQLITE 1670426213850017522 leader__barrier:449 not needed
LIBDQLITE 1670426213850020822 query_barrier_cb:491 query barrier cb status:0
LIBDQLITE 1670426213850050623 gateway__resume:1327 gateway resume - finished
LIBDQLITE 1670426213850086124 clientInit:17 init client fd 45
LIBDQLITE 1670426213850109525 conn__start:290 conn start
LIBDQLITE 1670426213850117625 gateway__init:18 gateway init
LIBDQLITE 1670426213850119325 clientSendHandshake:52 client send handshake fd 45
LIBDQLITE 1670426213850132925 clientSendTransfer:370 client send transfer fd 45 id 2
LIBDQLITE 1670426213850144225 clientRecvEmpty:379 client recv empty fd 45
LIBDQLITE 1670426213850162126 gateway__handle:1280 gateway handle
LIBDQLITE 1670426213850170226 handle_transfer:1216 handle transfer
LIBDQLITE 1670426214007846406 raftTransferCb:1207 transfer failed
LIBDQLITE 1670426214007908608 gateway__resume:1327 gateway resume - finished
LIBDQLITE 1670426214007982709 clientRecvEmpty:381 read decode failed rv 0)
Error: test/integration/test_membership.c:210: assertion failed: rv_ == 0 (1 == 0)

Looks like the leadership transfer fails sometimes.

[ganto]

I noticed the same when trying to build dqlite-1.12.0 on the Koji Fedora build system:

membership/transferPendingTransaction                       [ ERROR ]
Error: test/integration/test_membership.c:201: assertion failed: rv_ == 0 (1 == 0)
Error: child killed by signal 6 (Aborted)
membership/transfer                                         [ ERROR ]
Error: test/integration/test_membership.c:150: assertion failed: rv_ == 0 (1 == 0)
Error: child killed by signal 6 (Aborted)
membership/join                                             [ ERROR ]
Error: test/integration/test_membership.c:102: assertion failed: rv_ == 0 (1 == 0)
Error: child killed by signal 6 (Aborted)

One build on ppc64 failed but all the others where successful.

The same ppc64 build however built successfully on the Fedora COPR build system.