Qicheng (Jonathan) Hú

@rthallamko3 This issue was fixed by feec0ad4525b5cceb8f015d1911027903e398b32 Require backport to 2.14

Duplicate with #13389 , closing

``` W0719 05:17:02.595964 21016 consensus_peers.cc:543] T 79ca98e835854fb28277cc0ec104db0e P b48a054a12c74de28a75ea0e312c89b9 -> Peer 1a4db47f50b047b0bc490497a3a8d94e ([host: "10.9.214.176" port: 9100], []): Couldn't send request. Status: Network error (yb/util/net/socket.cc:540): recvmsg error: Connection refused (system error...

LB added load to a dead tserver because a tserver is set to dead only if not receiving its heartbeat for 60s. So there is a gap between resetting the...

We can wait for tablet servers gone before returning shrink is successfully done. As @rahuldesirazu suggested, we can let shrink do the following steps: Blacklist the nodes Destroy the instances...

``` yb-tserver.ip-172-151-18-254.us-west-2.compute.internal.yugabyte.log.INFO.20240227-133202.273248.gz:I0227 13:34:04.453986 273608 doc_read_context.cc:64] TBL 0000401b000030008000000000004226 T 00c52216ed3341aab818c29f44ec7f4c P d79ad410277a49fa85364466087c2ea2: DocReadContext, copy and filter: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,...

From the tablet meta, 16 non existing ``` yb-tserver.ip-172-151-18-254.us-west-2.compute.internal.yugabyte.log.INFO.20240227-133502.283758.gz:I0227 13:35:02.566885 283861 doc_read_context.cc:74] TBL 0000401b000030008000000000004226 T 00c52216ed3341aab818c29f44ec7f4c P d79ad410277a49fa85364466087c2ea2: LogAfterLoad: [21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,...

Reopen issue since it's happening on some overloaded clusters.