Lost window adjust packets (SSH_MSG_CHANNEL_WINDOW_ADJUST)

[evgeny-pasynkov]

Version

2.11

Bug description

Pretty often, when the client connects to Mina SSHD server via some ISPs, the connection got stuck. Mine investigation so far brought to lost window adjust packets. How the picture looks from client (with -vvv option):

debug2: channel 0: open confirm rwindow 2097152 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
debug2: channel 0: window 1995596 sent adjust 101556
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1933312 sent adjust 163840
debug2: channel 0: window 1933312 sent adjust 151217
debug2: channel 0: window 1986225 sent adjust 36814
debug2: channel 0: window 1924735 sent adjust 45813
debug2: channel 0: window 1937780 sent adjust 13441
debug2: channel 0: window 1951221 sent adjust 745
debug2: channel 0: window 1951966 sent adjust 790
debug2: channel 0: window 1952756 sent adjust 546
debug2: channel 0: window 1953302 sent adjust 1248
debug2: channel 0: window 1954550 sent adjust 732
......
debug2: channel 0: window 1986417 sent adjust 16758
debug3: send packet: type 1
client_loop: send disconnect: Broken pipe

How the logs look from server: point of view (filtered by SSH_MSG_CHANNEL_WINDOW_ADJUST word):

Dec 1, 2023 @ 14:58:21.200 | handleWindowAdjust([id=0, recipient=0]-SSHServerSessionEx[git@/95.90.194.56:32057]) SSH_MSG_CHANNEL_WINDOW_ADJUST window=131072
Dec 1, 2023 @ 14:58:21.200 | doHandleMessage(SSHServerSessionEx[git@/95.90.194.56:32057]) process #17 SSH_MSG_CHANNEL_WINDOW_ADJUST
Dec 1, 2023 @ 14:58:21.148 | doHandleMessage(SSHServerSessionEx[git@/95.90.194.56:32057]) process #16 SSH_MSG_CHANNEL_WINDOW_ADJUST
Dec 1, 2023 @ 14:58:21.148 | handleWindowAdjust([id=0, recipient=0]-SSHServerSessionEx[git@/95.90.194.56:32057]) SSH_MSG_CHANNEL_WINDOW_ADJUST window=101556

Note that only first two adjusts are processed, and no more these packets are handled! As a result, the server session exhausts the remaining remote window size and fails with timeout...

Any ideas how to debug this problem further?

Needless to say that the problem does not occur with native sshd, so I suppose there must be some packets which bring Mina Sshd session into inconsistent state

Actual behavior

The server waits for remote window indefinitely

Expected behavior

The server should work :)

Relevant log output

No response

Other information

No response

[tomaswolf]

Any ideas how to debug this problem further?

Not really. Never seen something like this. No other hints in the full log? I don't quite understand the filtered log excerpt you show above. doHandleMessage should always be logged before handleWindowAdjust.