viosock: calling select() in Windows guest results in BSOD

[tuxxi]

Describe the bug Running a server in Windows guest, connecting from a Linux host. Calling select() on an accepted socket handle causes the VM to crash.

To Reproduce Steps to reproduce the behaviour:

• Bind() address and Listen() for connections in VM guest
• Accept connection
• Call select() on socket
• Observe VM crash

It doesn't matter what parameters we pass in to select(), or whether the socket is in blocking or non-blocking mode.

Here is a minimal reproducible example. Note that error handling has been omitted for brevity

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>

#include <ws2spi.h>
#include <winsock2.h>

#include "vio_sockets.h"

#pragma comment(lib, "ws2_32.lib")

#define PORT 591

int main() {
    WSADATA wsaData = { 0 };
    WSAStartup(MAKEWORD(2, 2), &wsaData);
    ADDRESS_FAMILY af = ViosockGetAF();
    SOCKET sock = socket(af, SOCK_STREAM, 0);

    struct sockaddr_vm addr;
    struct sockaddr_vm peer_addr;

    memset(&addr, 0, sizeof(struct sockaddr_vm));
    addr.svm_family = af;
    addr.svm_port = PORT;
    addr.svm_cid = VMADDR_CID_ANY;

    bind(sock, (const struct sockaddr *)&addr, sizeof(struct sockaddr_vm));
    listen(sock, /*max connections=*/1);

    int peer_addr_size = sizeof(struct sockaddr_vm);
    int peer_fd = accept(sock, (struct sockaddr *)&peer_addr, &peer_addr_size);

    printf("Got client, CID: %d, port:%d\n", peer_addr.svm_cid, peer_addr.svm_port);
    
    FD_SET readSet;
    TIMEVAL timeout = {1, 0};

    size_t msg_len = 0;
    char buf[64];

    while(msg_len == 0) {
        FD_ZERO(&readSet);
        FD_SET(peer_fd, &readSet);

        select(0, &readSet, NULL, NULL, &timeout); // <----- fail here
        printf("select OK\n");
        if (FD_ISSET(peer_fd, &readSet)) {
            size_t recvd = recv(peer_fd, &buf[0], sizeof(buf), 0);
            if (recvd == WSAEWOULDBLOCK) {
                continue;
            }
            printf("Received %zu bytes from client: %.*s\n", recvd, (int)recvd, buf);
        }
    }
}

Expected behavior Expected that we can call select() on a socket, block and then get notified by kernel when ready to read or write.

Host: RHEL 8 Kernel 4.18.0-348.20.1.el8 QEMU emulator version 6.0.0 (qemu-kvm-6.0.0-33.el8)

VM: Windows Server 21H2 (Build 20348.1249) Latest from fedora virtio-win upstream-virtio - build 100.10.100.10000 (8/3/2022)

Additional context Stack trace from MEMORY.DMP (can upload if requested)

BUGCHECK_CODE:  3b

BUGCHECK_P1: c0000005

BUGCHECK_P2: fffff80059397395

BUGCHECK_P3: ffff900718b329c0

BUGCHECK_P4: 0

CONTEXT:  ffff900718b329c0 -- (.cxr 0xffff900718b329c0)
rax=fffff800593941d8 rbx=0000000000000000 rcx=fffff800593945a0
rdx=0000000000000001 rsi=0000000000000000 rdi=ffffba8fe21725c0
rip=fffff80059397395 rsp=ffff900718b333e0 rbp=0000000000000000
 r8=0000000000000000  r9=fffff780000003b0 r10=0000000000000001
r11=0000000000000000 r12=000045701d266f18 r13=0000000000000620
r14=fffff80059392801 r15=ffff900718b33500
iopl=0         nv up ei pl zr na po nc
cs=0010  ss=0018  ds=002b  es=002b  fs=0053  gs=002b             efl=00050246
viosock!VIOSockSelectCheckPkt+0x61:
fffff800`59397395 891e            mov     dword ptr [rsi],ebx ds:002b:00000000`00000000=????????
Resetting default scope

BLACKBOXBSD: 1 (!blackboxbsd)


BLACKBOXNTFS: 1 (!blackboxntfs)


BLACKBOXPNP: 1 (!blackboxpnp)


BLACKBOXWINLOGON: 1

PROCESS_NAME:  select_fail.exe

STACK_TEXT:  
ffff9007`18b333e0 fffff800`59397d8c     : 00000000`00000000 ffff9007`18b33490 00004570`1d266f18 00000000`00000000 : viosock!VIOSockSelectCheckPkt+0x61 [C:\cygwin64\tmp\build\source\internal-kvm-guest-drivers-windows\viosock\sys\Device.c @ 834] 
ffff9007`18b33420 fffff800`59396fa4     : 00000000`00000000 ffffba8f`e21725c0 ffffba8f`e3160280 00000000`00000620 : viosock!VIOSockSelect+0x34c [C:\cygwin64\tmp\build\source\internal-kvm-guest-drivers-windows\viosock\sys\Device.c @ 1155] 
ffff9007`18b334d0 fffff800`2fbfc3a5     : ffffba8f`e2d990e0 00000000`08013008 ffff9007`18b336e0 00000000`00000000 : viosock!VIOSockEvtIoDeviceControl+0x124 [C:\cygwin64\tmp\build\source\internal-kvm-guest-drivers-windows\viosock\sys\Device.c @ 682] 
ffff9007`18b33520 fffff800`2fbfb8bf     : ffffba8f`e1578200 00000000`00000001 00000000`00000001 ffffba8f`e81e57f0 : Wdf01000!FxIoQueue::DispatchRequestToDriver+0x2c5 [minkernel\wdf\framework\shared\irphandlers\io\fxioqueue.cpp @ 3325] 
ffff9007`18b33590 fffff800`2fbfa088     : ffffba8f`e81e57f0 ffffba8f`dff05a00 00000000`00000000 ffff9007`18b337a8 : Wdf01000!FxIoQueue::DispatchEvents+0x40f [minkernel\wdf\framework\shared\irphandlers\io\fxioqueue.cpp @ 3125] 
ffff9007`18b33670 fffff800`2fbf9b0f     : ffffba8f`dff05a00 ffffba8f`ee449d00 ffffba8f`e2d990e0 00000000`00000001 : Wdf01000!FxPkgIo::DispatchStep1+0x558 [minkernel\wdf\framework\shared\irphandlers\io\fxpkgio.cpp @ 324] 
ffff9007`18b33730 fffff800`2fbf5d89     : ffffba8f`ea5f49b0 00000000`00000000 00000000`00000000 00000000`00000000 : Wdf01000!FxPkgIo::Dispatch+0x5f [minkernel\wdf\framework\shared\irphandlers\io\fxpkgio.cpp @ 119] 
ffff9007`18b33790 fffff800`2dcea5d5     : 00000000`00000000 ffffba8f`ed4ba9c0 ffffba8f`dee7b0a0 ffffba8f`ee449d50 : Wdf01000!FxDevice::DispatchWithLock+0x1f9 [minkernel\wdf\framework\shared\core\fxdevice.cpp @ 1447] 
ffff9007`18b337f0 fffff800`2e0e0e69     : ffffba8f`ee449d50 00000000`00000000 ffff9007`18b33b60 00000000`00000280 : nt!IofCallDriver+0x55
ffff9007`18b33830 fffff800`2e08d931     : ffff9007`18b33b60 ffff9007`18b33b60 00000000`42536f49 ffff9007`18b33b60 : nt!IopSynchronousServiceTail+0x189
ffff9007`18b338d0 fffff800`2e08d9d6     : 00000000`00000000 00000000`00000000 00000000`00000001 00000000`00000000 : nt!IopXxxControlFile+0xc61
ffff9007`18b33a00 fffff800`2de2cf85     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!NtDeviceIoControlFile+0x56
ffff9007`18b33a70 00000000`77631cf3     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x25
00000000`006ee838 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x77631cf3


FAULTING_SOURCE_LINE:  C:\cygwin64\tmp\build\source\internal-kvm-guest-drivers-windows\viosock\sys\Device.c

FAULTING_SOURCE_FILE:  C:\cygwin64\tmp\build\source\internal-kvm-guest-drivers-windows\viosock\sys\Device.c

FAULTING_SOURCE_LINE_NUMBER:  834

FAULTING_SOURCE_CODE:  
No source found for 'C:\cygwin64\tmp\build\source\internal-kvm-guest-drivers-windows\viosock\sys\Device.c'


SYMBOL_NAME:  viosock!VIOSockSelectCheckPkt+61

MODULE_NAME: viosock

IMAGE_NAME:  viosock.sys

STACK_COMMAND:  .cxr 0xffff900718b329c0 ; kb

BUCKET_ID_FUNC_OFFSET:  61

FAILURE_BUCKET_ID:  AV_viosock!VIOSockSelectCheckPkt

OS_VERSION:  10.0.20348.859

BUILDLAB_STR:  fe_release_svc_prod2

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

FAILURE_ID_HASH:  {60c77142-bdfa-e94a-2d31-45df89422ca6}

Followup:     MachineOwner
---------```

[MartinDrab]

Hello,

thank you for reporting the issue. Can you please upload the MEMORY.DMP file somewhere?

[tuxxi]

@MartinDrab here you go: https://drive.google.com/file/d/1RViT2W9-Y2ATpu67ofyn4xqD6yKdGhVh/view?usp=share_link

I used the .pdb from here to get the stack trace: https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/upstream-virtio/virtio-win-08032022.zip

[xuehuihui]

I also encountered the same problem

[gaojunhao]

I also encountered the same problem +1, any solutions?

[MartinDrab]

Hello,

sorry for the delay. Some time ago, I analyzed the dump and saw where probably the viosock driver crashed the system. However, I was not able to reproduce the issue (maybe I used wrong version of the driver or I messed up something else). I will try again soon, especially when new versions of the drivers gets out (I have some work to do there as well).

[gaojunhao]

Hello,

sorry for the delay. Some time ago, I analyzed the dump and saw where probably the viosock driver crashed the system. However, I was not able to reproduce the issue (maybe I used wrong version of the driver or I messed up something else). I will try again soon, especially when new versions of the drivers gets out (I have some work to do there as well).

You could try the latest version of vsock driver in master branch, then crash in the 835 line FAULTING_SOURCE_FILE: D:\workspace\new\kvm-guest-drivers-windows\viosock\sys\Device.c

FAULTING_SOURCE_LINE_NUMBER: 835

FAULTING_SOURCE_CODE:
832: pFds = &pPkt->pSelect->Fdss[FDSET_READ]; 833: pHandleSet = pPkt->Fds; 834:

835: pFds->fd_count = 0; 836: for (i = 0; i < pPkt->FdCount[FDSET_READ]; ++i) 837: { 838: PSOCKET_CONTEXT pSocket = GetSocketContext(pHandleSet[i].Socket); 839: if (pSocket->Events & (FD_ACCEPT | FD_READ | FD_CLOSE)) 840: { windows version: windows server 2019 datacenter, 1809