Stack overflow in SDL_CreateWindow (with Vulkan Flag)

[abhineet99]

Stackoverflow link: https://stackoverflow.com/questions/66691721/any-info-on-this-sdl-createwindow-error-on-windows

Env: Windows10 Visual Studio 2019 SDL (tried with 2.0.14, building from current repo) Vulkan 1.2.154.1 Code snippet:

SDL_Init(SDL_INIT_VIDEO);

SDL_WindowFlags window_flags = (SDL_WindowFlags)(SDL_WINDOW_VULKAN);

_window = SDL_CreateWindow(
    "Vulkan Engine",
    SDL_WINDOWPOS_UNDEFINED,
    SDL_WINDOWPOS_UNDEFINED,
    _windowExtent.width,
    _windowExtent.height,
    window_flags
);

Line by line debugging points to this as the line where things break:

return CallWindowProc(data->wndproc, hwnd, msg, wParam, lParam); in SDL_windowsevents.c.

Exception:

Exception thrown at 0x00007FFF584A4F8E (ntdll.dll) in GraphicsModel.exe: 0xC00000FD: Stack 
overflow (parameters: 0x0000000000000001, 0x000000ED25003FB8).
Unhandled exception at 0x00007FFF584A4F8E (ntdll.dll) in GraphicsModel.exe: 0xC000041D: An unhandled exception was encountered during a user callback.

Callstack: https://pastebin.com/Y3zSWU3w

[icculus]

Just copying the callstack into this issue so we don't lose the pastebin contents:

 	ntdll.dll!00007ffb84e44f8e()	Unknown
 	user32.dll!00007ffb83345aa4()	Unknown
 	user32.dll!00007ffb833458ee()	Unknown
 	mfc140ud.dll!00007ffaf0091b05()	Unknown
 	user32.dll!00007ffb83345c1d()	Unknown
 	user32.dll!00007ffb833458ee()	Unknown
 	SDL2.dll!0000000075e4e7d0()	Unknown
 	user32.dll!00007ffb83345c1d()	Unknown
 	user32.dll!00007ffb833458ee()	Unknown
 	mfc140ud.dll!00007ffaf0091b05()	Unknown
 	user32.dll!00007ffb83345c1d()	Unknown
 	user32.dll!00007ffb833458ee()	Unknown



	THIS BLOCK REPEATS 'n' times
	---------------------------------------

 	SDL2.dll!0000000075e4e7d0()	Unknown
 	user32.dll!00007ffb83345c1d()	Unknown
 	user32.dll!00007ffb833458ee()	Unknown
 	mfc140ud.dll!00007ffaf0091b05()	Unknown
 	user32.dll!00007ffb83345c1d()	Unknown
 	user32.dll!00007ffb833458ee()	Unknown

	---------------------------------------


 	SDL2.dll!0000000075e4e7d0()	Unknown
 	user32.dll!00007ffb83345c1d()	Unknown
 	user32.dll!00007ffb833457ec()	Unknown
 	user32.dll!00007ffb83354432()	Unknown
>	ntdll.dll!00007ffb84ec0b04()	Unknown
 	win32u.dll!00007ffb82ea1524()	Unknown
 	SDL2.dll!0000000075e58ebe()	Unknown
 	SDL2.dll!0000000075e2172d()	Unknown
 	GraphicsModel.exe!VulkanEngine::init() Line 59	C++
 	GraphicsModel.exe!CMainFrame::OnCreate(tagCREATESTRUCTW * lpCreateStruct) Line 67	C++
 	[External Code]	
 	GraphicsModel.exe!CGraphicsModelApp::InitInstance() Line 106	C++
 	[External Code]	
 	GraphicsModel.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * hPrevInstance, wchar_t * lpCmdLine, int nCmdShow) Line 26	C++
 	[External Code]	

[abhineet99]

Was using the code in an MFC application. Have a theory that this might have happened due to a window instance already being used by the application. Simple window creation is working in a solitary project which just creates this SDL window. Closing the issue.

[gizmocuz]

And how would you create a windows in a MFC application? It is still crashing

[madebr]

I don't know whether this is supported, but creating a SDL_Window from a CDialog or a CMDIChildWndEx (=MDI child window) does not work. When used in CDialog, SDL_CreateWindowWithProperties and SDL_CreateRenderer succeed, but only render a black window. When used in CMDIChildWndEx, a stack overflow happens, very similar to the one given by OP.

I added 2 test programs: testmfcdialog and testmfcmdi at https://github.com/madebr/SDL/tree/testmfc-dialog

The question remains, do we support this, and if we do, how can this be done? A MFC MDI application with SDL subwindows looks like a reasonable usage for me.