Daemon icon indicating copy to clipboard operation
Daemon copied to clipboard

segfault in daemon when lacking memory

Open ghost opened this issue 2 years ago • 5 comments

I just experienced a weird bug: engine was segfaulting while loading a map. I was trying to join an online server, the map screen was loading, and I had the segfault before it ends. I tried with the 3 currently modded instance, same result.

After closing the web browsers I had opened, I tried anew: no more segfault. I do not know the exact commit from which this engine build comes from, it is very likely some upstream build compiled in release mode (had to do this to longer have the memleak). For certain, it's more recent than the release.

The symptoms and absence of them after freeing memory by closing notorious memory ogres makes me think there is a C dynamic allocation which have a non verified result, and when malloc or calloc returns NULL, value which got dereferenced later, leading to a segfault.

ghost avatar Nov 01 '23 23:11 ghost

Yeah there are some C-style allocator functions like Com_Allocate_Aligned and Z_Malloc. We should change them to call Sys::Error instead of returning null on failure. And also get rid of the handful of raw mallocs.

slipher avatar Nov 02 '23 00:11 slipher

Sys::Error might still not give much hint on what happens. Perhaps it would be interesting to start by reserving an error popup at start of process (thus, this one would not be affected by failure in future allocations) and use it when an unrecoverable error happens, like an out of memory one? Or, in the case of oom, it would be possible to wait for user to free some memory before crashing? That one sounds quite hard for a rather little usefulness though (or just use exceptions, and on uncaught one, use that memory to show a proper error, which would lean to a clean, instant, exit() when "ack" is clicked on? dunno.).

ghost avatar Nov 02 '23 00:11 ghost

Sys::Error might still not give much hint on what happens.

It would say "Out of memory", what more do you expect?

slipher avatar Nov 02 '23 00:11 slipher

If it reports meaningful messages to the user, then it's good. It's not something I'm used to with many programs, so I'm a bit wary, sorry.

ghost avatar Nov 02 '23 00:11 ghost

Just for the fun of it, it is possible to run the game with 6.7GiB free, 12GiB available, but this segfault still happens, that RAM is not enough for engine. The crash will happen (consistently) when I do devmap plat23 in those conditions:

% LANG=C free -h; ./games/UnvanquishedUpdater 
               total        used        free      shared  buff/cache   available
Mem:            14Gi       2.1Gi       6.7Gi       477Mi       6.6Gi        12Gi
Swap:             0B          0B          0B
Git version: v0.2.0
3 stored settings:
 - "settings/commandLineParameters" = "%command%"
 - "settings/currentVersion" = "0.54.0"
 - "settings/installPath" = "/home/berenger/Unvanquished"
Testing exception handling...
Exception handling works
QFontDatabase: Cannot find font directory /qt/lib/fonts.
Note that Qt no longer ships fonts. Deploy some (from https://dejavu-fonts.github.io/ for example) or switch to fontconfig.
CurrentVersionFetcher: fetched versions: updater = "v0.2.0" game = "0.54.0"
Previously-installed game version: "0.54.0"
Starting game with command line: "\"/home/berenger/Unvanquished/daemon\""
Unvanquished 0.54.0 Linux amd64 Oct  4 2023
cmdline:                                                                                                                                                                [FS] Lib path: /home/berenger/Unvanquished 
[FS] Home path: /home/berenger/.local/unvanquished 
[FS] Pak search path: /home/berenger/Unvanquished/pkg 
[FS] Pak search path: /home/berenger/.local/unvanquished/pkg 
[FS] Loading pak '/home/berenger/.local/unvanquished/pkg/unvanquished_0.54.0.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/tex-common_2.3.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-players_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-legacy_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-weapons_0.54-dirty.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-buildables_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-voices_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-soundtrack_0.54.dpk'... 
execing 'default.cfg' 
execing 'presets/input/wasd.cfg' 
execing 'autogen.cfg' 
execing 'keybindings.cfg' 
execing 'autoexec.cfg' 
----- Client Initialization ----- 
Loading RSA keys from pubkey 
Daemon RSA public-key found. 
----- Client Initialization Complete ----- 
IP: 127.0.0.1 
IP: 192.168.1.110 
IP6: ::1 
IP6: fd24:fb65:397b:3900:7285:c2ff:fea6:e36 
IP6: fe80::7285:c2ff:fea6:e36%eth1 
Opening IP6 socket: [::]:* 
Opening IP socket: 0.0.0.0:* 
--- Common Initialization Complete --- 
Calling GetRefAPI… 
SDL_Init( SDL_INIT_VIDEO )...  
Using SDL version 2.26.5 
SDL using driver "x11" 
Initializing OpenGL display 
Using GLEW version 2.2.0 
Display aspect: 1.779 
...setting mode -1: 1366×768 
Using preferred context - 24-bit OpenGL 3.2 core 
OpenGL Renderer: Mesa Intel(R) UHD Graphics 630 (CFL GT2) 
Warn: Provided OpenGL 4.6 is not the same as requested 3.2 version 
Using GL3 Renderer in OpenGL 3.x mode... 
Available modes: '1280x720 1366x768 640x480 800x600 1024x768 1280x1024 ' 
Detected graphics driver class 'OpenGL 3+' 
Detected graphics hardware class 'generic' 
Initializing OpenGL extensions 
...ignoring GL_ARB_debug_output 
...found shading language version 460 
...using GL_ARB_half_float_pixel 
...using GL_ARB_texture_float 
...using GL_EXT_gpu_shader4 
...using GL_EXT_texture_integer 
...using GL_ARB_texture_rg 
...using GL_ARB_texture_gather 
...using GL_EXT_texture_compression_s3tc 
...using GL_ARB_texture_compression_rgtc 
...using GL_EXT_texture_filter_anisotropic 
...using GL_ARB_half_float_vertex 
...using GL_ARB_framebuffer_object 
...using GL_ARB_get_program_binary 
...using GL_ARB_buffer_storage 
...using GL_ARB_uniform_buffer_object 
...using GL_ARB_map_buffer_range 
...using GL_ARB_sync 
GL_VENDOR: Intel 
GL_RENDERER: Mesa Intel(R) UHD Graphics 630 (CFL GT2) 
GL_VERSION: 4.6 (Core Profile) Mesa 22.3.6 
GL_MAX_TEXTURE_SIZE: 16384 
GL_SHADING_LANGUAGE_VERSION: 4.60 
GL_MAX_VERTEX_UNIFORM_COMPONENTS 16384 
GL_MAX_VERTEX_ATTRIBS 16 
Occlusion query bits: 64 
GL_MAX_DRAW_BUFFERS: 8 
GL_TEXTURE_MAX_ANISOTROPY_EXT: 16.000000 
GL_MAX_RENDERBUFFER_SIZE: 16384 
GL_MAX_COLOR_ATTACHMENTS: 8 
PIXELFORMAT: color(24-bits) 
MODE: -1, 1366 x 768 windowed hz: N/A 
Using OpenGL version 4.6, requested: 3.2 
Using OpenGL 3.x context. 
Using an OpenGL core profile. 
OpenGL 3.x context is not forward compatible. 
Using OpenGL extensions: GL_ARB_half_float_pixel GL_ARB_texture_float GL_EXT_gpu_shader4 GL_EXT_texture_integer GL_ARB_texture_rg GL_ARB_texture_gather GL_EXT_texture_compression_s3tc GL_ARB_texture_compression_rgtc GL_EXT_texture_filter_anisotropic GL_ARB_half_float_vertex GL_ARB_framebuffer_object GL_ARB_get_program_binary GL_ARB_buffer_storage GL_ARB_uniform_buffer_object GL_ARB_map_buffer_range GL_ARB_sync 
Using S3TC (DXTC) texture compression. 
Using GPU vertex skinning with max 256 bones in a single pass, models are hardware accelerated. 
Extracting VM module cgame-amd64.nexe from /home/berenger/.local/unvanquished/pkg/unvanquished_0.54.0.dpk...
 
Loading VM module cgame-amd64.nexe... 
Using loader args:  /home/berenger/Unvanquished/nacl_helper_bootstrap /home/berenger/Unvanquished/nacl_loader --r_debug=0xXXXXXXXXXXXXXXXX --reserved_at_zero=0xXXXXXXXXXXXXXXXX -v -B /home/berenger/Unvanquished/irt_core-amd64.nexe -e -i 100:27 -- /home/berenger/.local/unvanquished/cgame-amd64.nexe 100 
Loaded VM module in 95 msec 
Loading Lua plugin using a new Lua state. 
Loaded font face 'unifont' [regular] from 'fonts/unifont.ttf'. 
Loaded font face 'Roboto' [bold] from 'fonts/Roboto-Bold.ttf'. 
Loaded font face 'Roboto' [italic, bold] from 'fonts/Roboto-BoldItalic.ttf'. 
Loaded font face 'Roboto' [italic] from 'fonts/Roboto-Italic.ttf'. 
Loaded font face 'Roboto' [regular] from 'fonts/Roboto-Regular.ttf'. 
Loaded font face 'FontAwesome' [regular] from 'fonts/fontawesome-webfont.ttf'. 
Loaded font face 'DejaVu Sans' [regular] from 'fonts/DejaVuSansCondensed.ttf'. 
Loaded font face 'Material Icons' [regular] from 'fonts/MaterialIcons-Regular.ttf'. 
]/devmap plat23 
----- Server Initialization ----- 
Map: plat23 
[FS] Loading pak '/home/berenger/.local/unvanquished/pkg/unvanquished_0.54.0.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/tex-common_2.3.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-players_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-legacy_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-weapons_0.54-dirty.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-buildables_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-voices_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-soundtrack_0.54.dpk'... 
IP: 127.0.0.1 
IP: 192.168.1.110 
IP6: ::1 
IP6: fd24:fb65:397b:3900:7285:c2ff:fea6:e36 
IP6: fe80::7285:c2ff:fea6:e36%eth1 
Opening IP6 socket: [::]:27960 
Opening IP socket: 0.0.0.0:27960 
[FS] Loading pak '/home/berenger/.local/unvanquished/pkg/unvanquished_0.54.0.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/tex-common_2.3.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-players_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-legacy_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-weapons_0.54-dirty.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-buildables_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-voices_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/res-soundtrack_0.54.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/map-plat23_1.13.4.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/tex-pk02_1.3.dpk'... 
[FS] Loading pak '/home/berenger/Unvanquished/pkg/tex-space_1.3.dpk'... 
Extracting VM module sgame-amd64.nexe from /home/berenger/.local/unvanquished/pkg/unvanquished_0.54.0.dpk...
 
Loading VM module sgame-amd64.nexe... 
Using loader args:  /home/berenger/Unvanquished/nacl_helper_bootstrap /home/berenger/Unvanquished/nacl_loader --r_debug=0xXXXXXXXXXXXXXXXX --reserved_at_zero=0xXXXXXXXXXXXXXXXX -v -B /home/berenger/Unvanquished/irt_core-amd64.nexe -e -i 100:32 -- /home/berenger/.local/unvanquished/sgame-amd64.nexe 100 
Loaded VM module in 34 msec 
------- Game Initialization ------- 
gamename: base 
gamedate: Jan 30 2023 
Warn: readconfig: could not open admin config file admin.dat 
Using layout "default" from list (default ). 
Loading map configs for map: plat23 layout: default 
0 groups with 0 entities 
----------------------------------- 
-----------------------------------
 
Client 0 connecting 
Calling GetRefAPI… 
GL_VENDOR: Intel 
GL_RENDERER: Mesa Intel(R) UHD Graphics 630 (CFL GT2) 
GL_VERSION: 4.6 (Core Profile) Mesa 22.3.6 
GL_MAX_TEXTURE_SIZE: 16384 
GL_SHADING_LANGUAGE_VERSION: 4.60 
GL_MAX_VERTEX_UNIFORM_COMPONENTS 16384 
GL_MAX_VERTEX_ATTRIBS 16 
Occlusion query bits: 64 
GL_MAX_DRAW_BUFFERS: 8 
GL_TEXTURE_MAX_ANISOTROPY_EXT: 16.000000 
GL_MAX_RENDERBUFFER_SIZE: 16384 
GL_MAX_COLOR_ATTACHMENTS: 8 
PIXELFORMAT: color(24-bits) 
MODE: -1, 1366 x 768 windowed hz: N/A 
Using OpenGL version 4.6, requested: 3.2 
Using OpenGL 3.x context. 
Using an OpenGL core profile. 
OpenGL 3.x context is not forward compatible. 
Using OpenGL extensions: GL_ARB_half_float_pixel GL_ARB_texture_float GL_EXT_gpu_shader4 GL_EXT_texture_integer GL_ARB_texture_rg GL_ARB_texture_gather GL_EXT_texture_compression_s3tc GL_ARB_texture_compression_rgtc GL_EXT_texture_filter_anisotropic GL_ARB_half_float_vertex GL_ARB_framebuffer_object GL_ARB_get_program_binary GL_ARB_buffer_storage GL_ARB_uniform_buffer_object GL_ARB_map_buffer_range GL_ARB_sync 
Using S3TC (DXTC) texture compression. 
Using GPU vertex skinning with max 256 bones in a single pass, models are hardware accelerated. 
Extracting VM module cgame-amd64.nexe from /home/berenger/.local/unvanquished/pkg/unvanquished_0.54.0.dpk...
 
Loading VM module cgame-amd64.nexe... 
Using loader args:  /home/berenger/Unvanquished/nacl_helper_bootstrap /home/berenger/Unvanquished/nacl_loader --r_debug=0xXXXXXXXXXXXXXXXX --reserved_at_zero=0xXXXXXXXXXXXXXXXX -v -B /home/berenger/Unvanquished/irt_core-amd64.nexe -e -i 100:35 -- /home/berenger/.local/unvanquished/cgame-amd64.nexe 100 
Loaded VM module in 94 msec 
Loading Lua plugin using a new Lua state. 
Loaded font face 'unifont' [regular] from 'fonts/unifont.ttf'. 
Loaded font face 'Roboto' [bold] from 'fonts/Roboto-Bold.ttf'. 
Loaded font face 'Roboto' [italic, bold] from 'fonts/Roboto-BoldItalic.ttf'. 
Loaded font face 'Roboto' [italic] from 'fonts/Roboto-Italic.ttf'. 
Loaded font face 'Roboto' [regular] from 'fonts/Roboto-Regular.ttf'. 
Loaded font face 'FontAwesome' [regular] from 'fonts/fontawesome-webfont.ttf'. 
Loaded font face 'DejaVu Sans' [regular] from 'fonts/DejaVuSansCondensed.ttf'. 
Loaded font face 'Material Icons' [regular] from 'fonts/MaterialIcons-Regular.ttf'. 
Warn: Crashed with signal 11: Segmentation fault 
----- Server Shutdown ----- 
==== ShutdownGame ==== 

ghost avatar Nov 07 '23 19:11 ghost