2.6.1um in Linux, crashes when starting new game or demo in OpenGL

[Mrhnhrm]

Hello again. Alas, it seems like my return to Doom is ripe with game-crashing problems.

If OpenGL rendering is used, when starting a new game or waiting for a demo to start, the game crashes with a floating point exception.

Here is the debugger output at the offending line of code.

Thread 1 "prboom-plus" hit Breakpoint 1, gld_FrustrumSetup () at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/gl_clipper.c:438
438       NORMALIZE_PLANE(4);
(gdb) bt
#0  gld_FrustrumSetup ()
    at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/gl_clipper.c:438
#1  0x000055555563add5 in R_RenderPlayerView (player=0x555555733700 <players>)
    at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/r_main.c:1124
#2  0x00005555555ae156 in D_Display (frac=65536)
    at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/d_main.c:369
#3  0x00005555555b04f4 in D_DoomLoop ()
    at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/d_main.c:513
#4  D_DoomMain () at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/d_main.c:2259
#5  0x00005555555a3c23 in main (argc=<optimized out>, argv=0x7fffffffd728)
    at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/SDL/i_main.c:607
(gdb) n

Thread 1 "prboom-plus" received signal SIGFPE, Arithmetic exception.
0x0000555555662e1e in gld_FrustrumSetup () at /usr/src/debug/games-fps/prboom-plus-2.6.1/prboom-plus-2.6.1um/prboom2/src/gl_clipper.c:438
438       NORMALIZE_PLANE(4);

Maybe my config file will be necessary too: https://pastebin.com/3aAwJQ2S

Please let me know if there is anything else I should provide. Thanks in advance.

[JadingTsunami]

What hardware are you running on? And the software renderer is running OK?

[Mrhnhrm]

Software rendering works more or less fine (doesn't crash at least). OpenGL crashes regardless of resolution or whether fullscreen/windowed. Some system specifications:

• Ryzen 5 2600
• Radeon RX 480
• 64 bit Gentoo Linux, kernel version 5.13.14
• driver amdgpu 19.1.0
• Mesa 21.1.7
• X.Org X11 v. 1.20.13
• gcc 11.2.0 (prboom-plus is built from sources as an official package)

[JadingTsunami]

Yes, that's helpful. Could you also post the full console output?

[Mrhnhrm]

You probably won't believe in this nonsense. It appears that the problem is... Fluidsynth! Specifically, Fluidsynth built with extra debug messages. I still had this debug version of Fluidsynth as a holdover from my previous bug report. The Fluidsynth debug messages seemed irrelevant; so in order to clean up the console log, I installed regular Fluidsynth.

Now prboom+ doesn't crash in OpenGL.

Refusing to believe what I am seeing, I then install the debug variant of Fluidsynth again.

Prboom+ crashes again in OpenGL. Well, that's slightly comforting, even if in a creepy manner. The console log is attached. opengl_fluidsynth_crashlog.txt

[fabiangreffrath]

Maybe some clash of global variables? SIGFPE means something like division by zero or sqrt() of a negative value, so somehow the values of frustum[][] seem to be miscalcualted. Does it help if you declare that as static in gl_clipper.c?

[JadingTsunami]

Maybe some clash of global variables? SIGFPE means something like division by zero or sqrt() of a negative value, so somehow the values of frustum[][] seem to be miscalcualted. Does it help if you declare that as static in gl_clipper.c?

Hmm, interesting idea and I think it is possible. The globals that derive the frustum are modelMatrix and projMatrix which are not terribly unique names. I don't see those names in the FluidSynth source but it may be further upstream somewhere.

[fabiangreffrath]

@Mrhnhrm Could you please try to rename modelMatrix and projMatrix to something more unique across the sources, e.g. PRmodelMatrix and PRprojMatrix or something similar? Also, please declare frustum[][] as static in gl_clipper.c?

[Mrhnhrm]

I'll give it a try. It might take me a while to figure out how to precisely repeat the build procedure that the Gentoo package manager uses. I will report then.

[Mrhnhrm]

@fabiangreffrath ... I ended up creating a local package ebuild file to use the updated source, copying everything from the original ebuild and just changing the version number. And as near as I can tell, the resulting binary crashes the very same way as in my initial report and backtrace. To be a bit more specific now:

Thread 1 "prboom-plus" hit Breakpoint 1, gld_FrustrumSetup () at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/gl_clipper.c:438
438       NORMALIZE_PLANE(4);
(gdb) p frustum
$1 = {{0.599999726, 0, 0.800000191, 27.5500031}, {-0.599999785, 0, 0.800000012, 17.6500015}, {
    -4.63339873e-08, 0.847998261, 0.529999018, 14.7008476}, {-4.63339873e-08, -0.847998261, 0.529999018, 
    15.2440968}, {0, 0, 0, 0}, {0, 0, 0, 0}}
(gdb) p frustum[4]
$2 = {0, 0, 0, 0}
(gdb) n

Thread 1 "prboom-plus" received signal SIGFPE, Arithmetic exception.
0x0000555555662e1e in gld_FrustrumSetup () at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/gl_clipper.c:438
438       NORMALIZE_PLANE(4);
(gdb) bt
#0  0x0000555555662e1e in gld_FrustrumSetup ()
    at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/gl_clipper.c:438
#1  0x000055555563add5 in R_RenderPlayerView (player=0x555555733700 <players>)
    at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/r_main.c:1124
#2  0x00005555555ae156 in D_Display (frac=65536)
    at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/d_main.c:369
#3  0x00005555555b04f4 in D_DoomLoop ()
    at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/d_main.c:513
#4  D_DoomMain ()
    at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/d_main.c:2259
#5  0x00005555555a3c23 in main (argc=<optimized out>, argv=0x7fffffffd748)
    at /usr/src/debug/games-fps/prboom-plus-2.6.1999/prboom-plus-2.6.1999um/prboom2/src/SDL/i_main.c:607

Perhaps I should mention that while digging the aforementioned ebuild file, I noticed that the package manager includes another patch: prboom-plus-2.6.1-Guard-OpenGL-call-in-D_Display-with-GL_DOOM.patch.txt

[fabiangreffrath]

This should keep it from crashing, but does the game still work?

--- a/prboom2/src/gl_clipper.c
+++ b/prboom2/src/gl_clipper.c
@@ -372,6 +372,7 @@ angle_t gld_FrustumAngle(void)
     frustum[i][0] * frustum[i][0] + \
     frustum[i][1] * frustum[i][1] + \
     frustum[i][2] * frustum[i][2]); \
+  if (t == 0.0f) t = 1.0f; \
   frustum[i][0] /= t; \
   frustum[i][1] /= t; \
   frustum[i][2] /= t; \

[Mrhnhrm]

It would be interesting to try. But alas, I don't know when I'll have time for it. It might take several weeks before I catch a break. I'll report when I get to it.