[WIP] Add prototypical SDL support.

[ecraven]

This is just a Merge Request to discuss, it is not ready to be merged in any way.

UPDATE: read along, all of the following is not true any longer ;)

Build with cmake. This will create a new backend (ldesdl).

• Resolution can only be set by editing the variables in sdl.c (UPDATE: edit the init_SDL call in main.c)
• Key repeat does not work.
• Still problems with keysyms that implicitly contain modifiers.
• The entire screen is bitblted onto the SDL display every frame.

[ecraven]

Open points:

• Right now, the framebuffer is "manually" copied into the SDL surface. Ideally, SDL would just directly use it, but SDL does not seem to support 1-bit per pixel data, so unless we switch to the color display code in maiko, I think we always need to copy the data. It seems to be fast enough, so there might be no need to optimize this with only copying the dirty rectangle. (UPDATE: see last bullet point about "fast enough")
• Key repeating doesn't work, I haven't investigated closely, but I don't actually see how it would work, because the bit is already set, setting it again won't do anything? I need to look at the XWINDOW code some more to understand how it works there.
• Right now there is no support for moving the area you see of the screen (the right and bottom scrollbar are missing, as well as the bit gravity thing). The window can be resized, but it will always anchor on the upper left.
• If you have keys that directly produce "shifted" keysyms, those won't work correctly. There's a problem here with the order in which SDL sends the keydown events (for example, it seems to send 9 down, shift, 9 down, the second of which is ignored because it is already pressed, so the key seen by Medley is 9, not '('), still need to find out how to fix this. UPDATE: turns out this is due to the "bitblt" being so slow that it sometimes makes SDL read all keyboard input at the same time. SDL does not add the OS timestamps correctly, but uses its own, so it sometimes reorders them. I'd consider this a bug in SDL, but it should be possible to work around this by just making the bitblt faster ;)

[ecraven]

UPDATE: new commit added, which is a bit faster at bitblting, and also supports pixel scaling (doubling, trippling, etc.)

[masinter]

Is key repeating done in Lisp? On the D-machines it must have been...

[ecraven]

@nbriggs I think with the last push it should now only update the damaged parts. I didn't look into expose events in SDL yet, not sure how well those are supported. If you have a bit of time, could you maybe try whether this works for you on a mac and even possibly on windows? If it seems to work ok, I'll refactor everything so that the diffs are easier to understand ;)

[nbriggs]

I can try on MacOS, assuming I can install/compile SDL without having to use "brew" (or anything like it). I don't have a Windows box.

[ecraven]

I can try on MacOS, assuming I can install/compile SDL without having to use "brew" (or anything like it). I don't have a Windows box.

That would be great! Please take care to use SDL2, not any of the older versions ;)

[nbriggs]

After installing the macOS SDL development framework...

CMake Error at CMakeLists.txt:77 (FIND_PACKAGE):
  By not providing "FindSDL2.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "SDL2", but
  CMake did not find one.

  Could not find a package configuration file provided by "SDL2" with any of
  the following names:

    SDL2Config.cmake
    sdl2-config.cmake

  Add the installation prefix of "SDL2" to CMAKE_PREFIX_PATH or set
  "SDL2_DIR" to a directory containing one of the above files.  If "SDL2"
  provides a separate development package or SDK, be sure it has been
  installed.

While they provide a development setup with a Framework to install, they forgot to include any cmake configuration stuff in there, so it'll take a while to sort that out.

[nbriggs]

The CMake vs SDL answer is: While the SDL (version 2) disk image installs it as SDL2.framework, the CMake application for macOS, which is distributed with a .cmake file for SDL, believes that it is always called "SDL" not "SDL2" but is happy to locate that and tell you it's version 2. So... I renamed everything to just plain SDL and it configured fine. However -- it won't compile because:

src/main.c:519:3: warning: implicit declaration of function 'init_SDL' is invalid in C99 [-Wimplicit-function-declaration]
  init_SDL(1600, 1024, 1);
  ^

and

src/initdsp.c:277:3: warning: implicit declaration of function 'sdl_bitblt_to_screen' is invalid in C99 [-Wimplicit-function-declaration]
  sdl_bitblt_to_screen(0, 0, sdl_displaywidth, sdl_displayheight);
  ^
1 warning generated.

and the actual fatal one:

src/keyevent.c.o
In file included from /Users/briggs/Projects/maiko-ecraven/src/keyevent.c:61:
/Users/briggs/Projects/maiko-ecraven/inc/xwinmandefs.h:3:10: fatal error: 'X11/X.h' file not found
#include <X11/X.h>     /* for Window */
         ^
1 error generated.

But after I fixed that, and a few more similar, something still knows that it used to call itself SDL2. Perhaps I can take the opposite approach, which is to copy /Applications/CMake.app/Contents/share/cmake-3.19/Modules/FindSDL.cmake to FindSDL2.cmake and fix up bits there to reflect the SDL2 name where appropriate. I'll also check and see if cmake-3.22 (RC1) has updated anything to do with SDL(2).

[masinter]

smilar problems in Windows https://www.matsson.com/prog/sdl2-mingw-w64-tutorial.php still didn't solve the cmake lacunae.

[nbriggs]

I haven't got all the kinks out of the SDL2.cmake (copied/modified from the CMake distributed SDL.cmake) but I got it far enough that lde and ldesdl link and run

Even with the selective update, the sdl_bitblt_to_screen() is way way too slow. Try doing FB *.*;* or bringing up a TEdit window on a moderate size file. The Maiko "color" code isn't the answer -- that's for a separate color window which isn't the main screen. Basically, the display code needs to do 1-bpp bitblt screamingly fast, which seems to be where XPutImage has an advantage over SDL.

[nbriggs]

The includes in sdl.c for SDL files should be just "SDL.h" and "SDL_keycode.h" -- the SDL/ is deprecated (according to the CMake people) because FreeBSD doesn't put these includes in an SDL subdirectory.

[ecraven]

Hello!

Thanks for playing around with this! So in theory at least this works "natively" in MacOS, with no need to run any sort of X server?

I haven't got all the kinks out of the SDL2.cmake (copied/modified from the CMake distributed SDL.cmake) but I got it far enough that lde and ldesdl link and run

Even with the selective update, the sdl_bitblt_to_screen() is way way too slow. Try doing "FB .;*" or bringing up a TEdit window on a moderate size file. The Maiko "color" code isn't the answer -- that's for a separate color window which isn't the main screen. Basically, the display code needs to do 1-bpp bitblt screamingly fast, which seems to be where XPutImage has an advantage over SDL.

I've changed things to only do the bitblt once on each "frame", it now "feels" as fast as the X version on my machine (though at some point I should add some profiling, to find out exact numbers).

The includes in sdl.c for SDL files should be just "SDL.h" and "SDL_keycode.h" -- the SDL/ is deprecated (according to the CMake people) because FreeBSD doesn't put these includes in an SDL subdirectory.

Thanks, changed it and pushed.

If it isn't too much trouble, could you pull in my changes and see whether they work better?

Thanks for the help!

Greetings, Peter

[nbriggs]

Yeah, I've been following along. It works much better now. FB *.*;* in the Lispusers directory is a good test for how it's doing on single character BLTCHAR, and that seems to be acceptably fast now.

[ecraven]

Yeah, I've been following along. It works much better now. FB *.*;* in the Lispusers directory is a good test for how it's doing on single character BLTCHAR, and that seems to be acceptably fast now.

Playing around with cygwin, not much success so far ;)

So, do you think this SDL thing is something worse pursuing further? Should I clean up the code, to maybe start on an actual real Merge Request?

Thanks for the help! Greetings, Peter

[nbriggs]

If we can sort out the input (and things such as doing the 3-button mouse emulation) it's probably worthwhile pursuing. When I do a bunch of FB redisplays, over a period of 35s runtime, the sdl_bitblt_to_screen() is responsible for 36% of the execution time -- so it's still more than it should be. Under X11, clipping_Xbitblt() which is the equivalent, is too small to show up.

[ecraven]

If we can sort out the input (and things such as doing the 3-button mouse emulation) it's probably worthwhile pursuing. When I do a bunch of FB redisplays, over a period of 35s runtime, the sdl_bitblt_to_screen() is responsible for 36% of the execution time -- so it's still more than it should be. Under X11, clipping_Xbitblt() which is the equivalent, is too small to show up.

If we can't fix it any other way, I can probably write a shader that works directly off the 1-bit bitmap, but as far as I know, that isn't portable (at least not to Metal on MacOS).

I've been playing around with cygwin, there are some places where #ifdef XWINDOW are missing, but I've gotten it to compile. However, SDL then doesn't find any available video adapter.. I actually have no idea how to properly compile stuff for windows, I'll try to find out more about this :-/

Thanks for the help! Peter

[ecraven]

OK, finally got an mxe setup working for cross-compiling. From what it looks like, this codebase has never been compiled on anything like windows before, right? So we would need to add all those #ifdef _WIN32 all over the place?

[nbriggs]

No #ifdef _WIN32 in the source files -- it already has, in maiko/platform.h

#if defined(_WIN32) || defined(__WINDOWS__)
#  define MAIKO_OS_WINDOWS 1
#  define MAIKO_OS_NAME "Windows"
#  define MAIKO_OS_DETECTED 1
#endif

The last time it was compiled on a Windows-like system it wasn't Windows, it was MSDOS, so there would be a lot of stuff to adjust, and there are things that will likely be MAIKO_OS_UNIX_LIKE (or not).

I forgot to mention the #if defined(XWINDOWS) around #include "xwinmandefs.h" that was necessary.

[ecraven]

I think I already added and pushed that, ran across that too.. so, what's the best way to proceed here? I'm no windows programmer, but I can try to fix things up. However, this is mostly unrelated to SDL, so maybe we should try to separate those two changesets. Should we discuss the next steps on monday?

[nbriggs]

I think a Windows change set is orthogonal to SDL, though enabled by it. The last change you made to the CMake configuration stopped it from compiling on macOS

[  2%] Linking C executable lde
ld: library not found for -lSDL2
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I think I need to make more changes in the FindSDL2.cmake. I believe all the variables it creates have to be SDL2_... instead of SDL_...

Yes, we should talk about it on Monday. I'll continue to poke at getting things working properly on the mac between now and then, though.

[ecraven]

Sorry, it's the SDL2::SDL2 vs. SDL2 alone thing.. for windows, only SDL2 alone worked, on linux I need SDL2::SDL2 too.. I need to learn more cmake If there's anything I can help with to make things easier for you, please tell me. If that seems like a reasonable approach, I'll let you work on MacOS stuff, and not touch things too much for now. If it works satisfactorily on MacOS too, I'll try to clean up this entire changeset. What are the things that are still missing, apart from the following?

• 3 button emulation: is there code for this? how should it work?
• command line parameters: I'll just add three parameters, width, height and pixelscale, ok?
• input: as the "problem" only appears in my very specific setup, I don't think this will be an actual problem for others.. does keyboard and mouse input work ok for you? any glitches there?

[nbriggs]

These two changes broke compilation on macOS and FreeBSD --

-  SET(MAIKO_DISPLAY_SDL_LIBRARIES SDL2::SDL2)
+  SET(MAIKO_DISPLAY_SDL_LIBRARIES SDL2)

-  TARGET_LINK_LIBRARIES(lde SDL2::SDL2)
+  TARGET_LINK_LIBRARIES(lde SDL2)

[nbriggs]

• 3 button emulation: is there code for this? how should it work? On the mac, XQuartz does this outside of any of our code. One can configure (for example) rCommand and rOption to make the mouse left button click come through as middle or right button. Usually you'd just have a 3-button mouse, but a mac trackpad or "normal" mouse has only one button (but can be configured for right button on two-finger click)
• command line parameters: I'll just add three parameters, width, height and pixelscale, ok? That would work for a few days. It should match the existing options for things like -t "window title" and -m memorysize and the same -sc WxH syntax for window size, otherwise we end up having to special case everything in the run-medley script for no good reason.
• input... There seem to be some glitches, but I can't characterize them yet. I need to systematically explore. It will be easier when I have all the mouse buttons.

[nbriggs]

BTW -- on FreeBSD, with default SDL2 install, it relies on the X11 back end, compiles/links without error, but generates

$ lde
requested width: 1888, height: 1088
initialised
Window created
Creating renderer...
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
X Error of failed request:  BadValue (integer parameter out of range for operation)
  Major opcode of failed request:  149 (GLX)
  Minor opcode of failed request:  3 (X_GLXCreateContext)
  Value in failed request:  0x0
  Serial number of failed request:  144
  Current serial number in output stream:  145

if you try to run it when ssh'd into the box with X11 forwarding back to an XQuartz mac.

[ecraven]

BTW -- on FreeBSD, with default SDL2 install, it relies on the X11 back end, compiles/links without error, but generates

$ lde
requested width: 1888, height: 1088
initialised
Window created
Creating renderer...
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
X Error of failed request:  BadValue (integer parameter out of range for operation)
  Major opcode of failed request:  149 (GLX)
  Minor opcode of failed request:  3 (X_GLXCreateContext)
  Value in failed request:  0x0
  Serial number of failed request:  144
  Current serial number in output stream:  145

if you try to run it when ssh'd into the box with X11 forwarding back to an XQuartz mac.

It looks like it is trying to open an OpenGL context, and I'm not sure how well that would work over the network :-/ This is probably something that SDL cannot support.

[ecraven]

ok, fixed the build on ubuntu. still trying to understand how that SDL2::SDL2 stuff in cmake works exactly, so that we can make it work on macos and ubuntu and other linuxen. might be related to old versions of libsdl2 not shipping all the correct cmake files :-/

[nbriggs]

Well all the references to X11 libraries are X11::X11 so with no understanding of what it's really doing I'm not surprised that the SDL2 libraries are SDL2::SDL2

[nbriggs]

In all of these (CMake driven) builds it needs to do both X11 and SDL

[ecraven]

In all of these (CMake driven) builds it needs to do both X11 and SDL

Yes, I'll change that, I just need to figure out how to tell cmake which of them to build (I've changed it to default to SDL, for testing, but there is some way to just tell it which to build). I'll add a matrix to the build.yml file once I figure out the details ;)

[nbriggs]

/Users/briggs/Projects/maiko-ecraven/src/sdl.c:1:10: fatal error: 'SDL2/SDL.h' file not found
#include <SDL2/SDL.h>
         ^
1 error generated.

No SDL2/ should be necessary.