Move to using small hook shim

[mikesart]

Our preloaded dso pulls in a ton of dependencies, uses a lot of memory, and generally does far too much work in the game process. And if the game crashes we're stuck trying to flush things in signal handlers.

Going to move to using a small C only OGL shim which only captures the GL call stream data and forwards to a separate process.

Current dependency list is pretty crazy:

mikesart@mikesart-petra:~/dev/voglproj/vogl_chroot.bitbucket/vogl (master)$ ldd ../vogl_build/libvogltrace64.so                                                                     
        linux-vdso.so.1 =>  (0x00007fff3bffe000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3aa7ee9000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3aa7ce5000)
        libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6 (0x00007f3aa7adc000)
        libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6 (0x00007f3aa78c0000)
        libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f3aa758b000)
        libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007f3aa7378000)
        libturbojpeg.so.0 => /usr/lib/x86_64-linux-gnu/libturbojpeg.so.0 (0x00007f3aa711e000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3aa6f16000)
        liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f3aa6cf3000)
        libSDL2-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libSDL2-2.0.so.0 (0x00007f3aa69ee000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3aa66ea000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3aa63e3000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3aa61cd000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3aa5e07000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3aab0bc000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f3aa5c01000)
        libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f3aa59e2000)
        libasound.so.2 => /usr/lib/x86_64-linux-gnu/libasound.so.2 (0x00007f3aa56f1000)
        libpulse-simple.so.0 => /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0 (0x00007f3aa54ed000)
        libpulse.so.0 => /usr/lib/x86_64-linux-gnu/libpulse.so.0 (0x00007f3aa52a4000)
        libXcursor.so.1 => /usr/lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007f3aa5099000)
        libXinerama.so.1 => /usr/lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007f3aa4e96000)
        libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007f3aa4c86000)
        libXrandr.so.2 => /usr/lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007f3aa4a7b000)
        libXss.so.1 => /usr/lib/x86_64-linux-gnu/libXss.so.1 (0x00007f3aa4877000)
        libXxf86vm.so.1 => /usr/lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x00007f3aa4671000)
        libwayland-egl.so.1 => /usr/lib/x86_64-linux-gnu/libwayland-egl.so.1 (0x00007f3aa446e000)
        libwayland-client.so.0 => /usr/lib/x86_64-linux-gnu/libwayland-client.so.0 (0x00007f3aa4261000)
        libwayland-cursor.so.0 => /usr/lib/x86_64-linux-gnu/libwayland-cursor.so.0 (0x00007f3aa4059000)
        libxkbcommon.so.0 => /usr/lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007f3aa3e1e000)
        libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f3aa3c1a000)
        libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f3aa3a13000)
        libpulsecommon-4.0.so => /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-4.0.so (0x00007f3aa37ac000)
        libjson-c.so.2 => /lib/x86_64-linux-gnu/libjson-c.so.2 (0x00007f3aa35a0000)
        libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007f3aa335b000)
        libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f3aa3151000)
        libXfixes.so.3 => /usr/lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007f3aa2f4a000)
        libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f3aa2d42000)
        libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007f3aa2b37000)
        libsndfile.so.1 => /usr/lib/x86_64-linux-gnu/libsndfile.so.1 (0x00007f3aa28cf000)
        libasyncns.so.0 => /usr/lib/x86_64-linux-gnu/libasyncns.so.0 (0x00007f3aa26c9000)
        libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f3aa24ae000)
        libFLAC.so.8 => /usr/lib/x86_64-linux-gnu/libFLAC.so.8 (0x00007f3aa227d000)
        libvorbisenc.so.2 => /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007f3aa1dae000)
        libvorbis.so.0 => /usr/lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007f3aa1b80000)
        libogg.so.0 => /usr/lib/x86_64-linux-gnu/libogg.so.0 (0x00007f3aa1977000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f3aa175c000)

[armyofzin]

I was thinking about this last week. What are the plans for the IPC layer? e.g. boost.asio, boost::interprocess, etc.

[bmerry]

tl;dr version: it's definitely worth avoiding dependencies as far as possible, including libstdc++.

I can offer some of my experience as the author of BuGLe. I started writing BuGLe in C++, but I ran into problems when trying to interpose commercial games. The games typically shipped all their own libraries to avoid any version mismatches on the end-user system, but that meant that the libraries didn't match the libraries that got sucked in by BuGLe's preloaded DSO. Bad things happen if you try to use two different versions of libstdc++ in the same process. This was a long time ago, but I suspect things haven't improved much. And I don't want to even think about what will happen if you end up with libc++ and libstdc++ in the same process.

My solution for BuGLe was to do everything to C, but these days you have the option of using -static-libstdc++ (which then obviously means you need an equivalent option in any compiler you want to support). Of course, you'll also need to ensure that any other DSO dependencies you pull in don't end up linking to the shared libstdc++, so if you use Boost you'll also want to link that statically.

The other thing to watch out for is that the performance of the interposition layer is pretty critical, because GL involves a lot of API calls. If the data is being passed to another process, be sure to buffer it, rather than turning each API call into an IPC round trip.

[armyofzin]

I failed to parse the "C only" part of Mike's comment. No boost.

[mikesart]

I may try to get our new voglhook shim building with C++ and statically link libstdc++ initially (using -static-libgcc -static-libstdc++), but my experiences with that so far (testing on voglperf) indicate it can be a bit fragile. Others seem to back that up:

http://micro.nicholaswilson.me.uk/post/31855915892/rules-of-static-linking-libstdc-libc-libgcc http://mkwei.blogspot.com/2013/12/linking-libstdc-statically.html

libgcc_s.so and libstdc++.so are the two libraries C++ on Linux pulls in. I'm also guessing we will run into similar issues on other platforms and going straight C will save us headaches down the road even if we can get it to work on Linux. At RAD we did everything in C and it was glorious for porting and minimal invasive app goals.

And I don't want to even think about what will happen if you end up with libc++ and libstdc++ in the same process.

They aren't ABI compatible, so yeah - best case is linker errors, which they've theoretically tried to make happen.

http://lists.cs.uiuc.edu/pipermail/cfe-dev/2012-September/024594.html

What are the plans for the IPC layer?

Either shared memory or sockets I think. Given Bruce's comments, it sounds like we need some sort of memory block to buffer this stuff anyway so perhaps I'll go that route first.

Thanks for your advice Bruce - appreciate it.

[SirAnthony]

We using c-only ogl library in glsl-debugger. As for now it is lack of architecture, and using shared memory for ipc. Shared memory It is a bad way to do things, if you interesting, I had days of debugging of this crap and fed to the teeth. We are planning to rewrite it to use sockets-based ipc. Do you have any plans on ipc protocol? I think, it will be nice to have some standard protocol for different apps to communicate.

[mikesart]

We using c-only ogl library in glsl-debugger

This is your cpyFromProcess / cpyToProcess stuff?

I think I'm going to tackle this in stages.

• First switch from .spec to using the xml files from Khronos here: https://cvs.khronos.org/svn/repos/ogl/trunk/doc/registry/public/api/
• Then I'm going to switch away from the current macro method (which can be extremely difficult to debug) and do the hooking more like how https://github.com/nvMcJohn/gfxtrace or BuGLe does.

At that point we should have similar functionality to what we have right now but quite a bit cleaner. Then tackle trimming C++ and the other dependencies, etc.

So no plans on ipc protocol yet. Was going to look at this libuv thing for the socket IPC though: https://github.com/joyent/libuv

[SirAnthony]

cpyFromProcess / cpyToProcess

It is just weird method to do shared memory things. You need whole DebugLib folder, it is where all c library is.

First switch from .spec to using the xml files from Khronos here:

I wanted to do it too, even start to implement a python generator based on khronos one, but lack of time and some live circumstances stuck me on.

By protocol I mean data format, not the implementation of sockets. Actually, it must not be hard to client application to receive some data about current trace state and then interpret it way client want.