`clipper_inject` does not catch `dlopen` calls that bring in `libssl.so` (either directly or as dependency)

[lf-]

So I tried running Clipper on Python requests and to my surprise it did not work! But, this is actually revelatory of a missing feature: clipper_inject does not catch dynamic library loads and accordingly add new hooks for them if they are new loads.

Context: We can see that when python -c "import _ssl" is executed it loads a new .so that depends on libssl.so, so of course clipper_inject does not see it:

co/cpython - [tags/v3.9.6] » rr replay
Reading symbols from /usr/bin/python3.11...
Downloading separate debug info for /usr/bin/python3.11
Reading symbols from /home/jade/.cache/debuginfod_client/9efa8fdb1fce89c7a9f29802398a366b6c913a3e/debuginfo...      
Remote debugging using 127.0.0.1:20967
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /home/jade/.cache/debuginfod_client/f72a7ea28ae7a23cd90cdcc5537be1efa37c7b36/debuginfo...      
BFD: warning: system-supplied DSO at 0x6fffd000 has a section extending past end of file
Downloading separate debug info for system-supplied DSO at 0x6fffd000
0x00007f32436e6d70 in _start () from /lib64/ld-linux-x86-64.so.2                                                    
=> 0x00007f32436e6d70 <_start+0>:       48 89 e7                mov    %rsp,%rdi
(rr) catch load
Catchpoint 1 (load)
(rr) c
Continuing.
Downloading separate debug info for /usr/lib/libpython3.11.so.1.0
                                                                                                                    
Catchpoint 1
  Inferior loaded /usr/bin/../lib64/rr/librrpreload.so
    /usr/lib/libpython3.11.so.1.0
    /usr/lib/libc.so.6
    /usr/lib/libm.so.6
dl_main (phdr=<optimized out>, phnum=<optimized out>, user_entry=<optimized out>, auxv=<optimized out>)
    at rtld.c:2406
Downloading source file /usr/src/debug/glibc/glibc/elf/rtld.c
2406      LIBC_PROBE (init_complete, 2, LM_ID_BASE, r);                                                             
(rr) c
Continuing.
Downloading separate debug info for /usr/lib/python3.11/lib-dynload/_ssl.cpython-311-x86_64-linux-gnu.so
Downloading separate debug info for /usr/lib/libssl.so.3                                                            
Downloading separate debug info for /usr/lib/libcrypto.so.3                                                         
                                                                                                                    
Catchpoint 1
  Inferior loaded /usr/lib/python3.11/lib-dynload/_ssl.cpython-311-x86_64-linux-gnu.so
    /usr/lib/libssl.so.3
    /usr/lib/libcrypto.so.3
dl_open_worker_begin (a=a@entry=0x7ffe10911740) at dl-open.c:754
Downloading source file /usr/src/debug/glibc/glibc/elf/dl-open.c
754         LIBC_PROBE (reloc_complete, 3, args->nsid, r, new);                                                     
(rr) quit
Detaching from program: /usr/bin/python3.11, process 2380214
[Inferior 1 (process 2380214) detached]
co/cpython - [tags/v3.9.6] » ldd /usr/lib/python3.11/lib-dynload/_ssl.cpython-311-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007f6910ce9000)
        libssl.so.3 => /usr/lib/libssl.so.3 (0x00007f6910bc4000)
        libcrypto.so.3 => /usr/lib/libcrypto.so.3 (0x00007f69106c7000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f69104dd000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x00007f6910ceb000)

[lf-]

There are two approaches to fixing this. One is to use the systemtap probes shipped in many builds of glibc. Another is to simply hook dlopen. I think the latter is likely to be more tolerant of different systems.