Hook function inside shared library opened via `dlopen()` with `RTLD_DEEPBIND`?

[e4lam]

Hi, I'm looking for a way to hook malloc() inside a library that's loaded via dlopen() with RTLD_DEEPBIND. Is this possible with plthook? Thanks!

[kubo]

I'm not sure about RTLD_DEEPBIND but I guess that it is possible.

How about the following code? Well, I have not checked whether it works or not.

static void *hooked_malloc(size_t size)
{
    ....
    void *addr = malloc(size);
    ....
    return addr;
}

static void *hooked_dlopen(const char *filename, int flags)
{
    void *handle = dlopen(filename, flags);
    if (handle != NULL) {
        plthook_t *plthook;
        // Add error checking in your code.
        plthook_open_by_handle(&plthook, handle);
        plthook_replace(plthook, "malloc", (void *)hooked_malloc), NULL);
        plthook_close(plthook);
    }
}

void install_dlopen_hook()
{
    plthook_t *plthook;
    // Add error checking in your code.
    plthook_open(&plthook, "....");  // Otherwise, plthook_open_by_address or plthook_open_by_handle
    plthook_replace(plthook, "dlopen", (void *)hooked_dlopen), NULL);
    plthook_close(plthook);
}

[e4lam]

Sorry, I've been pulled away and haven't found time to really explore this again. So this should theoretically work with RTLD_DEEPBIND because we're injecting the hook after dlopen() has loaded all shared library and its dependencies? What if during the dlopen() itself, it makes dlopen(filename, RTLD_DEEPBIND) calls? Does that case still work? Thank you!