[Do not merge] CI test, cmake option to enable address santitizer.

[bernhardu]

This are the patches of my recent address santizer work.

With these all tests pass except two. But at least some take much longer, the whole run around 24 minutes instead of 9.

The tests nested_detach-no-syscallbuf and nested_detach-32-no-syscallbuf do not finish even with 980 seconds timeout, and given the tests nested_detach and nested_detach-32 finish also with asan in a few seconds, I guess they show a hanging test.

What do you think? Is there something already suitable for merging?

[bernhardu]

From #2883: We would also need to be able to run rr with the ASAN library dynamically linked, but NOT pass that library name through to rr's spawned children.

Here "dynamically linked" means this?

$ ldd bin/rr
        libasan.so.6 => /usr/lib/x86_64-linux-gnu/libasan.so.6 (0x00007f730ea2e000)
...

Is this "NOT pass that library name" just related to LD_PRELOAD? Could you please give some more details?

[rocallahan]

Is this "NOT pass that library name" just related to LD_PRELOAD? Could you please give some more details?

Ah, I was thinking that ASAN uses LD_PRELOAD, but I guess it doesn't the way you're building it.

[bernhardu]

Ah, I was thinking that ASAN uses LD_PRELOAD, but I guess it doesn't the way you're building it.

That would be at least needed when inspecting a library built with ASAN with an executable built without. But yes, while working at the strip_outer_ld_preload change, I got the impression that ASAN prepends itself to LD_PRELOAD when spawning childs to make sure it is the first library. But I could not trace any negative consequence back to it, when running the tests.

[Bob131]

Dang, you beat me to it! I've also been working on this, but it seems I spent too much time messing with hacks ;)

I have some comments (and maybe some patches) that might help.

[bernhardu]

This doesn't really solve the issue, and it actually masks the fact that support for running ASan binaries in rr is currently broken (I'll open a PR to fix that in a bit). This is actually what I've been stuck on for a few days.

I'm thinking maybe the right approach is to detect ASan-using binaries before they exec() and fix-up LD_PRELOAD there.

EDIT: Woops, this was regarding 23fafc1 (" Run child process with ASAN_OPTIONS=verify_asan_link_order=0. ")

Because I was specifically interested in running just the tests against an asan enabled rr (where I intentionally did not build the tests with asan), I took this "shortcut". This also gets just activated when building rr with asan. Running a asan enabled binary with a regular rr should still show the issue.

[bernhardu]

Rebased to current head, because a few cheap patches got merged separately. Remaining patches are unchanged.

[bernhardu]

@Bob131 I hope it is ok to integrate your patches to this draft?

[bernhardu]

Just rebased the patches from 2021 to current git tip. Half of them are still just for better control of running the tests.

[Keno]

Can we split this up into a few separate PRs?

• Just the CMake option to turn on ASAN?
• The sparse copy stuff
• The preload changes

and then if we still can't figure out the test failures, the extra changes to disable tests, but I'd prefer to just go through and debug why it's not working. I think it'll make it easier to review and faster to merge in isolated pieces.

[bernhardu]

Thanks for looking at it. I removed the test infrastructure changes as they seem at least nowadays needless for me, with Zen support in place and I guess other performance improvements.

With this I got now all tests succeed.

As a start I split off the CMake option to turn on ASAN into another clean PR.