gold plugin is incomplete

[stephenrkell]

Currently, the tools/allocscompilerwrapper.py contains a bunch of stuff that we do when producing a final linked binary. This includes using stubgen.h to create allocator wrappers, linking --wrap to interpose them, unbinding/globalizing the allocator functions if necessary so that interposition can work, creating aliases that we use in some cases (e.g. when linking a malloc() into an executable -- this needs special care), and actually generating the meta-object.

The medium-term goal is to move away from compiler wrappers. Instead, all of liballocs's toolchain interventions should be invokable simply by compiler or linker flags. One of these flags should enable a linker plugin, to do whatever we need to do at link time. That includes, at least, generating the meta-object.

A basic version of this overall approach, of relying on compiler flags not wrappers is already proof-of-concepted in libcrunch's cilpp front-end. Some helper scripts provide compiler options, and these include specifying a "wrapper" program on the gcc command line, which is used to replace the vanilla C preprocessing step with a cpp-then-CIL source-to-source pass. https://github.com/stephenrkell/libcrunch/blob/master/frontend/cilpp/bin/cilpp-cflags

We already also have a gold plugin that works for some simple cases. This needs to be finished. But we should not do this before tackling #11, because switching to binary instrumentation at run time will simplify the task considerably, by eliminating a lot of the link-time interventions.

[stephenrkell]

Note also that I now have a "base" repository, toolsub, for re-usable elements of toolchain subversion (https://github.com/stephenrkell/toolsub). A basic linker plugin probably belongs in there, with the liballocs-specific one built as an extension.

[stephenrkell]

I am increasingly keen to get this done, maybe this summer. At the same time, we should clean up the overall toolchain, including nastinesses such as usedtypes modifying each .o file after it is generated. I notice there's a related problem remarked in #18: we want to avoid linking uniqtypes into the executable, so that they are always overridable. But is this feasible? We want to preserve the 'run without library' property, so we will need undefined symbols in the executable. Weak symbols don't work during dynamic linking, so we can't make them weak. So how will they be satisfied?

Need to do some experiments with dynamic linking to find out what can work. I think executables with UND symbols actually do work, at least if they're PIEs. Need to check that we can define the symbols, ideally in the dlbind object or just maybe in the ld.so itself (it's already the case that we're the ld.so now, though only chain-loading).

If that doesn't work, it's possible that #18's aliasing problem can be resolved another way. It's tough though: imagine dlopening a DSO that defines a new alias for 'int'. It really needs to alias the existing 'int'.

[stephenrkell]

Revisiting this, I can see at least two possible ways to eliminate the need for a stubs library.

• link with -z undefs so that unresolved syms are fine (loses the check, though)
• link with -z dynamic-undefined-weak which seems to be exactly what we want (new?)

To avoid losing the check for undefineds, we can perhaps do it using the same pre-final all-objs relocatable link step that we do in allocscompilerwrapper (added to support the malloc-in-exe case, although I forget how). But on second thoughts, that doesn't help with defs in libraries, which are deliberately omitted from that stage of linking.

And as noted above, it's probably possible to build a PIE that has undefined symbols (though a quick attempt just now fails, unless adding -z undefs).