philipc
Overlap with faerie
This crate has a lot in common with faerie, but it presents a lower level API. Need to investigate how much scope there is for sharing with faerie.
cc @m4b I wrote this because I wanted to basically 'rewrite' the raw ELF sections as part of testing gimli, and faerie's API was too high level for that.
Also of interest, I think I got the alignment right here ;)
Sorry I’ve been meaning to respond more fully but I haven’t had a lot of time lately.
This project looks great ! So couple of random ideas thoughts:
- It does appear to have a lot in common with faerie, that’s fine, the more the merrier
- I haven’t given it a deep inspection unfortunately but it does appear to have a similar generic api like faerie, which means you may or may not end up writing similar higher level abstractions, not sure. Also of course there are multiple ways of doing similar things, maybe you will do something better !
- Id tentatively be interested in adding multiple elf/whatever backends to faerie? E.g. we could experiment with adding multiple elf backends, one native to faerie, another using object write as backend ? Maybe it’s a terrible idea, I dunno?
- On that note, might be a good idea to add the PE backend using this? Or maybe it doesn’t translate so well and all the logic ends up being similar but different from the object write PE backend ? Dunno.
Just some random thoughts. :)
So on that note, one of my original designs for faerie was to easily write object files, either as synthesized objects due to some analysis, or similar to a rust implementation of objcopy, similar to your cool example here!
Originally I had wanted to write a:
dynamic library -> synthesized object file -> static library
Ie, write a dynamic library to static library converter. This would have been very useful to me several times in various capacities. I had a prototype of the analysis using panopticon to extract function references and their transitive closure, along with data references; I then needed to de-relocate the functions. That’s when I started writing faerie since I’d need an object writer.
I’m not sure how safe it would have been; if you mess up even slightly the de-relocated function would behave highly unusually, or crash at runtime, but it sure would have been freaking cool.
Anyway I still think the api was “low level” enough to do all this, but perhaps I’m missing some things ? Would you mind either opening an issue in faerie or outlining really quickly (if you’re busy) some of the troubles you had ?
Anyway just some random thoughts and impressions I typed pretty fast :)
Thanks for the response!
I haven’t given it a deep inspection unfortunately but it does appear to have a similar generic api like faerie, which means you may or may not end up writing similar higher level abstractions, not sure.
It is a generic API on the surface, but it's a bit of a leaky abstraction. The different file formats have different conventions for what types of sections, symbols and relocations you need, and the use of this API needs to be written with that in mind. For something simple like an objcopy though, you really should only be copying between file formats that have the same conventions, and in that case this leaky abstraction doesn't matter.
I expect that I'll add more helper methods (eg append some data to a section), but I don't think the main API will become higher level.
On that note, might be a good idea to add the PE backend using this?
I have a rough version of this here: https://github.com/philipc/faerie/commit/d114cb59596fa8545867d4d23d97512f88a6e458. Making sure that the faerie prototype worked was very useful for testing that I had the goblin COFF support right. I think the main issue with doing this is that it adds a few more dependencies to faerie, and the cranelift people might want to avoid that.
we could experiment with adding multiple elf backends, one native to faerie, another using object write as backend
Not sure about this. What would the benefit to the user be? Having said that, the faerie commit works for using ELF too.
Anyway I still think the api was “low level” enough to do all this, but perhaps I’m missing some things ? Would you mind either opening an issue in faerie or outlining really quickly (if you’re busy) some of the troubles you had ?
The problem was the need to decompose the sections into functions and data objects and the references between them. ie the bit that it appears your panopticon stuff was doing.
What might make sense is for cranelift to switch to using object-write directly, since cranelift-module is kind of duplicating a lot of the higher API that faerie provides.