Segmentation fault while build an ELF

[niooss-ledger]

Describe the bug Building an ELF file with the new ELF builder (https://lief-project.github.io/blog/2022-01-23-new-elf-builder/) triggers a segmentation fault in LIEF.

To Reproduce Steps to reproduce the behavior:

• Install current LIEF master (commit 2ae5327e86f50fe87733d8641d4e7bc3774e3087)
• Run the following Python program to build an ELF file:

import lief

elf = lief.ELF.Binary("test", lief.ELF.ELF_CLASS.CLASS64)
elf.header.identity = bytes.fromhex("7f454c46010101000000000000000000").decode("ascii")
elf.header.file_type = lief.ELF.E_TYPE.DYNAMIC
elf.header.machine_type = lief.ELF.ARCH.x86_64
elf.header.object_file_version = lief.ELF.VERSION.CURRENT

# Add a segment, as described on https://lief-project.github.io/blog/2022-01-23-new-elf-builder/
segment = lief.ELF.Segment()
segment.type = lief.ELF.SEGMENT_TYPES.LOAD
segment.content = [0xcc] * 0x23
elf.add(segment)
elf.write("test.elf")

• Observe a segmentation fault issue (when elf.add(segment) is executed).

Expected behavior I expected the program to successfully build a test.elf file.

Environment (please complete the following information):

• System and Version : docker.io/library/python:3.9-slim container (with Debian 11)
• Target format: ELF
• LIEF commit version: python -c "import lief;print(lief.__version__)" gives 0.12.0-bb7faf3

Additional context I tried to follow instruction from https://lief-project.github.io/blog/2022-01-23-new-elf-builder/ to build an ELF file. Getting a segmentation fault issue instead of a clean Python exception is frustrating.

To debug the issue by myself, I launched gdb and got the following stack trace:

Program received signal SIGSEGV, Segmentation fault.
0x00007ff85241e6a2 in std::vector<unsigned char, std::allocator<unsigned char> >::size() const
    ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so
(gdb) bt
#0  0x00007ff85241e6a2 in std::vector<unsigned char, std::allocator<unsigned char> >::size() const ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so
#1  0x00007ff852e213d1 in LIEF::ELF::DataHandler::Handler::reserve(unsigned long, unsigned long) ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so
#2  0x00007ff852e212fd in LIEF::ELF::DataHandler::Handler::make_hole(unsigned long, unsigned long) ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so
#3  0x00007ff852d7fa0f in LIEF::ELF::Binary::relocate_phdr_table_pie() ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so
#4  0x00007ff852d7f857 in LIEF::ELF::Binary::relocate_phdr_table() ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so
#5  0x00007ff852d70aec in LIEF::ELF::Segment& LIEF::ELF::Binary::add_segment<(LIEF::ELF::E_TYPE)3, false>(LIEF::ELF::Segment const&, unsigned long) ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so
#6  0x00007ff852d77e87 in LIEF::ELF::Binary::add(LIEF::ELF::Segment const&, unsigned long) ()
   from /usr/local/lib/python3.9/site-packages/lief-0.12.0.dev0-py3.9-linux-x86_64.egg/lief.cpython-39-x86_64-linux-gnu.so

To get more information about the functions, I tried to build LIEF with debugging:

python setup.py build_ext --debug -j4
python setup.py install --debug

Unfortunately the second command did not work (install does not know --debug) and using setup.py install without --debug made everything to be recompiled without debug symbols. How is it possible to build LIEF and the Python bindings with debugging? (It would be nice to have such instructions on https://lief-project.github.io/doc/latest/compilation.html ).

Anyway, thanks for your great blog post! It was nice, even though LIEF does not work for my use-case.

[romainthomas]

Hi @niooss-ledger

Currently the new builder does still not enable to create an ELF from scratch. It assumes that the underlying ELF file is already setup with a minimum layout. Nevertheless, the new design will ease the implementation of creating such ELF file (if it's critical feature we can discuss about that).

I updated the documentation for your second point :)

[niooss-ledger]

Hello,

Thanks for your reply. I think there are two issues, which could be discussed separately:

1. A Python program using LIEF can segfault (when calling elf.add(segment) on a crafted elf object). In my humble opinion, it would be more developer-friendly to report an error (for example by raising an exception) instead of segfaulting.
2. LIEF currently does not support building ELF files from scratch. This is a "feature request", as I understand LIEF has been built with the idea of modifying existing ELF files.

For the second issue, it can be helpful that I describe my use-case more precisely. In my work, I am often encountering firmware in exotic formats. For example a few years ago, I encountered the firmware used by HP iLO 4 Baseboard Management Controller. This firmware defined for each process their sections, named in a way similar as in an ELF file (.text, .data, etc.). At the time, I wrote a tool which extracted the memory loaded for each process as an ELF file, crafting the sections by hand (cf. https://github.com/fishilico/ilo4_toolbox/blob/eea2d2e853b6292b927c028b0768eb555bb5f5ab/scripts/iLO4/ioonag_unpacker/unpack_firmware.py#L1363-L1390 for the ELF header, and https://github.com/fishilico/ilo4_toolbox/blob/eea2d2e853b6292b927c028b0768eb555bb5f5ab/scripts/iLO4/ioonag_unpacker/unpack_firmware.py#L1213-L1239 for each entry of program and section headers). It was quite painful to add features such as adding symbols for known functions, when extracting a firmware. So I wanted to use LIEF for it and failed.

Back to the present, I saw your blog post titled "New ELF Builder" and I thought that LIEF now enabled building ELF from scratch. However it requires "a minimum layout" which does not seem to be documented (and the fact that LIEF supports creating PE files from scratch also gave me false hope, https://lief-project.github.io/doc/latest/tutorials/02_pe_from_scratch.html).

What do you think of a feature which would enable building an ELF file from scratch, using defined sections which are for example extracted from a firmware? If I understand correctly, the approach of building a bare ELF and adding the sections one by one is hard to support, because the ELF program header could need to be moved around. Nevertheless if LIEF provided a way to create an ELF directly with given sections (and if it was then possible to define symbols), it would be great :)

[romainthomas]

1. A Python program using LIEF can segfault (when calling elf.add(segment) on a crafted elf object). In my humble opinion, it would be more developer-friendly to report an error (for example by raising an exception) instead of segfaulting.

Yes I completely agree and I removed the ELF Binary constructor as it actually does not construct a consistent object layout.

It was quite painful to add features such as adding symbols for known functions, when extracting a firmware. So I wanted to use LIEF for it and failed.

It was a similar request feature that @wisk had. He worked on it a while ago and his experimentation are here. They are based on the old ELF builder but they are worth reading.

Back to the present, I saw your blog post titled "New ELF Builder" and I thought that LIEF now enabled building ELF from scratch.

Yes the title is a bit confusing but it was more about the performances.

What do you think of a feature which would enable building an ELF file from scratch, using defined sections which are for example extracted from a firmware?

I thought about that for a while and my approach would be to initialize a Binary from a predefined set of section (or segments). It would be something like:

import lief

section1 = lief.ELF.Section(".test1")
...
section2 = lief.ELF.Section(".test2")
crafted = lief.ELF.Binary.create("test", lief.ELF.ELF_CLASS.CLASS64
                                 [section1, section2], symbol_table=True)
if crafted is None:
  print("Crafting failed")
  return

# ... continue the modification

I'll try to make a new PoC to get a better picture of this feature with the new ELF builder :blush:

Currently I'm focus on cleaning the code base of LIEF to prepare the next release

[romainthomas]

FYI, it already exists an issue for this request: #213