Using patchelf `--add-needed` and `--remove-needed` breaks PIE executable

[gamazeps]

I currently need patchelf to add a dependency to an elf file (in the jarify project if it's of any use). It seems to fail, here is a minimal repro.

I am building a simple C file compiled with -fPIE/-fpie and calling the following commands

$patchelf --remove-needed libpthread.so.0 thread
$ldd thread
Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != NULL' failed!
$patchelf --add-needed libpthread.so.0 thread
stat: No such file or director

The Makefile used here is:

CC=gcc
CFLAGS=-fPIE
LDFLAGS=-lpthread -fpie 

OBJECTS=thread.o

hello: $(OBJECTS)
	$(CC) $(CFLAGS) $(OBJECTS) -o thread $(LDFLAGS)

all:hello

.PHONY: clean
clean:
	rm -f *~ *.o thread

The c source file thread.c, (simply the wikipedia example of pthread in order to get a dynamic dependency)

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
 
#define NUM_THREADS     5
 
void* perform_work( void* argument )
{
  int passed_in_value;
 
  passed_in_value = *( ( int* )argument );
  printf( "Hello World! It's me, thread with argument %d!\n", passed_in_value );
 
  /* optionally: insert more useful stuff here */
 
  return NULL;
}
 
int main( int argc, char** argv )
{
  pthread_t threads[ NUM_THREADS ];
  int thread_args[ NUM_THREADS ];
  int result_code;
  unsigned index;
 
  // create all threads one by one
  for( index = 0; index < NUM_THREADS; ++index )
  {
    thread_args[ index ] = index;
    printf("In main: creating thread %d\n", index);
    result_code = pthread_create( &threads[index], NULL, perform_work, &thread_args[index] );
    assert( !result_code );
  }
 
  // wait for each thread to complete
  for( index = 0; index < NUM_THREADS; ++index )
  {
    // block until thread 'index' completes
    result_code = pthread_join( threads[ index ], NULL );
    assert( !result_code );
    printf( "In main: thread %d has completed\n", index );
   }
 
   printf( "In main: All threads completed successfully\n" );
   exit( EXIT_SUCCESS );
}

cc @mboes

[gamazeps]

A colleague on NixOS could not reproduce this.

I am using Ubuntu 16.04 LTS and uname -a gives the following output: Linux terrier 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

I am also using patchelf on current master version, that is commit hash: 327d80443672c397970738f9e216a7e86cbf3ad7

Sorry if the bug report is not very helpful, I will gladly provide any information that you deem necessary.

[dezgeg]

A probable cause is that --remove-needed doesn't update the symbol versioning table. I have some WIP patches that I might get finished at some point...

[knedlsepp]

Some derivations of my custom nixpkgs-overlay broke on the upgrade to 18.03 with a dl-version.c: 224: _dl_check_map_versions: Assertion needed != NULL' failed`. There hasn't been any patchelf version-bump in the meantime, but it's got something to do with the remove-needed. @dezgeg: Do you have this WIP somewhere on github?

[Knightingales]

I've stumbled upon this and it is still happening. Take a simple "Hello, world!" executable and compile it: gcc -pie -fPIE -o hello hello.c and run patchelf --add-needed libsomething.so ./hello Running the ELF post edit results in cannot execute binary file: Exec format error

[domenkozar]

Maybe try with new release? https://nixos.org/releases/patchelf/patchelf-0.10/

[ezyang]

I am able to confirm that --replace-needed did not work with patchelf-0.9 from my distro with this error message, but worked when I used master.

[domenkozar]

This is probably fixed, but needs a regression test.

[airlsyn]

I met the same error while using patchelf-0.11 to remove needed so on ubuntu 18.04

$ patchelf --version
patchelf 0.11

when edit so

$ patchelf --remove-needed libcublas.so.10.0 libonnxruntime.so

the following error would occur

$ ldd libonnxruntime.so
Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion `needed != NULL' failed!

[domenkozar]

@ericxsun could you attach libonnxruntime.so

[airlsyn]

@ericxsun could you attach libonnxruntime.so

@domenkozar pls see the shared file https://drive.google.com/file/d/1rQZYIkzOu4UGJ4Cczm-FVM2LrxYw1Xc0/view?usp=sharing

[airlsyn]

@domenkozar any progress?

[domenkozar]

I can reproduce.

libonnxruntime.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=bec1cd4af318b6fc16319023d88714328d027886, with debug_info, not stripped

Someone could rebase https://github.com/NixOS/patchelf/pull/166 and check if that PR fixes the problem.

[airlsyn]

I can reproduce.

libonnxruntime.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=bec1cd4af318b6fc16319023d88714328d027886, with debug_info, not stripped

Someone could rebase #166 and check if that PR fixes the problem.

I guess it won't be ok, i.e., will not fix this problem.

the problem here, looks like things in replaceNeeded, updating the .gnu.version_r section

[Ericson2314]

@domenkozar pls see the shared file https://drive.google.com/file/d/1rQZYIkzOu4UGJ4Cczm-FVM2LrxYw1Xc0/view?usp=sharing

Unfortunately the link is now bit-rotted, and the file was never committed.

In the future, we should get all test executables in draft PRs at a minimum, so we don't have such bit-rot problems.

(From-source test data might seem better, but the interesting thing is 99% the toolchain you used, not the code you wrote. So unless one can produce that exact toolchain from source (e.g. with an exact Nixpkgs revision), one is best off us just giving us a binary. So it is very good we finally got one. We just failed to commit it in time.)