ADIOS2 icon indicating copy to clipboard operation
ADIOS2 copied to clipboard

macOS: dyld RPath Library not loaded in 2.6.0

Open ax3l opened this issue 5 years ago • 14 comments

Describe the bug I am currently transitioning to ADIOS 2.6.0 in our CI and encounter problems at runtime with the new split libraries on macOS 10.13 and 10.14 of the form:

dyld: Library not loaded: @rpath/libadios2_c.2.dylib
  Referenced from: /Users/travis/build/bin/MyBinary
  Reason: image not found

To Reproduce I use a spack-installed ADIOS 2.6.0 and built against it with CMake. I run into issues as seen in: https://travis-ci.com/github/openPMD/openPMD-api/builds/170615793

We use the native C++11 API of ADIOS2. Other dependencies in our downstream CMakeLists.txt might activate the C language in CMake, but we do not use the ADIOS2 C bindings directly.

Expected behavior Did build & run with ADIOS 2.5.0 :)

Desktop (please complete the following information):

  • OS/Platform:
    • macOS 10.13 with AppleClang 10.0 & libc++ and
    • macOS 10.14 with AppleClang 11.0
  • Build: cmake 3.12.0 and shared builds

Additional context

  • CI PR: https://github.com/openPMD/openPMD-api/pull/754
  • Travis CI build output: https://travis-ci.com/github/openPMD/openPMD-api/builds/170615793

ax3l avatar Jun 10 '20 03:06 ax3l

I switched now to the new adios2::cxx11 and adios2::cxx11_mpi targets and get:

dyld: Library not loaded: @rpath/libadios2_cxx11.2.dylib
  Referenced from: /Users/travis/build/bin/MyBinary
  Reason: image not found

ax3l avatar Jun 10 '20 07:06 ax3l

@chuckatkins @bradking do you have an idea what's off on macOS? This one is a blocker for us to migrate to ADIOS 2.6.0 (and with it finalizing complex support for our users).

ax3l avatar Jun 12 '20 06:06 ax3l

We need to figure out where that @rpath/ reference is coming from.

I use a spack-installed ADIOS 2.6.0

I just ran spack install [email protected] on macOS and found that LC_ID_DYLIB in opt/spack/darwin-catalina-x86_64/clang-11.0.0-apple/adios2-2.6.0-ioxjevk4au455pt4rop2smac3kpcbgv5/lib/libadios2_cxx11.2.6.0.dylib is an absolute path and does not mention @rpath anywhere. Is that the case for yours too? You can find that file and run otool -l $file to see.

bradking avatar Jun 15 '20 12:06 bradking

Thanks, my libadios2_cxx11.2.dylib looks like this. CI: https://travis-ci.com/github/openPMD/openPMD-api/jobs/349960561

/Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.6.0-62caxyc6a74de7kgb7e2l4w3obuhddtd/lib/libadios2_cxx11.2.dylib:
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777223          3  0x00           6    26       3200 0x00118085
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __TEXT
   vmaddr 0x0000000000000000
   vmsize 0x00000000000a2000
  fileoff 0
 filesize 663552
  maxprot 0x00000005
 initprot 0x00000005
   nsects 7
    flags 0x0
Section
  sectname __text
   segname __TEXT
      addr 0x0000000000002240
      size 0x000000000008f36f
    offset 8768
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __stubs
   segname __TEXT
      addr 0x00000000000915b0
      size 0x00000000000012a2
    offset 595376
     align 2^1 (2)
    reloff 0
    nreloc 0
     flags 0x80000408
 reserved1 0 (index into indirect symbol table)
 reserved2 6 (size of stubs)
Section
  sectname __stub_helper
   segname __TEXT
      addr 0x0000000000092854
      size 0x00000000000009ac
    offset 600148
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __gcc_except_tab
   segname __TEXT
      addr 0x0000000000093200
      size 0x0000000000009150
    offset 602624
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __TEXT
      addr 0x000000000009c350
      size 0x0000000000000614
    offset 639824
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __cstring
   segname __TEXT
      addr 0x000000000009c964
      size 0x00000000000014b1
    offset 641380
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x00000002
 reserved1 0
 reserved2 0
Section
  sectname __unwind_info
   segname __TEXT
      addr 0x000000000009de18
      size 0x00000000000041e4
    offset 646680
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __DATA
   vmaddr 0x00000000000a2000
   vmsize 0x0000000000003000
  fileoff 663552
 filesize 8192
  maxprot 0x00000003
 initprot 0x00000003
   nsects 7
    flags 0x0
Section
  sectname __nl_symbol_ptr
   segname __DATA
      addr 0x00000000000a2000
      size 0x0000000000000008
    offset 663552
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 795 (index into indirect symbol table)
 reserved2 0
Section
  sectname __got
   segname __DATA
      addr 0x00000000000a2008
      size 0x00000000000000d8
    offset 663560
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 796 (index into indirect symbol table)
 reserved2 0
Section
  sectname __la_symbol_ptr
   segname __DATA
      addr 0x00000000000a20e0
      size 0x00000000000018d8
    offset 663776
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000007
 reserved1 823 (index into indirect symbol table)
 reserved2 0
Section
  sectname __mod_init_func
   segname __DATA
      addr 0x00000000000a39b8
      size 0x0000000000000050
    offset 670136
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000009
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __DATA
      addr 0x00000000000a3a08
      size 0x00000000000004f8
    offset 670216
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __data
   segname __DATA
      addr 0x00000000000a3f00
      size 0x0000000000000010
    offset 671488
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __bss
   segname __DATA
      addr 0x00000000000a3f10
      size 0x00000000000001d8
    offset 0
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
   vmaddr 0x00000000000a5000
   vmsize 0x0000000000095000
  fileoff 671744
 filesize 606656
  maxprot 0x00000001
 initprot 0x00000001
   nsects 0
    flags 0x0
Load command 3
          cmd LC_ID_DYLIB
      cmdsize 56
         name @rpath/libadios2_cxx11.2.dylib (offset 24)
   time stamp 1 Thu Jan  1 00:00:01 1970
      current version 2.6.0
compatibility version 2.0.0
Load command 4
            cmd LC_DYLD_INFO_ONLY
        cmdsize 48
     rebase_off 671744
    rebase_size 104
       bind_off 671848
      bind_size 67448
  weak_bind_off 739296
 weak_bind_size 68720
  lazy_bind_off 808016
 lazy_bind_size 17088
     export_off 825104
    export_size 83096
Load command 5
     cmd LC_SYMTAB
 cmdsize 24
  symoff 912368
   nsyms 4602
  stroff 992472
 strsize 285928
Load command 6
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 1804
     iextdefsym 1804
     nextdefsym 2001
      iundefsym 3805
      nundefsym 797
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 986000
  nindirectsyms 1618
      extreloff 0
        nextrel 0
      locreloff 0
        nlocrel 0
Load command 7
     cmd LC_UUID
 cmdsize 24
    uuid 96AEDB87-7C94-3182-B541-C6BB463DD9D2
Load command 8
       cmd LC_BUILD_VERSION
   cmdsize 32
  platform 1
       sdk 10.15
     minos 10.14
    ntools 1
      tool 3
   version 520.0
Load command 9
      cmd LC_SOURCE_VERSION
  cmdsize 16
  version 0.0
Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libadios2_core.2.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 2.6.0
compatibility version 2.0.0
Load command 11
          cmd LC_LOAD_DYLIB
      cmdsize 48
         name /usr/lib/libc++.1.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 800.7.0
compatibility version 1.0.0
Load command 12
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 1281.0.0
compatibility version 1.0.0
Load command 13
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0adios2-2.6.0-62caxyc6a74de7kgb7e2l4w3obuhddtd/lib (offset 12)
Load command 14
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0adios2-2.6.0-62caxyc6a74de7kgb7e2l4w3obuhddtd/lib64 (offset 12)
Load command 15
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/zstd-.4.5-7tsdvwjlqy33gyhfnnndtutehi6xxg2a/lib (offset 12)
Load command 16
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/libfabric-1.9.1-76anhiidtsbqlmfausybg7m3lqheyrq/lib (offset 12)
Load command 17
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/bzip2-1.0.8-tdq3ofob2xue6gkhp6pjphj7wldt4yl/lib (offset 12)
Load command 18
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/zlib-1.2.11-6m74cvmbqpza7ljfyle3ccmilsxyebe/lib (offset 12)
Load command 19
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/c-blosc-1.17.0-qqoaza5pizhtc7iojvxnzmj6rhg5vpp/lib (offset 12)
Load command 20
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/snappy-1.1.7-hduzhnbapqnnskr6hgfpe45t7k7sv2j/lib (offset 12)
Load command 21
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/lz4-1.9.2-wcj7r4347j3bfksp7nkyzj5zanszgpw/lib (offset 12)
Load command 22
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/libffi-3.3-l3xx7qvdynex6icppsprszz5z3jwjc4/lib (offset 12)
Load command 23
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 908200
 datasize 3936
Load command 24
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 912136
 datasize 232
Load command 25
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/../lib (offset 12)

ax3l avatar Jun 16 '20 19:06 ax3l

@ax3l thanks. Your libadios2_cxx11.2.dylib indeed has LC_ID_DYLIB set to @rpath/libadios2_cxx11.2.dylib. That is different than on my machine's spack installation. Does spack have any options for that?

What did the adios2 .dylib files in ADIOS2 2.5 have for you?

bradking avatar Jun 17 '20 12:06 bradking

Thanks @bradking. In ADIOS2 2.5.0 the symbols looks like this: https://travis-ci.com/github/openPMD/openPMD-api/builds/172164002

/Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.5.0-sf6eyfn6xjlwvbvgs7dajgdpvs6ihrnb/lib/libadios2.2.dylib:
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777223          3  0x00           6    28       3864 0x00118085
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __TEXT
   vmaddr 0x0000000000000000
   vmsize 0x0000000000488000
  fileoff 0
 filesize 4751360
  maxprot 0x00000005
 initprot 0x00000005
   nsects 7
    flags 0x0
Section
  sectname __text
   segname __TEXT
      addr 0x0000000000003380
      size 0x0000000000430750
    offset 13184
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __stubs
   segname __TEXT
      addr 0x0000000000433ad0
      size 0x000000000000200a
    offset 4405968
     align 2^1 (2)
    reloff 0
    nreloc 0
     flags 0x80000408
 reserved1 0 (index into indirect symbol table)
 reserved2 6 (size of stubs)
Section
  sectname __stub_helper
   segname __TEXT
      addr 0x0000000000435adc
      size 0x0000000000000baa
    offset 4414172
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __TEXT
      addr 0x0000000000436690
      size 0x00000000000022f0
    offset 4417168
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __cstring
   segname __TEXT
      addr 0x0000000000438980
      size 0x000000000000b305
    offset 4426112
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x00000002
 reserved1 0
 reserved2 0
Section
  sectname __gcc_except_tab
   segname __TEXT
      addr 0x0000000000443c88
      size 0x0000000000035100
    offset 4471944
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __unwind_info
   segname __TEXT
      addr 0x0000000000478d88
      size 0x000000000000f278
    offset 4689288
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 712
  segname __DATA
   vmaddr 0x0000000000488000
   vmsize 0x000000000000d000
  fileoff 4751360
 filesize 49152
  maxprot 0x00000003
 initprot 0x00000003
   nsects 8
    flags 0x0
Section
  sectname __nl_symbol_ptr
   segname __DATA
      addr 0x0000000000488000
      size 0x0000000000000008
    offset 4751360
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 1367 (index into indirect symbol table)
 reserved2 0
Section
  sectname __got
   segname __DATA
      addr 0x0000000000488008
      size 0x00000000000005c0
    offset 4751368
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 1368 (index into indirect symbol table)
 reserved2 0
Section
  sectname __la_symbol_ptr
   segname __DATA
      addr 0x00000000004885c8
      size 0x0000000000002ab8
    offset 4752840
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000007
 reserved1 1552 (index into indirect symbol table)
 reserved2 0
Section
  sectname __mod_init_func
   segname __DATA
      addr 0x000000000048b080
      size 0x00000000000002b0
    offset 4763776
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000009
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __DATA
      addr 0x000000000048b330
      size 0x0000000000007c60
    offset 4764464
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __data
   segname __DATA
      addr 0x0000000000492f90
      size 0x00000000000000d0
    offset 4796304
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __bss
   segname __DATA
      addr 0x0000000000493060
      size 0x0000000000001048
    offset 0
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Section
  sectname __common
   segname __DATA
      addr 0x00000000004940a8
      size 0x0000000000000068
    offset 0
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
   vmaddr 0x0000000000495000
   vmsize 0x0000000000194000
  fileoff 4800512
 filesize 1651412
  maxprot 0x00000001
 initprot 0x00000001
   nsects 0
    flags 0x0
Load command 3
          cmd LC_ID_DYLIB
      cmdsize 168
         name /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.5.0-sf6eyfn6xjlwvbvgs7dajgdpvs6ihrnb/lib/libadios2.2.dylib (offset 24)
   time stamp 1 Thu Jan  1 00:00:01 1970
      current version 2.5.0
compatibility version 2.0.0
Load command 4
            cmd LC_DYLD_INFO_ONLY
        cmdsize 48
     rebase_off 4800512
    rebase_size 752
       bind_off 4801264
      bind_size 3776
  weak_bind_off 4805040
 weak_bind_size 143816
  lazy_bind_off 4948856
 lazy_bind_size 12112
     export_off 4960968
    export_size 310616
Load command 5
     cmd LC_SYMTAB
 cmdsize 24
  symoff 5286544
   nsyms 14286
  stroff 5526796
 strsize 925128
Load command 6
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 8090
     iextdefsym 8090
     nextdefsym 5832
      iundefsym 13922
      nundefsym 364
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 5515120
  nindirectsyms 2919
      extreloff 0
        nextrel 0
      locreloff 0
        nlocrel 0
Load command 7
     cmd LC_UUID
 cmdsize 24
    uuid F533A1D5-FC81-3DBC-8760-11843D80C04B
Load command 8
       cmd LC_BUILD_VERSION
   cmdsize 32
  platform 1
       sdk 10.15
     minos 10.14
    ntools 1
      tool 3
   version 520.0
Load command 9
      cmd LC_SOURCE_VERSION
  cmdsize 16
  version 0.0
Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 168
         name /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.5.0-sf6eyfn6xjlwvbvgs7dajgdpvs6ihrnb/lib/libtaustubs.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 0.0.0
compatibility version 0.0.0
Load command 11
          cmd LC_LOAD_DYLIB
      cmdsize 168
         name /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.5.0-sf6eyfn6xjlwvbvgs7dajgdpvs6ihrnb/lib/libadios2_sst.2.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 2.5.0
compatibility version 2.0.0
Load command 12
          cmd LC_LOAD_DYLIB
      cmdsize 176
         name /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/c-blosc-1.17.0-zqqoaza5pizhtc7iojvxnzmj6rhg5vpp/lib/libblosc.1.17.0.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 1.17.0
compatibility version 1.0.0
Load command 13
          cmd LC_LOAD_DYLIB
      cmdsize 48
         name @rpath/libbz2.1.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 1.0.0
compatibility version 1.0.8
Load command 14
          cmd LC_LOAD_DYLIB
      cmdsize 48
         name /usr/lib/libc++.1.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 800.7.0
compatibility version 1.0.0
Load command 15
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Thu Jan  1 00:00:02 1970
      current version 1281.0.0
compatibility version 1.0.0
Load command 16
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.5.0-sf6eyfn6xjlwvbvgs7dajgdpvs6ihrnb/lib (offset 12)
Load command 17
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.5.0-sf6eyfn6xjlwvbvgs7dajgdpvs6ihrnb/lib64 (offset 12)
Load command 18
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/c-blosc-1.17.0-zqqoaza5pizhtc7iojvxnzmj6rhg5vpp/lib (offset 12)
Load command 19
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/libfabric-1.9.1-h76anhiidtsbqlmfausybg7m3lqheyrq/lib (offset 12)
Load command 20
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/snappy-1.1.7-ehduzhnbapqnnskr6hgfpe45t7k7sv2j/lib (offset 12)
Load command 21
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/libffi-3.3-znztsxzjr4rrkb6llnsp3wpwbvxzjn7a/lib (offset 12)
Load command 22
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/zlib-1.2.11-c6m74cvmbqpza7ljfyle3ccmilsxyebe/lib (offset 12)
Load command 23
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/lz4-1.9.2-ywcj7r4347j3bfksp7nkyzj5zanszgpw/lib (offset 12)
Load command 24
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/zstd-1.4.5-7tsdvwjlqy33gyhfnnndtutehi6xxg2a/lib (offset 12)
Load command 25
          cmd LC_RPATH
      cmdsize 136
         path /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/bzip2-1.0.8-stdq3ofob2xue6gkhp6pjphj7wldt4yl/lib (offset 12)
Load command 26
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 5271584
 datasize 14016
Load command 27
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 5285600
 datasize 944

both use the same (latest) version of Spack to install.

ax3l avatar Jun 19 '20 06:06 ax3l

@ax3l thanks. That ADIOS 2.5 library has LC_ID_DYLIB set to /Users/travis/.cache/spack/opt/spack/darwin-mojave-x86_64/clang-11.0.0/adios2-2.5.0-sf6eyfn6xjlwvbvgs7dajgdpvs6ihrnb/lib/libadios2.2.dylib which is expected and does not mention @rpath/.

It also matches what I see on my installations for ADIOS 2.5 and 2.6, which I installed on macOS in a brand new spack tree using spack install [email protected] and spack install [email protected]. So the question remains: why does your spack-installed ADIOS2 2.6 use @rpath while mine does not? That is a question for a spack expert.

bradking avatar Jun 19 '20 10:06 bradking

Intriguing, thank you for testing this. Just to collect all data: which version of CMake did you use? With CMake 3.12.0 I can reproduce this on macOS 10.13 (AppleClang 10.0) and 10.14 (AppleClang 11.0) on Travis-CI and set the following Spack configs: https://github.com/openPMD/openPMD-api/tree/3ff42bedbef2ab00457d26f5fd5f9d23dc7dcfe8/.github/ci/spack . None of them should be related to rpaths but I build ADIOS2 with the variant modifcations ~zfp ~sz ~png ~dataman ~python ~fortran ~ssc.

This is the Spack CMake-related RPath info I can find in the docs:

  • https://spack.readthedocs.io/en/latest/workflows.html#write-the-cmake-build
  • https://spack.readthedocs.io/en/latest/search.html?q=rpath&check_keywords=yes&area=default

Also inquired upstream.

There is the logic in build_environment.py to create SPACK_COMPILER_IMPLICIT_RPATHS, SPACK_COMPILER_EXTRA_RPATHS, SPACK_RPATH_DIRS, and SPACK_RPATH_DEPS. If you follow those you should see how they were constructed. If the library is pointing to itself, I would guess that is coming from SPACK_RPATH_DIRS or SPACK_RPATH_DEPS (someone forgot to exclude the root from a DAG traversal) or it could be the package itself.

ax3l avatar Jun 25 '20 07:06 ax3l

I ran those spack install commands on macOS 10.15 with AppleClang 11.0. CMake 3.17.0 was in my PATH.

bradking avatar Jun 26 '20 10:06 bradking

Still had no time to debug this inside Spack, but will work-around this for now by building a adios2 ~shared ADIOS2 in our CI to keep going.

Now get an interesting Library not loaded: @rpath/libbz2.1.dylib when running an executable that links against the dynamic libopenPMD, which in turn was built against a static adios2. Will disable that as well via adios2 ~bzip2.

ax3l avatar Jul 30 '20 04:07 ax3l

Any updates on this? Is the Mac python bindings not working/usable at the moment because of this? I haven't been able to install/use them.

manauref avatar Nov 02 '21 20:11 manauref

@manauref in case you don't need the latest development, there is the adios2 brew package that @ax3l put together. Hope it helps.

williamfgc avatar Nov 02 '21 21:11 williamfgc