./a.out: symbol lookup error: c.so: undefined symbol: __start___verbose

[marxin]

Also taken from binutils test suite:

a.c:

extern int __start___verbose[];
extern int __stop___verbose[];
int
foo1 (void)
{
  static int my_var __attribute__((used, section("__verbose"))) = 5;
  if (& __start___verbose[0] == & __stop___verbose[0]
      || __start___verbose[0] != 5)
    return -1;
  else
    return 0;
}

b.c:

extern int __start___verbose[];
extern int __stop___verbose[];
int
foo2 (void)
{
  static int my_var __attribute__((used, section("__verbose"))) = 10;
  if (& __start___verbose[0] == & __stop___verbose[0]
      || __start___verbose[0] != 10)
    return -1;
  else
    return 0;
}

c.c:

cat c.c
extern int __start___verbose[];
extern int __stop___verbose[];
int
foo3 (void)
{
  if (& __start___verbose[0] == & __stop___verbose[0]
      || __start___verbose[0] != 6)
    return -1;
  else
    return 0;
}

d.c:

$ #include <stdio.h>

extern int foo1 (void);
extern int foo2 (void);

static int my_var __attribute__((used, section("__verbose"))) = 6;

int
main ()
{
  if (foo1 () == 0
      && foo2 () == 0
      && foo3 () == 0)
    printf ("PASS\n");
  return 0;
}

$ gcc a.c -shared -o a.so -fPIC
$ gcc b.c -shared -o b.so -fPIC
$ gcc c.c -shared -o c.so -fPIC
$ gcc d.c [abc].so && ./a.out
./a.out: symbol lookup error: c.so: undefined symbol: __start___verbose

While ld.bfd is fine:

$ gcc-12 d.c [abc].so && ./a.out
PASS

[marxin]

Simplified test-case:

lib.c:

extern int __start___verbose[];
extern int __stop___verbose[];

// int my_var2 __attribute__((section("__verbose"))) = 6;

void
foo (void)
{
  if (& __start___verbose[0] == & __stop___verbose[0]
      || __start___verbose[0] != 6)
    __builtin_abort();
}

main.c

cat main.c
extern void foo (void);

static int my_var __attribute__((used, section("__verbose"))) = 6;

int
main ()
{
  foo();
  return 0;
}

$ gcc-12 -B ~/Programming/mold/objdir lib.c -shared -o libtest.so -fPIC && gcc main.c libtest.so && ./a.out
./a.out: symbol lookup error: libtest.so: undefined symbol: __start___verbose

The problem is quite obvious as lib.c does not define any symbol in __verbose section and thus the synthetic symbols __start___verbose and __stop___verbose are not created by mold. If I uncomment the comment in the file, all is fine:

$ nm libtest.so | grep __verb
0000000000003940 d __start___verbose
0000000000003944 d __stop___verbose

So it's quite a weird test-case, I must admit that.

[marxin]

The difference is that BFD puts __start___verbose to .dynsym:

$ gcc-12 main.c libtest.so && readelf -sW a.out | less
...
Symbol table '.dynsym' contains 8 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
...
     6: 000000000040401c     0 NOTYPE  GLOBAL PROTECTED   26 __stop___verbose
     7: 0000000000404018     0 NOTYPE  GLOBAL PROTECTED   26 __start___verbose

and that's why it can be resolved by dynamic linker.

[rui314]

Ah, __start/__stop symbols are by default global in BFD. Here is a quote from GNU ld's man page.

           start-stop-visibility=value
               Specify the ELF symbol visibility for synthesized
               "__start_SECNAME" and "__stop_SECNAME" symbols.  value must be
               exactly default, internal, hidden, or protected.  If no -z
               start-stop-visibility option is given, protected is used for
               compatibility with historical practice.  However, it's highly
               recommended to use -z start-stop-visibility=hidden in new
               programs and shared libraries so that these symbols are not
               exported between shared objects, which is not usually what's
               intended.

So it sounds like it was a misfeature to export the symbols by default. Our __start/__stop symbols are hidden by default.

[marxin]

Yeah, I would not implement start-stop-visibility=value option now as it's probably only the binutils test-case that is affected. What do you think about usability of such an option?

[rui314]

We may want to ignore -z start-stop-visibility=hidden for the sake of compatibility with programs that follow the recommendation in this man page. Other than that, I don't think there's a practical use of this option.

[marxin]

We may want to ignore -z start-stop-visibility=hidden for the sake of compatibility with programs that follow the recommendation in this man page.

Yep, I would do that. Thanks.

[bullno1]

We may want to ignore -z start-stop-visibility=hidden for the sake of compatibility with programs that follow the recommendation in this man page. Other than that, I don't think there's a practical use of this option.

My project make use of this option. It is a live reload library. It aims to also provide state persistence across reloads.

The user can mark persistent variables with a macro: https://github.com/bullno1/remodule/blob/master/remodule.h#L79 The host will scan through the list of marked variables and save values to a heap-allocated buffer. After reload, the variable values will be copied back into the new locations.

I implement it using the section feature. When the __start/__stop symbols are invisible, the dynamic linker just crashes on dlopen.

Granted, it is pretty niche.

[rui314]

I don't want to change the mold's default from -z start-stop-visibility=hidden to -z start-stop-visibility=protected, but I'm open to support -z start-stop-visibility=protected.

[bullno1]

Thanks. My knowledge of linker is limited but is it a matter of:

1. Add the visibility to context: https://github.com/rui314/mold/blob/d432e987a019ba213a21cfed89b01ba9041e1a2c/elf/mold.h#L1627
2. Parse cmd args and save the visibility instead of rejecting: https://github.com/rui314/mold/blob/22c9ec8cb6bf32ff67da5b68c3c9cc23cc76ec63/elf/cmdline.cc#L805-L806
3. Based on context, change the visibility: https://github.com/rui314/mold/blob/d432e987a019ba213a21cfed89b01ba9041e1a2c/elf/passes.cc#L835

I'll try to make a PR.

[rui314]

Actually I've already started working on it.