scala-native icon indicating copy to clipboard operation
scala-native copied to clipboard

posixlib errno & kin should be extern var, not defs

Open LeeTibbert opened this issue 2 years ago • 9 comments

PR #3048 surfaces the fact that Scala Native has been able to use extern vars directly since PR #1348. The latter is now 4+ years old. I believe it is still tested on each CI run.

A few cycles could be saved on each access to posixlib errno & kin (gai_errno?, h_herrno?, ???) if the were changed from the current def to direct access as an extern var.

LeeTibbert avatar Dec 26 '22 17:12 LeeTibbert

Implementation note: By its definition clib (hence, posixlib) MATH_ERRNO should not be changed. It is a constant, not an actual extern storage location to be accessed.

Gotta keep the sheep & the goats separated; they can look similar.

LeeTibbert avatar Dec 26 '22 19:12 LeeTibbert

stdio also has binding function, but they return void pointer. Can/Should they replaced with var? (I don't think so)

void *scalanative_stdin() { return stdin; }

void *scalanative_stdout() { return stdout; }

void *scalanative_stderr() { return stderr; }

i10416 avatar Dec 28 '22 00:12 i10416

extern var in trait is missing🤔

The following code causes NoSuchElementException: key not found: Member(Top(scala.scalanative.libc.errno),C5errno) while linking. @name("errno") annotation won't help. I will explore whether

  1. this is because scala-native doesn't generate extern var in trait properly
  2. ~~this is because @name annotation doesn't support extern var at all~~
  3. ~~this is because @name annotation doesn't support extern var defined in trait~~
// in scala.scalanative.libc and scala.scalanative.posix
@extern object errno extends errno

// in scala.scalanative.libc
@extern private[scalanative] trait errno {
  var errno: CInt = extern
  @name("scalanative_edom")
  def EDOM: CInt = extern
  @name("scalanative_eilseq")
  def EILSEQ: CInt = extern
  @name("scalanative_erange")
  def ERANGE: CInt = extern
}

ref https://github.com/scala-native/scala-native/pull/3057

i10416 avatar Dec 28 '22 10:12 i10416

The ability to use traits and extern is new. See https://github.com/scala-native/scala-native/pull/2988 and https://github.com/scala-native/scala-native/pull/3001

cc/ @WojciechMazur

ekrich avatar Dec 28 '22 16:12 ekrich

Implementation of vars is quite specific. What's the most important we don't define extern var as field, instead we only process its accessors. Probably we generated the accessors in posix.errno, but not for libc.errno. There's also possibly that libc.errno was removed because we assumed it's not used. I'll try to check this out tomorrow

WojciechMazur avatar Dec 28 '22 23:12 WojciechMazur

This is also a case where we really don't need a posix trait - https://github.com/scala-native/scala-native/blob/main/posixlib/src/main/scala/scala/scalanative/posix/errno.scala

The object can extend the trait and in the body can add posix specific code.

That is probably not related but the errno is defined in C lib trait.

ekrich avatar Dec 28 '22 23:12 ekrich

At this point we have 3 problem that are blocking replacing errno with var instead of manually defined accessors:

  • Bug in Scala 3 handling of extern var in extern trait, leading to error when lowering code - fixed in #3059
  • errno is defined as thread_local. Currently we don't have a concept of marking fields in Scala Native with such a behaviour. Example of this problem can be seen in these log lines:
[error] /usr/bin/ld: errno: TLS definition in /lib/x86_64-linux-gnu/libc.so.6 section .tbss mismatches non-TLS reference in /tmp/native-test-out16453956119813239852/native/java/nio/channels.ll.oestOnly 16s
[error] /usr/bin/ld: /lib/x86_64-linux-gnu/libc.so.6: error adding symbols: bad value
  • errno on Windows is defined as macro delegating to _errno field. Becouse of that we would need a glue layer anyway. Becouse of this single issue I think it's best to not change errno signature, and use existing glue layer with manually defined accessors.

WojciechMazur avatar Dec 29 '22 10:12 WojciechMazur

Thank you for investigating this issue!

i10416 avatar Dec 29 '22 12:12 i10416

stdio also has binding function, but they return void pointer. Can/Should they replaced with var? (I don't think so)

Sorry, just catching up. I think that you are right in not changing them at present. There are probably other places, such as javalib subsecond file times, where the payback for invested effort is better.

I suspect that the change is possible and might save a few cycles each time one of stdin & friends is accessed. Depends upon how smart the SN optimizer is (when enabled). The drawback is that because these are so critical, it would take a lot of work so show that the change is safe, over operating systems and complex file usage; such as freopen of stdout to a file.

A bold person who had lots of time on their hands could do a sandbox SN build where stdin, etc. were changed to `extern var'. On Linux dumping the symbols in libc.so.6 gives a global symbol (in glibc, I believe).

{root}# readelf -s libc.so.6 | grep -i stdin
   150: 00000000001f7850     8 OBJECT  GLOBAL DEFAULT   35 stdin@@GLIBC_2.2.5

Now, Windows is most likely to be the difficult case.

LeeTibbert avatar Jan 08 '23 04:01 LeeTibbert