cats-effect icon indicating copy to clipboard operation
cats-effect copied to clipboard

3.7.0-RC1 broken with SN 0.5.9

Open majk-p opened this issue 2 months ago • 11 comments

I wanted to try cats-effect 3.7.0-RC1 against the latest release of Scala Native 0.5.9. Unfortunately producing binary fails with a following error:

Compiling project (Scala 3.7.2, Scala Native 0.5.9)
Compiled project (Scala 3.7.2, Scala Native 0.5.9)
[info] Linking (multithreadingEnabled=true, disable if not used) (9700 ms)
[info] Discovered 2666 classes and 17685 methods after classloading
[info] Checking intermediate code (quick) (497 ms)
[info] Discovered 2638 classes and 13835 methods after optimization
[info] Optimizing (debug mode) (20444 ms)
[info] Produced 13 LLVM IR files
[info] Generating intermediate code (14405 ms)
[error] /tmp/3168152424178068910/.scala-build/3168152424178068910_16c8be8ad5-2b2bc60792/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c:9:22: error: variable has incomplete type 'struct sigaction'
[error]     9 |     struct sigaction action;
[error]       |                      ^
[error] /tmp/3168152424178068910/.scala-build/3168152424178068910_16c8be8ad5-2b2bc60792/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c:9:12: note: forward declaration of 'struct sigaction'
[error]     9 |     struct sigaction action;
[error]       |            ^
[error] /tmp/3168152424178068910/.scala-build/3168152424178068910_16c8be8ad5-2b2bc60792/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c:12:13: error: call to undeclared function 'sigemptyset'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
[error]    12 |     error = sigemptyset(&action.sa_mask);
[error]       |             ^
[error] /tmp/3168152424178068910/.scala-build/3168152424178068910_16c8be8ad5-2b2bc60792/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c:16:13: error: call to undeclared function 'sigaddset'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
[error]    16 |     error = sigaddset(&action.sa_mask, 13); // mask SIGPIPE
[error]       |             ^
[error] /tmp/3168152424178068910/.scala-build/3168152424178068910_16c8be8ad5-2b2bc60792/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c:20:12: error: call to undeclared function 'sigaction'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
[error]    20 |     return sigaction(signum, &action, NULL);
[error]       |            ^
[error] 4 errors generated.
[info] Compiling to native code (10068 ms)
[info] Total (52793 ms)
Exception in thread "main" scala.scalanative.build.BuildException: Failed to compile /tmp/3168152424178068910/.scala-build/3168152424178068910_16c8be8ad5-2b2bc60792/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c
        at scala.scalanative.build.LLVM$.$anonfun$compileFile$1(LLVM.scala:123)
        at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
        at scala.util.Success.$anonfun$map$1(Try.scala:255)
        at scala.util.Success.map(Try.scala:213)
        at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
        at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Error: scala.build.errors.ScalaNativeBuildError: Error compiling with Scala Native

Here's a minimal gist to reproduce it https://gist.github.com/majk-p/af00a196ab8c193994992bf8ad7d2c89

To reproduce run: scala run https://gist.github.com/majk-p/af00a196ab8c193994992bf8ad7d2c89

majk-p avatar Oct 16 '25 17:10 majk-p

Sorry you experienced this problem & thank you for reporting it.

struct sigaction action; looks like C or C++ code.

I did a quick search through the Scala Native .c * * .cpp, .h, .h* files and did not find that. A quick guess is that a header file, probably signal.h is missing in your build environment.

After that, it would help to know the usual suspects: architecture, os, & os version.

I have a SN CI run waiting to finish. I'll see if I can replicate, given the info you so kindly provided.

LeeTibbert avatar Oct 16 '25 17:10 LeeTibbert

There were some Cmumble exception handling changes in 0.5.8 but, on a quick check of the SN 0.5.9 release notes I saw nothing relevant. Not to say it is not there, just that it is not obvious.

LeeTibbert avatar Oct 16 '25 17:10 LeeTibbert

I'm running on Ubuntu 24.04, x86-64. I also forgot to mention that changing //> using nativeVersion 0.5.9 to //> using nativeVersion 0.5.8 solves the issue.

majk-p avatar Oct 16 '25 17:10 majk-p

FYI there's a relevant discussion on #scala-native channel in Scala discord

majk-p avatar Oct 16 '25 17:10 majk-p

  • Could not replicate using macOS arm, 16.mumble scala-cli broken_ce.scala where the file is the gist above.

  • Did replicate on Intel Linux 6.17.3+

Optimizing (debug mode) (7824 ms)
[error] /mnt/AthaBHdd1/lee49/DevoHdd1/scala_native/WorkSpace_1/2025/.scala-build/2025_b3557dfd11-dc7b6f9c58/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c:9:22: error: variable has incomplete type 'struct sigaction'
[error]     9 |     struct sigaction action;
[error]       |                      ^
[error] /mnt/AthaBHdd1/lee49/DevoHdd1/scala_native/WorkSpace_1/2025/.scala-build/2025_b3557dfd11-dc7b6f9c58/native/native/dependencies/cats-effect_native0.5_3-3.7.0-RC1-1/scala-native/signal_helper.c:9:12: note: forward declaration of 'struct sigaction'
[error]     9 |     struct sigaction action;
[error]       |            ^

// And a bunch more

This adds evidence to the missing .h file and shows CE signal_helper.c as the probably problem child.

Let me know if I can help resolve this.

LeeTibbert avatar Oct 16 '25 18:10 LeeTibbert

Some success, thanks to @ekrich & the reproducer above

$ scala-cli  --native-compile "-D_GNU_SOURCE" broken_ce.scala
[info] Linking (multithreadingEnabled=true, disable if not used) (5900 ms)
[info] Discovered 2666 classes and 17685 methods after classloading
[info] Checking intermediate code (quick) (288 ms)
[info] Discovered 2638 classes and 13835 methods after optimization
[info] Optimizing (debug mode) (7811 ms)
[error] /mnt/AthaBHdd1/lee49/DevoHdd1/scala_native/WorkSpace_1/2025/.scala-build/2025_b3557dfd11-dc7b6f9c58/native/native/dependencies/javalib_native0.5_3-0.5.9-0/scala-native/sys/linux_sched.c:11:9: warning: '_GNU_SOURCE' macro redefined [-Wmacro-redefined]
[error]    11 | #define _GNU_SOURCE
[error]       |         ^
[error] <command line>:1:9: note: previous definition is here
[error]     1 | #define _GNU_SOURCE 1
[error]       |         ^
[error] 1 warning generated.
[info] Produced 5 LLVM IR files
[info] Generating intermediate code (11084 ms)
[info] Compiling to native code (7308 ms)
[info] Linking with [pthread, dl, m]
[info] Linking native code (immix gc, none lto) (448 ms)
[info] Postprocessing (0 ms)
[info] Total (30318 ms)
Hello!

Yes, the redefinition message should probably be fixed for SN 0.5.10-SNAPSHOT to be guarded by a roughly, not compiled:

#ifndef _GNU_SOURCE
# define _GNU_SOURCE 1
#endif

I think unconditionally defining it to the same thing it already is would work also, but guarding it is clearer, at least to me.

Since the file is 'linux_mumble', there is probably not much chance to expunge the use of _GNU_SOURCE.

A quick fix for CE would be to add that guard code to the top of signal_helper.c.

I am off to rat around in linux signal.h to figure out why it seems to need _GNU_SOURCE. There may be a way to eliminate that requirement.

LeeTibbert avatar Oct 16 '25 19:10 LeeTibbert

When I use either gcc or clang to compile the CE file signal_helper.c by itself in a 'clean" environment it fails with "" and succeeds with "clang -c -std=gnu11 signal_helper.c" and fails with "clang -c -std=c11 signal_helper.c".

The file <bits/sigaction.h> probably has something GNU specific. I did not catch it the first time around. May have to use to compiler to do pre-process only to help my search.

LeeTibbert avatar Oct 16 '25 19:10 LeeTibbert

sigaction is documented to need _POSIX_C_SOURCE. And indeed #define _POSIX_C_SOURCE 1 works on my machine (linux). We'll see what CI thinks about that (WIP PR: #4510).

(I don't know what's the exact relation of _GNU_SOURCE and _POSIX_C_SOURCE.)

durban avatar Oct 16 '25 19:10 durban

OK, progress, if not a solution.

SN 0.5.9 changed from a default "-std=gnu99" to a "-std=c11" and did not describe the change. The change to a 14 year old standard is almost certainly for the better, IMO but it hosed people. Sorry.

"-std=c11" means, for both clang & gcc, standard ISO C. The structure and functions, now missing in CE 'signal_helper.c' are described in the Open Group (POSIX) 2024 signal.h specification are marked 'CX'. That means that they are extensions to ISO C. It also means that "-std=c11" (or other -std=cMumble) will not include them.

One must specify either "__POSIX_SOURCE" or "__GNU_SOURCE" either in the file before any other "#include" or on the command line "clang -D __POSIX_SOURCE foo.c", If in the file, should probably be guarded by an "#ifndef' or, better yet, code to detect a conflicting "-std' such a "__GNU_SOURCE".

The two activate slightly different code paths and definitions. Both allow 'signal_helper.c' to compile. I did not check if using one or the other made any difference in the file. Using "__GNU_SOURCE" for that file restores the SN 0.5.8 behaviour, for that file only. Using "__POSIX_SOURCE" is more current and more likely to port, if one can say that signals are portable.

Short story: I suggest a small edit to CE 'signal_helper.c'. Since this is a SN change, if pressed hard enough I could probably contribute the PR.

LeeTibbert avatar Oct 16 '25 20:10 LeeTibbert

I think it should be set with the version you would like. e.g. #define _POSIX_C_SOURCE 200809L

ekrich avatar Oct 17 '25 00:10 ekrich

Following up on @ekrich suggestion

#define _POSIX_C_SOURCE 202405L // Open Group 2024 Issue 8

The comment grounds the magic number. Using Issue 8 value of 202405 rather than empty or 1, shows that one has considered changes to POSIX/Open_Group since 1999 or so.

Probably more important when one is using a function, macro, or constant introduced in a particular version, say version 6 rather than 1. IIUC, that is not the case in this file currently. I get into the habit of writing defensively and also informatively for people who later have to maintain.

LeeTibbert avatar Oct 17 '25 08:10 LeeTibbert