Is it possible to support cosmopolitan?

[duke13137]

Hi, thank you for the very neat library bringing C one step closer to Go :-)

Have you look into supporting cosmopolitan?

Looking at https://justine.lol/cosmopolitan/functions.html, cosmopolitan covers most of POSIX.

I changed one line in neco.c trying to compile one example.

-#elif defined(__EMSCRIPTEN__) || defined(_WIN32)
+#elif defined(__EMSCRIPTEN__) || defined(_WIN32) || defined(__COSMOCC__)

It seems only dladdr and _Unwind_Backtrace are missing to support llco

$ cosmocc -I../ ../neco.c channels.c -o channel
../neco.c: In function ‘llco_getsymbol’:
../neco.c:1352:20: warning: implicit declaration of function ‘dladdr’ [-Wimplicit-function-declaration]
 1352 |     if (sym->ip && dladdr(sym->ip, (void*)&dlinfo)) {
      |                    ^~~~~~
../neco.c: At top level:
../neco.c:3902:18: warning: not sure if modding const structs is good

 3902 | static const int ALLOWED_SIGNALS[] = {
      |                  ^~~~~~~~~~~~~~~
../neco.c:3912:18: warning: not sure if modding const structs is good

 3912 | static const int TRAPPED_SIGNALS[] = {
      |                  ^~~~~~~~~~~~~~~
cc1: note: rewrote 2 switch statements
cc1: note: modified 3 initializations
/opt/cosmocc/bin/../libexec/gcc/x86_64-linux-cosmo/12.3.0/ld.bfd: /tmp/fatcosmocc.6206z09eh66x1.o: in function `llco_getsymbol':
neco.c:(.text+0x7f9): undefined reference to `dladdr'
/opt/cosmocc/bin/../libexec/gcc/x86_64-linux-cosmo/12.3.0/ld.bfd: /tmp/fatcosmocc.6206z09eh66x1.o: in function `llco_unwind':
neco.c:(.text+0xa5e): undefined reference to `_Unwind_Backtrace'
/opt/cosmocc/bin/../libexec/gcc/x86_64-linux-cosmo/12.3.0/ld.bfd: /tmp/fatcosmocc.6206z09eh66x1.o: in function `is_main_thread':
neco.c:(.text+0x7683): undefined reference to `pthread_main_np'
collect2: error: ld returned 1 exit status

[tidwall]

I haven't considered cosmopolitain support, but it does seems interesting.

Have you tried adding LLCO_NOUNWIND define?

cosmocc -DLLCO_NOUNWIND -I../ ../neco.c channels.c -o channel

[duke13137]

Most of example/*.c work! (except portscan/echo-server/client.c SIGV :-)

❯ ./channel ;./coroutines; ./generators; ./select
ping
coroutine: A (0)
coroutine: B (0)
coroutine: C (0)
coroutine: A (1)
coroutine: B (1)
coroutine: C (1)
coroutine: A (2)
coroutine: B (2)
coroutine: C (2)
coroutine: A (3)
coroutine: B (3)
coroutine: C (3)
coroutine: A (4)
coroutine: B (4)
coroutine: C (4)
done
0
1
2
3
4
5
6
7
8
9
received one
received two

with these two #define changes.

diff --git a/neco.c b/neco.c
index 5735360..e329b77 100644
--- a/neco.c
+++ b/neco.c
@@ -2874,7 +2874,7 @@ bool worker_submit(struct worker *worker, int64_t pin, void(*work)(void *udata),
 #include <sys/epoll.h>
 #include <sys/eventfd.h>
 #define NECO_POLL_EPOLL
-#elif defined(__EMSCRIPTEN__) || defined(_WIN32)
+#elif defined(__EMSCRIPTEN__) || defined(_WIN32) || defined(__COSMOCC__)
 // #warning Webassembly has no polling
 #define NECO_POLL_DISABLED
 #else
@@ -3691,7 +3691,7 @@ static int is_main_thread(void) {
 static int is_main_thread(void) {
     return getpid() == (pid_t)syscall(SYS_gettid);
 }
-#elif defined(__EMSCRIPTEN__)
+#elif defined(__EMSCRIPTEN__)  || defined(__COSMOCC__)
 int gettid(void);
 static int is_main_thread(void) {
     return getpid() == gettid();

[tidwall]

Wow. That's kinda amazing actually.

[tidwall]

Just pushed an update to allow for basic support for Cosmopolitan.

[duke13137]

Hi, @tidwall thanks and congrats for the support!

It'd be great to hear your opinion if cosmos' epoll code is sufficient to support neco fd/socket features.

https://github.com/jart/cosmopolitan/blob/a31d5ea399a5fae9ff82c2944dab98e4a12e2bfa/libc/sock/epoll.c#L338

Cheers!

[duke13137]

I gave it a try with the patch below ...

CC=cosmocc tests/run.sh passed 💯!

diff --git a/neco.c b/neco.c
index 47dc2fe..a892c85 100644
--- a/neco.c
+++ b/neco.c
@@ -2878,7 +2878,11 @@ bool worker_submit(struct worker *worker, int64_t pin, void(*work)(void *udata),
 #include <sys/epoll.h>
 #include <sys/eventfd.h>
 #define NECO_POLL_EPOLL
-#elif defined(__EMSCRIPTEN__) || defined(_WIN32) || defined(__COSMOCC__)
+#elif defined(__COSMOCC__)
+#include "libc/sock/epoll.h"
+#include "libc/sysv/consts/epoll.h"
+#define NECO_POLL_EPOLL
+#elif defined(__EMSCRIPTEN__) || defined(_WIN32)
 // #warning Webassembly has no polling
 #define NECO_POLL_DISABLED
 #else
@@ -6486,7 +6490,9 @@ static void gai_args_free(struct getaddrinfo_args *args) {
         free0(args->service);
         free0(args->node);
         free0(args->hints);
-        freeaddrinfo(args->res);
+        if (args->res) {
+            freeaddrinfo(args->res);
+        }
         if (args->fds[0]) {
             must(close(args->fds[0]) == 0);
         }

[duke13137]

FYI, also copied those COMSO APE binaries to Windows 10 (compiled in WLS2), only two tests didn't pass. Looks like comso's windows posix layer have some portability issues.

test_net.exe

test_wait.exe

[tidwall]

I suspect this has something to do with the limitations of wepoll and the expectations that Neco has for epoll interface.

Neco uses epoll for all types of file descriptors, not just sockets. And it expects that sockets are real file descriptors, not just a Windows HANDLE.

Wepoll on its own doesn't fix this problem for Neco.

If cosmopolitan uses wepoll for all epoll functionality then those limitations will be extended to all programs that compile with cosmocc.

To provide real network and epoll-like support for neco on windows there needs to be:

1. Better windows/posix layer, where more file descriptor compatibly is extended to socket handles, and an epoll/kqueue equivalent that works on any file descriptor. To make this work it would be a big undertaking.

2. Or, it may be better to just use the native Windows sockets API and IO Completion Port, and wrap the code in #ifdef _WIN32 blocks.