HigherOrderCO
Windows support
Right now the generated C code can't be compiled on Windows due to the use of <pthreads.h> etc.
One way it could be done would be to compile it using a compatibility layer like Cygwin, but I never did this before.
Help from some Windows programmer would be appreciated. :3
@quleuber Nowadays, they no longer use Cygwin. They simply use a real Linux kernel on Windows that is called Windows Subsystem for Linux WSL2. A full environment of a Linux distribution such as Ubuntu can be installed on WSL2, even GUI applications. https://ubuntu.com/wsl In the latest situation, they can install and launch the Ubuntu terminal with a few clicks.
So I guess we don't have to be concerned about this. Improvements needed, and planned features #38
@stken2050 WSL is great for development, but it's very different from running native programs from the "actual" user's point of view, which is what I'm talking about here.
This is not high priority but it would be great if we could do so with little maintenance overhead.
Btw, when did MS start to integrate GUI (X11/Wayland) in WSL? That's nice .o.
@quleuber Eah, I understand.
Btw, when did MS start to integrate GUI (X11/Wayland) in WSL? That's nice .o.
It's called WSLg https://github.com/microsoft/wslg on the timing of the release of Windows11 and the latest Windows10 update makes it default usage. Actually, I had swiched to Windows for a while because of WSL2 and WSLg, it's nice and easy.
I also noticed the C template depends on sys/time.h which would need to be replaced.
We can use the native MSVC thread APIs to replace <pthread.h>. I'm working on a branch now where I'm replacing all the POSIX function references with wrappers/shims that can be implemented for each platform. I'm using these two links as reference:
- docs.microsoft.com: Multithreading with C and Win32
- docs.microsoft.com: Sample Multithread C program
If I'm not wrong, this code can then be compiled by Clang, MSVC, TCC, or any other reasonable C compiler on Windows: it just depends on the Win32 API's header and DLL.
A decision must be made about how to provide/distribute/link these platform-dependent symbols:
- Providing symbols at link-time: in this approach, we would provide a
.lib/.afile or a.dll/.dylib/.sofile that the user must link against when compiling any C code that HVM generates. - Providing symbols at compile-time: we can load the right C code depending on the detected/requested target and just emit it with/before the template.
Approach 1 would preserve the readability of any generated code better, but would make any generated code less efficient: even with link-time optimization, I think compilers are restricted to peephole optimizations in object-code. On the other hand, approach 2 would yield better performance, but render the first few lines of the rendered C code gibberish. Of course, we could emit inline headers and provide the implementations later (like a human programmer would write it), but this makes the compiler a bit slower (two string concats over one)...
Finally, one might use OpenMP since it is very widely supported, but this would involve refactoring the template, and still would not solve the dependency on sys/time.h or any other platform-dependent libraries in the future. Once we have either approach above in place, we can call into a runtime to perform more than just multithreading and time calls (e.g. file I/O, networking, etc...)
@quleuber Does this work for the dev team? Do you have any preference between approaches 1 and 2 above? I'd be happy to implement, document, and test this feature if it's assigned to me.
Approach 1 would preserve the readability of any generated code better, but would make any generated code less efficient: even with link-time optimization, I think compilers are restricted to peephole optimizations in object-code. On the other hand, approach 2 would yield better performance, but render the first few lines of the rendered C code gibberish. Of course, we could emit inline headers and provide the implementations later (like a human programmer would write it), but this makes the compiler a bit slower (two string concats over one)...
I don't think it's either or. You can happily emit both (nicely readable non-performant as well as unreadable performant) depending of some command line argument passed to the emitter/compiler.
Btw. the links from https://github.com/Kindelia/HVM/issues/38#issuecomment-1027715103 might come handy for you when implementing Windows support (especially regarding TCC which has some slightly unexpected corner cases... - but TCC is definitely worth the hassle!).
@quleuber I have a WIP branch here: currently, I'm using the generator tokens as 'includes'.
This should be a 'best of both worlds' approach: the runtime.c file is mostly unmodified, and source files should be independently readable, even if autocomplete is confused.
https://github.com/tsnl/HVM/tree/windows_support
Looking forward to your feedback. You can ignore any files in the diff whose directories start with an underscore.
I'm liking this solution is heading so far. But I would really prefer if the base file (runtime.c) stayed valid - let's say, on POSIX - with some default code. We just need to implement block replacement and do something like this:
/*! BLOCK_DEPENDENCY_TIME */
#include <sys/time.h>
/* BLOCK_DEPENDENCY_TIME !*/
I'll work to implement this block substitution feature on master.
Edit: done e08f17e91cba65f9a94d0bbbc3365b0c7dd2c264
@quleuber would something like this work? The user would need to add a -I/path/to/hvm/src/runtime_deps flag when compiling. This can be a hassle to set up with some editors, but avoids us duplicating the relevant implementations.
One alternative would be to define relevant bits of the POSIX API for different platforms, e.g. we splice in a pthread.inl.c file whose interface exactly matches the pthread API, regardless of your platform. This is an appealing short-term option, but there's a risk we'll run into limitations given platform differences. I'm not an expert on this though. Do you know what the platform-dependent runtime code might need to do? I can try to pull together a subset of functions that are implementable across APIs/find something existing that does this if you're interested.
The other option is to just copy/paste the POSIX API file contents into the C file manually.
I'm going to keep working on the -I option, but perhaps someone here can think of something better?
/// ...
// Dep: Basic
// ----------
/*! GENERATED_DEPENDENCY_BASIC */
#include "basic/basic.inl.c"
/* GENERATED_DEPENDENCY_BASIC !*/
// Dep: Time
// ---------
/*! GENERATED_DEPENDENCY_TIME */
#include "time/time.inl.c"
#include "time/epilogue-posix.inl.c"
/* GENERATED_DEPENDENCY_TIME !*/
// Dep: Thread
// -----------
#ifdef PARALLEL
/*! GENERATED_DEPENDENCY_THREAD */
#include "thread/thread.inl.c"
#include "thread/epilogue-posix.inl.c"
/* GENERATED_DEPENDENCY_THREAD !*/
#endif
/// ...
UPDATE:
I'm afraid the current block replacement doesn't work for multi-line contents. Any chance you could fix this so the above template will work?
If implemented, I can also include a subset of V's MIT licensed headers for stdatomic.h using the same mechanism, allowing #62 to be closed.
@tsln multi-line block replacement should be working now (e636ed8927c56c4a11f3c75a5a8756fc25e466ab). (I keep forgetting about de semantics of . on regex :P)
I'm not sure if I understand.
The user would need to add a -I/path/to/hvm/src/runtime_deps flag when compiling. This can be a hassle to set up with some editors, but avoids us duplicating the relevant implementations.
Do you mean, to compile the standalone runtime.c file, while developing? That would be OK. But would be pretty bad on user side, it would make it a lot harder to ship.
My ideia is to keep those includes:
#include "basic/basic.inl.c"
so the file stays valid/compilable. But the block:
/*! GENERATED_DEPENDENCY_BASIC */
#include "basic/basic.inl.c"
/* GENERATED_DEPENDENCY_BASIC !*/
would be replaced by the actual platform-dependent code, instead of include directives, so we would not need an -I flag.
Are we on the same page?
Do you mean, to compile the standalone runtime.c file, while developing?
Yes, precisely!
But would be pretty bad on user side, it would make it a lot harder to ship.
I misspoke, the user wouldn't need to add the include flag since the content will be replaced using the block generator.
Thanks for implementing the block replacement!
I've implemented the cross-platform interface for time and thread.
While Linux seems to use it without too many problems, Windows is still missing functions for Condition Variables, which will be trickier than I first thought. I still need to debug and test both implementations.
I also tried adding support for stdatomic using V's headers. This works with GCC and MSVC where it relaxes the C11 constraint. Unfortunately, TCC still fails because the intrinsics providing atomics cannot be found at link-time. I read that linking libatomic would fix this, but I could not get this working on my machine. I've attached the compiler error I get, though you can reproduce all my results by running $ make all in the _scratch subdirectory.
I should wrap up the remaining Windows functions in a few more days at most. After this, I'll throw together some tests per-module, and then it should be good to PR.
I could not get the latest multiline tag replacement regex to work, so I did my best to fix it. Can you please tell me if my solution looks OK? I'm not very familiar with the Rust regex crate.
TCC error output: (test1.c generated from test1.hvm)
$ tcc -std=c99 -pthread test2.c -o tcc-test2
test1.c:470: warning: assignment from incompatible pointer type
test1.c:470: warning: assignment discards qualifiers from pointer target type
test1.c:474: warning: assignment from incompatible pointer type
test1.c:474: warning: assignment discards qualifiers from pointer target type
tcc: error: undefined symbol '__atomic_exchange_8'
tcc: error: undefined symbol '__atomic_store_8'
I could not get the latest multiline tag replacement regex to work, so I did my best to fix it. Can you please tell me if my solution looks OK?
It look OK, I just don't see how it work's differently from the other code. 🤔
@quleuber I believe the previous version just replaced the opening tag, and did not match across multiple lines. This one replaces the whole 'open.body.close' chunk.
Sorry I haven't had a chance to work on this, I have not been able to make the time to do the experimentation I need to do to support Windows threads. As a temporary compromise, I will get a version using ptw32 running in the next few days. This may complicate the build process, but I'll do my best to keep it simple. We can replace this later.
Will put in a PR in the next few days. Sorry for the foot-dragging!
This was closed because HVM now uses cross-platform Rust as a compile target instead of C, right? ( https://github.com/Kindelia/HVM/blob/master/guide/README.md#compiling-a-program )
Yes, sorry, forgot to add a comment. But we need to test it to make sure it indeed works as expected. Let me know otherwise.
