dosemu2 icon indicating copy to clipboard operation
dosemu2 copied to clipboard

use MAP_JIT on MacOS

Open stsp opened this issue 2 years ago • 38 comments

According to https://github.com/dosemu2/dosemu2/issues/1005#issuecomment-1821069604 it may be interesting to share the dosmem fd. It will also allow to clean up the fdpp API a bit. Unfortunately, the MacOS port currently doesn't support this.

https://keith.github.io/xcode-man-pages/mmap.2.html This page mentions MAP_JIT that seems to allow PROT_WRITE|PROT_EXEC together with MAP_SHARED. @bartoldeman can you check out this MAP_JIT thing?

stsp avatar Nov 21 '23 21:11 stsp

Hi Bart, why not to check MAP_JIT on MacOS? Should take a few minutes, not more...

stsp avatar Feb 05 '24 06:02 stsp

I removed mapashm again, as it is no longer compatible with native DPMI (can't leverage the remote mapping hook). So MAP_JIT needs to be tried instead.

stsp avatar Dec 07 '24 12:12 stsp

Unfortunately, the MacOS port currently doesn't support this.

I've searched for the MacOS port everywhere I can think of, but can't find it. Can anyone point me in the right direction? Many thanks.

emendelson avatar Jul 02 '25 14:07 emendelson

Maybe @bartoldeman knows better, but afaik you have to build it there from sources. I haven't set up any mac build farm. :(

Среда, 2 июля 2025 г получено от emendelson:

emendelson left a comment (dosemu2/dosemu2#2123)

Unfortunately, the MacOS port currently doesn't support this.

I've searched for the MacOS port everywhere I can think of, but can't find it. Can anyone point me in the right direction? Many thanks.

-- Reply to this email directly or view it on GitHub: https://github.com/dosemu2/dosemu2/issues/2123#issuecomment-3028144656 You are receiving this because you authored the thread.

Message ID: @.***

stsp avatar Jul 02 '25 15:07 stsp

When I tried to build with autogen.sh, default-configure, and make, I got this error: Please install i386-elf-binutils from https://github.com/nativeos/homebrew-i386-elf-toolchain. I tried to do exactly that, following the instructions in the README.md file to the letter (how to tap the repository with brew), but I consistently got error messages, and the installation didn't work.

I'll file a bug report with the homebriew-i386-elf-toolchain site in the hope of making it work. It would be terrific to have doesemu in macOS.

emendelson avatar Jul 03 '25 00:07 emendelson

This is most likely simply unneeded. Bart introduced this in 765f2e3f (Sat Jul 1 13:02:45 2023), but later 82f9fe37 (Tue Sep 12 19:14:04 2023) I started to support clang as as.

So I just removed this check. I'll also see what can be done with fdpp. While currently we don't link fdpp with lld, there are plenty of options. From downloading the pre-built fdppkernel.elf from CI, to linking the unrelocatable kernel.

stsp avatar Jul 03 '25 07:07 stsp

Will start a new issue on building under macOS:

https://github.com/dosemu2/dosemu2/issues/2491

emendelson avatar Jul 03 '25 12:07 emendelson

I haven't really touched dosemu for over a year, just regular life and work taking over. I'll try to make it build again.

bartoldeman avatar Jul 03 '25 17:07 bartoldeman

Wow, Bart is here! You haven't touched dosemu for over 2 years, to say the least. :) Good to know everything is allright.

stsp avatar Jul 03 '25 18:07 stsp

@bartoldeman if it gets to the point it builds again, maybe it's worth adding to CI.

stuaxo avatar Jul 04 '25 00:07 stuaxo

There are indeed a few github action for this, like that one: https://github.com/marketplace/actions/setup-osxcross-timmmm Also there seems to be the docker for off-line cross-compilation: https://github.com/crazy-max/docker-osxcross Does anyone want to try these things out? :)

stsp avatar Jul 04 '25 07:07 stsp

If I get it to build on work Mac, I'd give it a go... I'll see what happens there first.

stuaxo avatar Jul 04 '25 09:07 stuaxo

How to use that docker image? I built it, but there are no further instructions...

stsp avatar Jul 04 '25 09:07 stsp

I can have a look at the docker file later. In the meantime, I got as far as the libsearpc dependency, and then had trouble building that, so I'll look at that later too.

stuaxo avatar Jul 04 '25 10:07 stuaxo

--disable-searpc

stsp avatar Jul 04 '25 10:07 stsp

OK. configure completed.

...

Enabling plugins:

configure: WARNING: No video plugins available!

configure: WARNING: No audio plugins available!

configure: WARNING: fdpp not available!

configure: WARNING: searpc not available!

configure: WARNING: libacl not available!

configure: WARNING: libcap not available!
$ make
...
../../../src/plugin/plugin_config.hh:1:1: error: expected external declaration
    1 | -n
      | ^
../../../src/plugin/plugin_config.hh:1:2: error: unknown type name 'n'
    1 | -n
      |  ^
2 errors generated.
make[2]: *** [text.o] Error 1
make[1]: *** [base/video] Error 2
make: *** [default] Error 2

stuaxo avatar Jul 04 '25 11:07 stuaxo

$ cat src/plugin/plugin_config.hh
-n
#include "../plugin/Xkmaps/config/plugin_config.h"
#include "../plugin/modemu/config/plugin_config.h"

stuaxo avatar Jul 04 '25 11:07 stuaxo

You need at least sdl2! Otherwise you build mostly a non-functional thing. Of course this won't solve the compilation problem, but at least configure should do better.

stsp avatar Jul 04 '25 11:07 stsp

$ cat src/plugin/plugin_config.hh

In scripts/mkpluginhooks remove -n after echo and try again.

stsp avatar Jul 04 '25 11:07 stsp

That got further.

I checked and "sh" doesn't seem to have -n on MacOS. "dash", the minimal posix shell they have since Catalina does as well as bash and zsh.

$ make
...
ld: unknown options: -melf_i386
make[2]: *** [vesabios_pm.o] Error 1
make[1]: *** [base/dev/vga] Error 2
make: *** [default] Error 2

https://scriptingosx.com/2020/06/about-bash-zsh-sh-and-dash-in-macos-catalina-and-beyond/

stuaxo avatar Jul 04 '25 11:07 stuaxo

What kind of ld is that? Must be from cross binutils with i386 elf support.

Пятница, 4 июля 2025 г получено от Stuart Axon:

stuaxo left a comment (dosemu2/dosemu2#2123)

That got further.

I checked and "sh" doesn't seem to have -n on MacOS. "dash", the minimal posix shell they have since Catalina does as well as bash and zsh.

$ make
...
ld: unknown options: -melf_i386
make[2]: *** [vesabios_pm.o] Error 1
make[1]: *** [base/dev/vga] Error 2
make: *** [default] Error 2

https://scriptingosx.com/2020/06/about-bash-zsh-sh-and-dash-in-macos-catalina-and-beyond/

-- Reply to this email directly or view it on GitHub: https://github.com/dosemu2/dosemu2/issues/2123#issuecomment-3035759015 You are receiving this because you authored the thread.

Message ID: @.***

stsp avatar Jul 04 '25 11:07 stsp

Yes, I got it to build again, but it only boots and crashes trying to load emufs, then can't read anything. Needs a few patches though.

"echo -n xxx" can be replaced by printf %s xxx except for "echo -n *", which is used in hdimage, which was my mistake 2 years ago, so Stas reversed it. printf "%s " * may do the trick (with a trailing space), or echo * | tr -d '\n'

You need to install i386-elf-ld for vesabios_pm.o. Configure will find it. https://github.com/nativeos/homebrew-i386-elf-toolchain

there are a few other issues with linker flags (you need -Wl,-undefined,dynamic_lookup for dynamic plugins, which configure sets via DOSEMU_LDFLAGS but plugins don't use any more, on the other hand, -Wl,-Bsymbolic -Wl,-soname= is not understood). My build is quick and dirty, I'll get back to you later.

bartoldeman avatar Jul 04 '25 11:07 bartoldeman

Yes and indeed github actions supports a mac build, that'd would at least help avoiding accidentally breaking compilation.

bartoldeman avatar Jul 04 '25 11:07 bartoldeman

Ci won't save the day, as once something big is added again (like was searpc or dj64 support), then it will break and I won't have an idea how to fix. But it would still be good to have. Can then be disabled, re-enabled and so on.

Пятница, 4 июля 2025 г получено от Bart Oldeman:

bartoldeman left a comment (dosemu2/dosemu2#2123)

Yes and indeed github actions supports a mac build, that'd would at least help avoiding accidentally breaking compilation.

-- Reply to this email directly or view it on GitHub: https://github.com/dosemu2/dosemu2/issues/2123#issuecomment-3035844596 You are receiving this because you authored the thread.

Message ID: @.***

stsp avatar Jul 04 '25 12:07 stsp

I realized I should still answer the original question and then continue in the new issue https://github.com/dosemu2/dosemu2/issues/2491

MAP_JIT won't help me here, as MAP_JIT|MAP_SHARED is rejected with "Invalid argument", it only works with ANON; and I can't get executable shared memory working on a Mac.

It's needed on Macs with ARM chips (Apple silicon) to create jit-able memory (really memory that can be toggled rx<->rw, rwx is simply not allowed there)

On Intel Macs (one of which I'm working with so far), rwx is allowed (only) on anon mappings without needing MAP_JIT, unless I use the hardened runtime (which is off by default).

this all goes back to the solution in https://github.com/dosemu2/dosemu2/pull/2049#issuecomment-1630001169 but that same PR has another solution of simply not marking shared memory as executable unless native DPMI is used. I don't think native DPMI can work on MacOS anyway. Presently just getting the simulator back to work would be something.

bartoldeman avatar Jul 04 '25 14:07 bartoldeman

What kind of ld is that? Must be from cross binutils with i386 elf support.

Пятница, 4 июля 2025 г получено от Stuart Axon:

I think it's just the default one.

❯ which ld
/usr/bin/ld

░▒▓   …/dosemu2   devel !?   12:45 
❯ ld -v
@(#)PROGRAM:ld PROJECT:ld-1167.4.1
BUILD 20:00:10 Mar 12 2025
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
will use ld-classic for: armv6 armv7 armv7s i386 armv6m armv7k armv7m armv7em
LTO support using: LLVM version 17.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 17.0.0 (tapi-1700.0.3.4)

I tried: AS_LDFLAGS:=-arch i386 -platform_version macos 10.15 10.15

To get this far:

/usr/bin/ld -arch i386 -platform_version macos 10.15 10.15 -static  -o vesabios_pm.o.elf _vesabios_pm.o
ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)
ld: warning: ignoring file _vesabios_pm.o, building for macOS-i386 but attempting to link with file built for unknown-unsupported file format ( 0x7F 0x45 0x4C 0x46 0x01 0x01 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 )
Undefined symbols for architecture i386:
  "start", referenced from:
     -u command line option
ld: symbol(s) not found for architecture i386
make[2]: *** [vesabios_pm.o] Error 1
make[1]: *** [base/dev/vga] Error 2
make: *** [default] Error 2

I tried another suggestion of using clang as linker AS_LD:=/usr/bin/clang though get the same output.

stuaxo avatar Jul 04 '25 15:07 stuaxo

I got further specifying the entrypoint when building the vesa bios.

Current manual options in Makefile.conf:

XASFLAGS:=-x assembler -target i386-apple-darwin -c -
AS_LD:=/usr/bin/clang
AS_LDFLAGS:=-arch i386 -platform_version macos 10.15 10.15 -e _start

Gets to:

$ make
...
utilities.c:392:54: error: use of undeclared identifier 'memrchr'; did you mean 'memchr'?
  392 |                                 const char * slash = memrchr(res, '/', res_len);
      |                                                      ^~~~~~~
      |                                                      memchr
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_string.h:74:3: note: 'memchr' declared here
   74 |                 memchr(const void *_LIBC_SIZE(__n) __s, int __c, size_t __n);
      |                 ^
1 error generated.
make[2]: *** [utilities.o] Error 1
make[1]: *** [base/misc] Error 2
make: *** [default] Error 2

EDIT: It looks like memrchr isn't on MacOS

stuaxo avatar Jul 04 '25 15:07 stuaxo

@stuaxo I open coded memrchr. Attached is a quick and dirty diff, I'll clean it up for a PR. I played the first level of the original Prince of Persia with sound using $_cpuemu = (1) (ie. using simulation) on an Intel Mac, was all good.

dirtydiff.txt

bartoldeman avatar Jul 04 '25 17:07 bartoldeman

ANON; and I can't get executable shared memory working on a Mac.

Hi Bart, please find the mapping subsystem in a much better shape than 2 years ago. :) We no longer need executable shared mapping, well unless for vm86. Native DPMI is now remote, so at least in the parent it doesn't need PROT_EXEC. In #2492 I remove PROT_EXEC from most mem_bases.

stsp avatar Jul 04 '25 19:07 stsp

Thanks @bartoldeman I'll give it a go. From what I saw implementing it is definitely an option on MacOS.

stuaxo avatar Jul 04 '25 21:07 stuaxo