CosmoCC compiler components not detected as executable

[aronson]

Describe the bug The Cosmopolitan compiler does not work out of the box with meson.

To Reproduce Use the toy example from the tutorial.

Install the CosmoCC toolchain and binaries into /opt/cosmos/ or similar and use this cross file:

[constants]
path = '/opt/cosmos/bin'

[binaries]
c = path / 'x86_64-unknown-cosmo-cc'
cpp = path / 'x86_64-unknown-cosmo-c++'
objc = path / 'x86_64-unknown-cosmo-gcc'
ar = path / 'x86_64-unknown-cosmo-ar'
as = path / 'x86_64-unknown-cosmo-as'
strip = path / 'x86_64-unknown-cosmo-strip'
pkg-config = path / 'x86_64-unknown-cosmo-pkg-config'
windres = path / 'x86_64-unknown-cosmo-windres'

[properties]
# Directory that contains 'bin', 'lib', etc
root = '/opt/cosmos'
# Directory that contains 'bin', 'lib', etc for the toolchain and system libraries
sys_root = '/opt/cosmos/x86_64-linux-cosmo'
needs_exe_wrapper = false

Meson setup will fail claiming ar can't be executed, despite me executing it in my shell one command later successfully.

Expected behavior Setup allows the build system to compile.

The file in question executes just fine on all the platforms I tested it on. Seems Meson executable detection does not handle CosmoCC fat binaries.

system parameters

• Is this a cross build or just a plain native build (for the same computer)?
  • CosmoCC creates fat binaries for all systems but can be considered a native binary on every system building it
• what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.)
  • MacOS Ventura and Ubuntu 22.04 (tried both, on both x86_64 and ARM64)
• what Python version are you using e.g. 3.8.0
  • 3.9.7
• what meson --version
  • 1.4.0
• what ninja --version if it's a Ninja build
  • 1.11.1.git.kitware.jobserver-1

[aronson]

I tried a workaround by making the x86_64-unknown-cosmo-ar executable be wrapped by a bash script and pointing meson to the script. It actually worked and I was able to proceed with my more complex builds using Cosmopolitan with few issues. I hope there's a better fix included upstream, but this is an effective workaround.

[tristan957]

You're going to need to add support for the compiler.

[aronson]

No, I'm not going to do that. Are you going to close the issue as won't fix?

[eli-schwartz]

Meson setup will fail claiming ar can't be executed, despite me executing it in my shell one command later successfully.

Can you post a useful error message? No screenshots please -- use textual code blocks -- and especially not screenshots that contain background wallpapers that make the screenshot of text, very hard to read.

[aronson]

Sure.

The Meson build system
Version: 1.4.0
Source dir: /Users/isaac/sources/error
Build dir: /Users/isaac/sources/error/build-cosmos
Build type: cross build
Project name: tutorial
Project version: undefined
C compiler for the host machine: /opt/cosmos/bin/x86_64-unknown-cosmo-cc C linker for the host machine: /opt/cosmos/bin/x86_64-unknown-cosmo-cc
(gcc 12.3.0 "x86_64-unknown-cosmo-cc (GCC) 12.3.0" )
ld.bfd 2.42
meson.build:1:0: ERROR: Failed running '/opt/cosmos/bin/x86_64-unknown-cosmo-ar'/ binary or interpreter not executable.
Possibly wrong architecture or the executable bit is not set.
A full log can be found at /Users/isaac/sources/error/build-cosmos/meson-logs/meson-log.txt

I can execute it in my terminal directly.

[eli-schwartz]

Thanks. Can you upload the file /Users/isaac/sources/error/build-cosmos/meson-logs/meson-log.txt ?

[aronson]

Yes. It will be a few hours as I am away from my machine. On it.

[tristan957]

No, I'm not going to do that. Are you going to close the issue as won't fix?

Not possible for us to implement support for every compiler that exists because we only have a finite number of systems and willpower.

That being said, looks like Eli is helping out.

[eli-schwartz]

cosmocc's compiler and linker drivers either are GCC and binutils based, or masquerade as them and expect GCC/binutils style arguments.

That's good enough for meson to partially support cosmocc, which implies we may not need to do anything, but the static archiver is erroring out due to an exec error, rather than because meson couldn't identify which type of archiver it is and what argument style to use.

I'm interested to see why it fails to exec. We can do a better analysis once that is figured out.

[aronson]

I believe cosmocc's core tools do indeed masquerade as GCC/binutils and generally accept that style of arguments.

The error log you requested:

Build started at 2024-04-08T12:13:15.409691
Main binary: /Users/isaac/.pyenv/versions/3.9.7/bin/python3.9
Build Options: --cross-file=cosmos.ini
Python system: Darwin
The Meson build system
Version: 1.4.0
Source dir: /Users/isaac/sources/error
Build dir: /Users/isaac/sources/error/build-cosmos
Build type: cross build
Project name: tutorial
Project version: undefined
-----------
Detecting compiler via: `/opt/cosmos/bin/x86_64-unknown-cosmo-cc --version` -> 0
stdout:
x86_64-unknown-cosmo-cc (GCC) 12.3.0
Copyright (c) 2024 Justine Alexandra Roberts Tunney
Cosmopolitan Libc and LLVM libcxx/compiler-rt are subject to non-GPL
notice licenses, e.g. ISC, MIT, etc. Your compiled programs must embed
our copyright notices. This toolchain is configured to do so default.
Cosmopolitan comes with absolutely NO WARRANTY of any kind.
For more information, see the Cosmopolitan LICENSE files.
Copyright (C) 2022 Free Software Foundation, Inc.
This launches GNU GCC/Binutils subprocesses, which is free software; see
cosmocc's LICENSE files for source code and copying conditions. There is
NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-----------
Running command: /opt/cosmos/bin/x86_64-unknown-cosmo-cc -E -dM -
-----
-----------
Detecting linker via: `/opt/cosmos/bin/x86_64-unknown-cosmo-cc -Wl,--version` -> 0
stdout:
GNU ld (GNU Binutils) 2.42
Copyright (C) 2024 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
-----------
stderr:
collect2 version 12.3.0
/opt/cosmos/bin/../libexec/gcc/x86_64-linux-cosmo/12.3.0/ld.bfd -m elf_x86_64 -static -L/opt/cosmos/bin/../x86_64-linux-cosmo/lib -L/opt/cosmos/bin/../lib/gcc /opt/cosmos/bin/../x86_64-linux-cosmo/lib/ape-no-modify-self.o /opt/cosmos/bin/../x86_64-linux-cosmo/lib/crt.o --version -z noexecstack -T /opt/cosmos/bin/../x86_64-linux-cosmo/lib/ape.lds -z common-page-size=4096 -z max-page-size=16384 -lcosmo
-----------
Sanity testing C compiler: /opt/cosmos/bin/x86_64-unknown-cosmo-cc
Is cross compiler: True.
Sanity check compiler command line: /opt/cosmos/bin/x86_64-unknown-cosmo-cc sanitycheckc.c -o sanitycheckc_cross.exe -c
Sanity check compile stdout:

-----
Sanity check compile stderr:

-----
C compiler for the host machine: /opt/cosmos/bin/x86_64-unknown-cosmo-cc (gcc 12.3.0 "x86_64-unknown-cosmo-cc (GCC) 12.3.0")
C linker for the host machine: /opt/cosmos/bin/x86_64-unknown-cosmo-cc ld.bfd 2.42
-----------
Detecting archiver via: `/opt/cosmos/bin/x86_64-unknown-cosmo-ar --version` -> Failed running '/opt/cosmos/bin/x86_64-unknown-cosmo-ar', binary or interpreter not executable.
Possibly wrong architecture or the executable bit is not set.

meson.build:1:0: ERROR: Failed running '/opt/cosmos/bin/x86_64-unknown-cosmo-ar', binary or interpreter not executable.
Possibly wrong architecture or the executable bit is not set.

This is the output of /opt/cosmos/bin/x86_64-unknown-cosmo-ar when run in my shell:

Usage: /opt/cosmos/bin/x86_64-unknown-cosmo-ar [emulation options] [-]{dmpqrstx}[abcDfilMNoOPsSTuvV] [--plugin <name>] [member-name] [count] archive-file file...
       /opt/cosmos/bin/x86_64-unknown-cosmo-ar -M [<mri-script]
 ...[continues for many lines with options]

The screenshots from earlier demonstrate the OS has the capability to execute the binary like any other, so I tried this python script: execme.py

import subprocess
subprocess.Popen(["/opt/cosmos/bin/x86_64-unknown-cosmo-ar"])

And got this response:

$ python execme.py

Traceback (most recent call last):
  File "/Users/isaac/sources/error/execme.py", line 2, in <module>
    subprocess.Popen(["/opt/cosmos/bin/x86_64-unknown-cosmo-ar"])
  File "/Users/isaac/.pyenv/versions/3.9.7/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/isaac/.pyenv/versions/3.9.7/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/opt/cosmos/bin/x86_64-unknown-cosmo-ar'

Perhaps this is a python "bug"? As noted I was using 3.9, so I tried 3.12 as well and got the same error.

As a sanity check, replacing /opt/cosmos/bin/x86_64-unknown-cosmo-ar with /bin/bash in the Popen call results in no error:

python3.12 execbash.py

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
bash-3.2$ exit

Replacing it with /opt/cosmos/bin/x86_64-unknown-cosmo-cc also succeeds, which is backed up by the meson log.

Some cosmocc files load via python's subprocess (cc, ld), some don't (ar). I'm not sure this is in scope for the meson project itself. To support cosmocc as-is I believe meson would have to wrap its test calls to cosmocc binaries in a shell.

[eli-schwartz]

Have you tried filing a bug report against cosmocc?

If I had to hazard a guess it's because python is using exec directly rather than what shells often do -- which is, treat a script without a shebang as implicitly executed by either /bin/sh or $SHELL, depending on personal taste of the shell author.

[aronson]

I will do that. Thanks for looking at this. Closing it as it's out of scope.