MacOS 12 (x86_64) seems to use R10 instead of RCX as the fourth arg register

[golddranks]

trafficstars

Stumbled upon this just a moment ago.

Trying to issue posix_spawn system call. Here's the schematic: rdi: pointer to a int where to store the spawned process' pid. rsi: path rdx: pointer to a struct of settings, can be null rcx: pointer to argv r8: pointer to envp

And here's the code:

global start
start:
	mov r9, [rsp] ; argc
	lea rcx, [rsp + 8] ; argv: {"./syscall2\0", "/bin/test\0"}
	lea r8, [rsp + 8 + r9*8 + 8] ; envp
	push 0 ; pid
	mov rax, 0x020000F4 ; posix_spawn syscall
	mov rdi, rsp ; pointer to pid
	mov rsi, [rcx+8] ; argv[1]
	mov rdx, 0
	syscall
	mov rax, 0x02000001
	syscall

However, trying this out doesn't work. Spying the syscall in another terminal windows, by:

sudo dtrace -n 'syscall::posix_spawn*:entry { printf("%s %p %s %p %p %p",execname,arg0,copyinstr(arg1),arg2,arg3,arg4); }'

And launching the assembly program with nasm -f macho64 syscall2.asm && ld syscall2.o -static -o syscall2 && ./syscall2 /bin/test

dtrace finds that the call looks slightly off: posix_spawn:entry syscall2 7ff7bfeff6a0 /bin/test 0 0 7ff7bfeff6c8 The arg2 after /bin/test is supposed to be zero, but the arg3 is not! Clearly it's expecting arg3 in some other register!

After trial and error, I noticed that this code works:

global start
start:
	mov r9, [rsp] ; argc
	lea r10, [rsp + 8] ; argv: {"./syscall2\0", "/bin/test\0"}
	lea r8, [rsp + 8 + r9*8 + 8] ; envp
	push 0 ; pid
	mov rax, 0x020000F4 ; posix_spawn syscall
	mov rdi, rsp ; pointer to pid
	mov rsi, [r10+8] ; argv[1]
	mov rdx, 0
	syscall
	mov rax, 0x02000001
	syscall

The only difference is that rcx is changed to r10.

I don't have a clue, when this change has taken place or does it only happen on specific versions / hardware.

[golddranks]

Found an old blog post (https://filippo.io/making-system-calls-from-assembly-in-mac-os-x/) that states: "OS X (and GNU/Linux and everyone except Windows) on 64 architectures adopt the System V AMD64 ABI reference. Jump to section A.2.1 for the syscall calling convention."

The reference (https://refspecs.linuxbase.org/elf/x86_64-abi-0.99.pdf ) says: "1. User-level applications use as integer registers for passing the sequence %rdi, %rsi, %rdx, %rcx, %r8 and %r9. The kernel interface uses %rdi, %rsi, %rdx, %r10, %r8 and %r9."

Indeed, %rdi, %rsi, %rdx, %r10, %r8 and %r9 is the sequence that seems to really work. Maybe this cheat sheet needs fixing?