LKL does not use x86-64 Linux syscall numbers

[mikbras]

lkl and sgx-lkl-musl seem to be using the ARM 64-bit (aarch64) syscall numbers rather than the X86 64-bit (x86_64) syscall numbers.

While debugging I noticed that close() passes 57 to lkl_syscall().

./arch/aarch64/bits/syscall.h.in:
#define __NR_close 57

./arch/x86_64/bits/syscall.h.in:
#define __NR_close 3

Syscall log from running helloworld:

[   SGX-LKL  ] sgxlkl_enclave_init(ethread_id=0)
[[  SGX-LKL ]] wg0 has public key 2/osfyWzeq5L/evtMsqoUGPFJyhUDPbRJKTsNXyJ9zo=
[[  SGX-LKL ]] WARN: ********** sgxlkl_user_enter()
[[  SGX-LKL ]] WARN: syscall begin: no=56 // openat
[[  SGX-LKL ]] WARN: syscall begin: no=63 // read
[[  SGX-LKL ]] WARN: syscall begin: no=62 // lseek
[[  SGX-LKL ]] WARN: syscall begin: no=63 // read
[[  SGX-LKL ]] WARN: syscall begin: no=62 // lseek
[[  SGX-LKL ]] WARN: syscall begin: no=63 // read
[[  SGX-LKL ]] WARN: syscall begin: no=63 // read
[[  SGX-LKL ]] WARN: syscall begin: no=57 // close
[[  SGX-LKL ]] WARN: syscall begin: no=96  // set_tid_address
[[  SGX-LKL ]] WARN: syscall begin: no=56 // openat
[[  SGX-LKL ]] WARN: syscall begin: no=167 // prctl
[[  SGX-LKL ]] WARN: syscall begin: no=57 // close
[[  SGX-LKL ]] WARN: syscall begin: no=167 // prctl
[[  SGX-LKL ]] WARN: syscall begin: no=56 // openat
[[  SGX-LKL ]] WARN: syscall begin: no=63 // read
[[  SGX-LKL ]] WARN: syscall begin: no=29 // ioctl
[[  SGX-LKL ]] WARN: syscall begin: no=66 // writev
Hello SGX-LKL-OE World!
[[  SGX-LKL ]] WARN: syscall begin: no=94 // exit_group
[    0.407723] reboot: Restarting system
[   SGX-LKL  ] init (0: 0 exit=0)
[   SGX-LKL  ] oe_terminate_enclave... done
[   SGX-LKL  ] SGX-LKL-OE exit: exit_status=0

[mikbras]

Update: sgx-lkl-musl is getting its syscall number definitions from here:

sgx-lkl/build_musl/lkl/include/lkl/bits.h

[vtikoo]

Its not clear to me where the generator of that file gets the syscall numbers from.

[davidchisnall]

LKL uses a small tweak on the generic Linux syscall numbers. x86-64 uses a large tweak to those numbers. Linux does not standardise syscall numbers across architectures. Musl has code that maps from x86-64 to LKL syscall numbers for callers of syscall.

It would be quite nice to move that into LKL and allow LKL to provide an x86-64 or AArch64-shaped syscall table, but given that the only way of issuing a syscall from outside libc is via the syscall function (the syscall instruction does not work) it is not a high priority.

[paulcallen]

I would like this to be re-triaged in our Triage meeting where all the folks related to this should be present.