Deferred signals investigation

[Sonicadvance1]

Overview

This has interesting complications and can resolve unity problems.

Current high level idea

Add a new config option or two that enables "deferred/safe signal handling"

• Introduce the concept of "exit points" in the JIT, places where we can handle state
• Sprinkle "exit points" around in blocks in such a way that there is a bounded time between them
• Modify signal setters to block all signals on signals entry
• Modify signal handlers to resume JIT execution up to the next "exit point", and then handle the signal
• Edge cases: signals on syscalls, signals in non-jit code
• Add a signal torture test or two to our tests

Other related ideas

• Add signal safe mode for all locks

Curent low level ideas for first implementation

• use ldb/stb [ctx-1], #0 + a guard page before the context for low cost handling of "should we exit from exit point" checks

(Description updated by @skmp)

[skmp]

Spent some time today on this as a break from the SMC work, and an inconvenience is that fully restoring a context seems quite hard on aarch64, and possibly will require some sort of SMC magic (limited branch range) or a dedicated scratch register (returns to JIT only) or slightly abusing rt_sigreturn (see https://github.com/deroko/switch/blob/master/switch.md for some juicy fun).

Another approach is to resume from the signal and implement our own signal queues in user mode, or to modify the restored signal state. I think i'll explore this route first.

[skmp]

Investigating further while waiting for the SMC reviews,

• Implemented the usermode queuing (can only queue one signal per thread right now) using the str [ctx-1], 0 trick + mprotect to trigger a second fault, later on, that is used to dispatch the pending signal.
• Modified signal dispatch logic to always build guest ucontext, and to not store host context
• Modified signal returns to discard host context and redispatch the guest ucontext
• Still returns using ud2, though implementing rt_sigreturn will be trivial at this point
• Several incompatibilities with the kernel though
• Added special support for syscalls that self-restart to not defer the signals.
• Added basic exception support for LoadMemTSO/StoreMemTSO (via a special op, MemCheck)
• Works for x86-64 host / guest pair

Also hastily merged in the SMC work as unity depends on that too. Will rebase cleanly.

This gets Unity 2020.1 /much/ further, with Mango getting ingame.

Interestingly, unity (mono) needs full ucontext to recover from SIGSEGVs around null pointer accesses. Looks like mono generates NullReferenceExceptions and gracefully recovers there.

The code is very hasty at places. POC branch >> https://github.com/FEX-Emu/FEX/compare/skmp/deferred-signals-poc?expand=1

[skmp]

Spent some time on this over the weekend, I think we should split to two parts.

Deferring guest signals for (a) signal safety and for (b) state reconstruction / recovery.

(a) Is easy to do by adding a "is signal pending" check before returns to the JIT (syscalls, compile code, thunks). This can be easily done either in the dispatcher/jit side, or the C++ side. The only complication is automatically restarted system calls. It has near zero overhead, and doesn't suffer from execution 'overshot', just delayed signal delivery. This will largely resolve signal safety issues.

(b) Is more tricky and will come with a perf cost.

[skmp]

For thunks, we may want to allow them to be interrupted, as that would be closer to what the guest application can observe when running natively. Recovering the context is not a worry in that case.