Support calls with unaligned stack on x86

[marcin-szmagara]

As far as I can see, on x86, this library assumes that target functions will always be called with %rsp properly aligned.

However, some of the functions I'd like to hook are called with %rsp unaligned. If transit function is not used, it jumps to the hook function and the stack is still unaligned. This may cause issues.

It gets even worse when the transit procedure is used, as it contains movdqa instructions which immediately segfault.

From what I can see, to support this, one needs to:

1. force transit usage
2. add code to check and fix alignment in the transit function.

Is this right?

[marcin-szmagara]

I tried following the above steps:

1. I use the new prehook functionality which always uses transit.
2. I made some changes to transit function for x86_64-sysv

It seems to be working for me.

[kubo]

Fixing alignment for hook functions isn't that simple. (1) Fixed alignment must be back to the original before returning to the caller. (2) Hook functions get incorrect arguments when alignment is fixed and the arguments are passed on the stack.

On the other hand, fixing alignment for prehook is simple as you did.

2. I made some changes to transit function for x86_64-sysv

I would do it as follows.

1. add or $0xfffffffffffffff0, %rsp just after sub $0xd0, %rsp.
2. replace all movdqa with movdqu.