get_gdt_base invokes undefined behaviour on x64

[FortuneDan]

https://github.com/ayoubfaouzi/al-khaser/blob/master/al-khaser/Shared/Utils.cpp#L869

ULONG get_gdt_base()
{
	// Get the base of Global Descriptor Table (GDT)

	UCHAR gdtr[6];
	ULONG gdt = 0;

	// sgdt instruction stores the contents of the GDT Register
	// (the GDTR which points to the GDT) in a processor register.
#if defined (ENV32BIT)
	_asm sgdt gdtr
#endif
	gdt = *((unsigned long *)&gdtr[2]);
	// printf("GDT base: 0x%x\n", gdt);

	return gdt;
}

gdtr is uninitialized local array, and ONLY on 32bit is it filled with the sgdt opcode.

On x64 it just stores the uninitialized value from gdtr into gdt.

Then the check the GDT trick looks to see if the upper byte is FF:

BOOL gdt_trick()
{
	UINT gdt_base = get_gdt_base();

	if ((gdt_base >> 24) == 0xff)
		return TRUE; // VMWare detected	

	else
		return FALSE;
}

I received a random failure of this test, that's what lead me to investigate. The failure was on VirtualBox, never failed ever before just randomly once, and this test clearly looks for "vmware".

The fix is to initialize the local variable, or IMO the better fix is to wrap the code that is actually used for 32bit in the 32bit ifdefs:

diff --git a/al-khaser/Shared/Utils.cpp b/al-khaser/Shared/Utils.cpp
index 7f07fd503..d4b699508 100644
--- a/al-khaser/Shared/Utils.cpp
+++ b/al-khaser/Shared/Utils.cpp
@@ -942,15 +942,15 @@ ULONG get_gdt_base()
 {
        // Get the base of Global Descriptor Table (GDT)

-       UCHAR gdtr[6];
        ULONG gdt = 0;

        // sgdt instruction stores the contents of the GDT Register
        // (the GDTR which points to the GDT) in a processor register.
 #if defined (ENV32BIT)
+       UCHAR gdtr[6] = {0};
        _asm sgdt gdtr
-#endif
        gdt = *((unsigned long *)&gdtr[2]);
+#endif
        // printf("GDT base: 0x%x\n", gdt);

        return gdt;