Emulate Golang compiled executables

[miaoski]

Hi. I was testing Qiling with a standard Hello World written in GO. There are several unimplemented APIs and every Go executables I've tested resulted in the same error. Sorry that I'm a newbie to Qiling and I don't know how to implement the APIs.

hello.go

package main

import "fmt"

func main() {
    fmt.Println("Hello, World")
}

Compiled on Ubuntu 22.04 as Windows x86 and x86_64 executables,

$ go version
go version go1.19 linux/amd64
$ env GOOS=windows GOARCH=386 go build ./hello.go 
$ env GOOS=windows GOARCH=amd64 go build -o hello_x64.exe ./hello.go 

Qiling is the latest Dev branch on Python 3.10. I have manually copied mscoree.dll to rootfs.

In [1]: from qiling import *
In [2]: ql = Qiling(['/home/user/qiling/rootfs/x86_windows/bin/hello.exe'], '/home/user/qiling/rootfs/x86_windows/')
...
[!] 	api GlobalMemoryStatus (kernel32) is not implemented
[!] 	api GlobalMemoryStatusEx (kernelbase) is not implemented
[!] 	api RtlNtStatusToDosError (ntdll) is not implemented
[!] 	api RtlSetLastWin32Error (ntdll) is not implemented
[x] 	Error encountered while running mscoree.dll DllMain, bailing
[!] 	api _initialize_onexit_table (ucrtbase) is not implemented
...
In [3]: ql.run()
...
[x] 	eax	: 0x0
...
[x] 	Hexdump:
[x] 	8b 80 00 00 00 00 8b 40
[x] 	Disassembly:
[=] 	00432232 [hello.exe            + 0x032232]  8b 80 00 00 00 00    mov                  eax, dword ptr [eax]
...
[x] 	PC = 0x00432232 (/home/phil/qiling/rootfs/x86_windows/bin/hello.exe + 0x32232)
...
UcError: Invalid memory read (UC_ERR_READ_UNMAPPED)

x86_64 targets are slightly different, but still crashed in the same way. I have copied api-ms-win-*.dll to rootfs, but they are still not resolved.

In [4]: ql = Qiling(['/home/user/qiling/rootfs/x8664_windows/bin/hello.exe'], '/home/user/qiling/rootfs/x8664_windows/')
...
[!] 	api _CorImageUnloading (mscoree) is not implemented
[!] 	api _initialize_onexit_table (ucrtbase) is not implemented
...
In [5]: ql.run()
...
[=] 	Loading advapi32.dll ...
[=] 	Loading msvcrt.dll ...
[=] 	Done loading msvcrt.dll
[!] 	Failed to resolve api-ms-win-eventing-controller-l1-1-0.dll
[!] 	Failed to resolve api-ms-win-eventing-consumer-l1-1-0.dll
[!] 	Failed to resolve api-ms-win-eventing-consumer-l1-1-1.dll
[=] 	Loading sechost.dll ...
[=] 	Loading rpcrt4.dll ...
[=] 	Done loading rpcrt4.dll
[=] 	Done loading sechost.dll
[!] 	Failed to resolve api-ms-win-service-core-l1-1-0.dll
[!] 	Failed to resolve api-ms-win-service-core-l1-1-1.dll
[!] 	Failed to resolve api-ms-win-service-core-l1-1-2.dll
...
[!] 	api AddVectoredContinueHandler (kernel32) is not implemented
...
[x] 	rdi	: 0x6000000
[x] 	rbp	: 0x80000001cee0
[x] 	Hexdump:
[x] 	65 63 74 6f 72 65 64 43
[x] 	Disassembly:
[=] 	0000000180282e71 [kernel32.dll         + 0x092e71]  65 63 74 6f 72       movsxd               rsi, dword ptr gs:[rdi + rbp*2 + 0x72]
[x] 	007ffffffde000 - 0080000001e000   rwx     [stack]          
...

[miaoski]

hello-go.zip The zip includes hello.go and the x86 and x86_64 binaries.

[miaoski]

It looks like the cause is the unimplemented AddVectoredContinueHandler in case of x86_64.

[+] 	0x000000018020a360: GetProcAddress(hModule = 0x1801f0000, lpProcName = "AddVectoredContinueHandler") = 0x180282e64
[!] 	api AddVectoredContinueHandler (kernel32) is not implemented
[=] 	0000000180282e71 [kernel32.dll         + 0x092e71]  65 63 74 6f 72       movsxd               rsi, dword ptr gs:[rdi + rbp*2 + 0x72]

The disassembled code matches with x64_dbg, but the code is not executed in Win10.

0x180282e64:	push	rsp
0x180282e66:	push	rdx
0x180282e6b:	je	0x180282ed9
0x180282e6d:	push	rsi
0x180282e71:	movsxd	rsi, dword ptr gs:[rdi + rbp*2 + 0x72]
0x180282e76:	outsd	dx, dword ptr fs:[rsi]
0x180282e7a:	outsb	dx, byte ptr [rsi]
0x180282e7b:	je	0x180282ee6
0x180282e7d:	outsb	dx, byte ptr [rsi]
0x180282e7e:	jne	0x180282ee5

I'm not sure if I can implement it. Will try and would probably fail. :)

[miaoski]

The implementation results in incoherent results, which are very weird.

In [5]: ql = Qiling(['/home/phil/qiling/x8664_windows/bin/hello.exe'], '/home/phil/qiling/x8664_windows/', verbose=QL_VERBOSE.DEBUG)

In [6]: ql.run()
# Result in panic.go and dumps being successfully executed.

In [7]: ql.run()
# Result in UC_ERR_READ_UNMAPPED, but the disassembled code reads just right.

The patch and the testing result are attached. I don't really understand what went wrong and how to make it right. ipython.log golang-test-1.diff.gz

[newthis]

May you need to debug to find out which api really influences the program, follow the instruction at 0x432232

[miaoski]

I think the cause might be that GO heavily depends on AddVectoredContinueHandler, while I haven't really implemented it. Can't just passthru=True, either.

[elicn]

Hi @miaoski, welcome. The "unimplemented" messages may be misleading; they only mean there is no mock or signature available for that API, but the native code (if exists) is still called. You can get better insights if you run this with enhanced tracing:

from qiling.extensions import trace

# ql init goes here...

trace.enable_full_trace(ql)
ql.run()

Notes:

• DLLs are intentionally excluded from tracing; it would take forever otherwise
• If you get lost in the excessive output, pipe it to less or bat so you could browse and search through it
• If tracing takes too much time, you can reduce it by viewing only n records before crash. Replace the enable_full_trace line with: trace.enable_history_trace(ql, n) for any n value you find reasonable