feat(r2): backtrace and control flow

[chinggg]

Checklist

Which kind of PR do you create?

• [ ] This PR only contains minor fixes.
• [ ] This PR contains major feature update.
• [x] This PR introduces a new function/api for Qiling Framework.

Coding convention?

• [x] The new code conforms to Qiling Framework naming convention.
• [x] The imports are arranged properly.
• [x] Essential comments are added.
• [x] The reference of the new code is pointed out.

Extra tests?

• [ ] No extra tests are needed for this PR.
• [ ] I have added enough tests for this PR.
• [x] Tests will be added after some discussion and review.

Changelog?

• [ ] This PR doesn't need to update Changelog.
• [x] Changelog will be updated after some proper review.
• [ ] Changelog has been updated in my PR.

Target branch?

• [x] The target branch is dev branch.

One last thing

• [x] I have read the contribution guide

---

Now I have implemented callstack and backtrace, though the features have not been fully tested yet. I am currently working on deflat as a show case of control flow analysis ability

[elicn]

I am failing to understand why you had to change the memory module.. Was there something missing or not behaving as expected?

[wtdcode]

I am failing to understand why you had to change the memory module.. Was there something missing or not behaving as expected?

By changing the memory module, Qiling and r2 will share the same memory and thus solve the previously mentioned memory sync problem.

[elicn]

By changing the memory module, Qiling and r2 will share the same memory and thus solve the previously mentioned memory sync problem.

Pfff.. thigs are getting more and more complicated. Having the memory content being managed in Python code is not a great idea.. I guess that is going to come with a great performance hit and tons of bugs. As a first exmaple, the memory range splitting is handling it wrong (ranges are splitted, but their data isn't).

[wtdcode]

By changing the memory module, Qiling and r2 will share the same memory and thus solve the previously mentioned memory sync problem.

Pfff.. thigs are getting more and more complicated. Having the memory content being managed in Python code is not a great idea.. I guess that is going to come with a great performance hit and tons of bugs. As a first exmaple, the memory range splitting is handling it wrong (ranges are splitted, but their data isn't).

For performance, I don't think it will have any performance overhead as we are passing the raw pointer to Unicorn. The only difference is that now the memory is allocated by Python. @chinggg maybe you could do a small benchmark on uc_mem_map and uc_mem_map_ptr with uc_mem_write/read

For bugs, the memory splitting handling is buggy indeed (and probably results in the CI crash @chinggg) but can be fixed easily. I don't think it will introduce too many bugs as we have enough tests. Note even unicorn wrote the memory split on its own instead of using any qemu code.

[chinggg]

I know little about Unicorn internal and cannot find methods to pass uc mem to r2, so we can either allocate memory in r2 or Python code if we want to sync memory between them. I admit it is weird to deal with memory management stuff in Python since that should be the strength of Unicorn. But it seems to be more acceptable than to malloc in r2 after making PoC of both approaches. Now the memory is not handled well so all the CIs failed, but what is promising is that most simulations can still work without too much refactoring. I hope it can be fixed easily and the performance is not bad as @wtdcode assumed. I have another concern for r2's functionality. Since we just pass mapped memory instead of the whole file, it may miss much information about symbols and file structures. But the static information is really what we want at first.

[chinggg]

It seems now there leaves only two errors in test_android, but I cannot figure out why, maybe some multithreading issue. Hoping someone can have a quick review on it so I can solve it easily.

[chinggg]

I add assert_mem_equal check to ql mem operations and syscalls like brk and mmap so more errors occured. According to the CI testing result https://github.com/qilingframework/qiling/actions/runs/3065273326/jobs/4949204918, there seems to be 3 kind of error in test_elf:

• For binary of MIPS architecture, the map address is wrong when exceeding 0x80000000 https://github.com/unicorn-engine/unicorn/issues/1708
• the one in ld.so Memory region 3 0x47e1000 - 0x47e2000 != map_info[3] ld-linux.so.2
• the one in libc.so Memory region 10 0x7fffb81c1000 - 0x7fffb81c3000 != map_info[10] [mmap] libc.so.6

I find that execution may still be successful if these assertions are removed, since there is only bytes level difference between uc mem and ql map_info according to the dump result.