unicorn-engine
`test_arm64_mem_hook_read_write` fails on `riscv64` and `s390x`
I am experiencing the following test_arm and test_arm64 errors after successful build from current master on riscv64 and s390x arch under Alpine Linux 3.21:
Test test_arm64_mem_hook_read_write... [ FAILED ]
test_arm64.c:649: Check counter[0] == 4 && counter[1] == 4... failed
and
Test test_arm_mem_hook_read_write... [ FAILED ]
test_arm.c:864: Check counter[0] == 2 && counter[1] == 2... failed
Additionally s390x throws errors for test_mem, test_riscv and test_x86, I'll open a separate issue for that if necessary after this one.
Build logs are here: https://gitlab.alpinelinux.org/strophy/aports/-/pipelines/304520 I'm happy to push builds from specific commits through the Alpine CI system when you tag me, or you can do this yourself with a free account.
Building unicorn 2.1.3 on Fedora Rawhide has similar results. A log of the build is available at the URL below. The bottom of the log documents the failing tests.
https://download.copr.fedorainfracloud.org/results/mikep/scratch/fedora-rawhide-s390x/08742932-unicorn/builder-live.log.gz
Building unicorn 2.1.3 on Fedora Rawhide has similar results. A log of the build is available at the URL below. The bottom of the log documents the failing tests.
https://download.copr.fedorainfracloud.org/results/mikep/scratch/fedora-rawhide-s390x/08742932-unicorn/builder-live.log.gz
Thanks for updating here. Unfortunately, as I replied elsewhere, I can't do too much without access to a s390x environment. I will tey to emulate one via qemu later.
hi @wtdcode , if access to a s390x system would help, then please let me know, I can arrange that.
hi @wtdcode , if access to a s390x system would help, then please let me know, I can arrange that.
Thanks for that. You might send the way to access the environment by emailing me: [email protected]
During porting to s390x, I submitted another patch to QEMU:
https://lore.kernel.org/qemu-devel/[email protected]/T/#t
I'm done with initial port to s390x in acb638c40a9e108cedbf1212ddf646c7379f5736 but still we need some further testing for the incoming 2.2.0 with several big changes.
[mio@lfedora2 build]$ ctest
Test project /home/mio/unicorn/build
Start 1: test_x86
1/12 Test #1: test_x86 ......................... Passed 1.15 sec
Start 2: test_arm
2/12 Test #2: test_arm ......................... Passed 0.21 sec
Start 3: test_arm64
3/12 Test #3: test_arm64 ....................... Passed 0.06 sec
Start 4: test_m68k
4/12 Test #4: test_m68k ........................ Passed 0.01 sec
Start 5: test_mips
5/12 Test #5: test_mips ........................ Passed 0.02 sec
Start 6: test_sparc
6/12 Test #6: test_sparc ....................... Passed 0.00 sec
Start 7: test_ppc
7/12 Test #7: test_ppc ......................... Passed 0.03 sec
Start 8: test_riscv
8/12 Test #8: test_riscv ....................... Passed 0.06 sec
Start 9: test_s390x
9/12 Test #9: test_s390x ....................... Passed 0.01 sec
Start 10: test_tricore
10/12 Test #10: test_tricore ..................... Passed 0.00 sec
Start 11: test_mem
11/12 Test #11: test_mem ......................... Passed 0.03 sec
Start 12: test_ctl
12/12 Test #12: test_ctl ......................... Passed 0.05 sec
100% tests passed, 0 tests failed out of 12
Total Test time (real) = 1.65 sec
[mio@lfedora2 build]$
However, as this is no CI support, we are likely to break it some time in the future, I would appreciate it if anyone would like to offer a s390x CI environment =).
I think we can add one more matrix element in the workflow using the qemu action with that arch settings
I'm done with initial port to s390x in acb638c but still we need some further testing for the incoming 2.2.0 with several big changes.
However, as this is no CI support, we are likely to break it some time in the future, I would appreciate it if anyone would like to offer a s390x CI environment =).
Isn't s390x available in the github actions? I have found some references that it should be ...
I'm done with initial port to s390x in acb638c but still we need some further testing for the incoming 2.2.0 with several big changes. However, as this is no CI support, we are likely to break it some time in the future, I would appreciate it if anyone would like to offer a s390x CI environment =).
Isn't s390x available in the github actions? I have found some references that it should be ...
Github doesn't provide officially hosted runners and thus the only way is to spin up a qemu emulated s390x, which is painful slow. As previously tested, this could take ~30 minutes to complete building and testing.
Ah, I see, then self-hosted runners can be used per https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/mingmay-pang2/2024/02/05/github-actions-runner-for-ibm-z-and-linuxone
Ah, I see, then self-hosted runners can be used per https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/mingmay-pang2/2024/02/05/github-actions-runner-for-ibm-z-and-linuxone
I submitted a request there. Let's see. =)
Hi @wtdcode, thanks for the awesome work!
Would a riscv64 environment and/or CI runner be needed? We PLCT Lab can help.
Hi @wtdcode, thanks for the awesome work!
Would a riscv64 environment and/or CI runner be needed? We PLCT Lab can help.
Yeah, I'm reachable via telegram or emails.
Hello, is this still an open issue on RISC-V? Specifically, I am finding that the "sw" instruction is not consistently invoking the "UC_HOOK_MEM_READ".
2.1.3 seems to build on s390x:
https://copr.fedorainfracloud.org/coprs/mikep/scratch/build/9406307/
Hello, is this still an open issue on RISC-V? Specifically, I am finding that the "sw" instruction is not consistently invoking the "UC_HOOK_MEM_READ".
@LatinScribe That's a different issue (and should be resolved on dev already).
RISC-V port is still undergoing by the way while s390x port is done.
Hello, is this still an open issue on RISC-V? Specifically, I am finding that the "sw" instruction is not consistently invoking the "UC_HOOK_MEM_READ".
@LatinScribe That's a different issue (and should be resolved on dev already).
RISC-V port is still undergoing by the way while s390x port is done.
Thanks for the info! Is there any way to checkout the dev branch as a python package?
I've tried using "pip install git+https://github.com/unicorn-engine/unicorn.git@dev", but it seems like this repo is not set up as a python project ("ERROR: git+https://github.com/unicorn-engine/unicorn.git@dev does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found").
Any help would be greatly appreciated!
Figured it out, needed to use "pip3 install "git+https://github.com/unicorn-engine/unicorn.git@dev#subdirectory=bindings/python/""
Unfortunately, this has not solved the hook not working on "sw" instructions.
If you have any further suggestions, let me know, any ideas would be appreciated.
Just to clarify, I realized I made a typo on my previous comment. It's "lw" that is not invoking "UC_HOOK_MEM_READ", not "sw". sw (storing) seems to work fine, it's only when using "lw" (loading) that we are having issues.
Sorry if that caused any confusion!
Just to clarify, I realized I made a typo on my previous comment. It's "lw" that is not invoking "UC_HOOK_MEM_READ", not "sw". sw (storing) seems to work fine, it's only when using "lw" (loading) that we are having issues.
Sorry if that caused any confusion!
Thanks for letting me know your trial. Could you raise another issue with some reproduction code?
Just to clarify, I realized I made a typo on my previous comment. It's "lw" that is not invoking "UC_HOOK_MEM_READ", not "sw". sw (storing) seems to work fine, it's only when using "lw" (loading) that we are having issues. Sorry if that caused any confusion!
Thanks for letting me know your trial. Could you raise another issue with some reproduction code?
Done! See https://github.com/unicorn-engine/unicorn/issues/2229
Testing with 2.1.4, test_arm64_mem_hook_read_write still fails on riscv64 with:
Test test_arm64_mem_hook_read_write... [ FAILED ]
test_arm64.c:646: Check counter[0] == 4 && counter[1] == 4... failed
Test test_arm_mem_hook_read_write... [ FAILED ]
test_arm.c:862: Check counter[0] == 2 && counter[1] == 2... failed
The test_arm(64)_mem_hook_read_write issue seems to be resolved on s390x, but now there is a new test error on this arch:
Test test_x86_read_virtual... [ FAILED ]
test_x86.c:1693: Check parrent == 60... failed
test_x86.c:1694: Check child == 42... failed
Test test_virtual_write... [ FAILED ]
test_mem.c:587: Check rax == res... failed
Build logs from Alpine CI are here: https://gitlab.alpinelinux.org/strophy/aports/-/pipelines/360188
No hurry to fix this, just posting the current status. Thanks for your continued work on this!