criu
criu copied to clipboard
checkpoint-restore
Support for RISC-V
Hello Team,
I am fairly new to CRIU, and am quite interested in porting it to RISCV by my own but I feel kind of lost as there are lot of touch points. And am unable to wrap my head around the code.
So would love to know if are there are any plans to support for RISC-V, would love to pitch in.
Also would appreciate any help, if someone can point me to any documentation. I use this link : https://criu.org/Category:Under_the_hood but things are sorted by Alphabetical order and not the way they are used in code or would say I am unable to find any documentation on how CRIU works. So it's difficult to read the components and try to know where it's used in code.
Would love to contribute to this project, would very much appreciate any help.
Looking forward to it.
Thank you in advance.
Here am trying few things but could very wrong and naive, apologies am very new to all systems programming as well : https://github.com/checkpoint-restore/criu/compare/criu-dev...rushi47:mod_criu
@rushi47 awesome idea to port CRIU in RISC-V.
Please, tell me do you have RISC-V hardware, or do you want to port and test CRIU on the virtualized RISC-V environment?
First of all, I think you need to play with compel. Just forget about CRIU itself. Try to play with the examples in compel/test folder, try to run and understand the code. Then you need to make the compel work on the RISC-V arch.
The reason is simple - compel is the concentration point of the arch-specific code in CRIU. Because it needs to dump/restore tasks registers, make remote mappings, deal with signals, and so on and so forth. So if you get compel working it will be about 80% of all job.
Feel free to reach us.
Regards, Alex
The 2nd step will be criu/pie folder. It contains two interesting pieces - parasite (used on the dump stage) and restorer (used on the restore stage). Both are just PIEs which compiled using compel and injected into victim processes to do some useful things. For instance, parasite is used on the dump to call prctl() syscalls to get some victim processes information. restorer is called when we need to restore, for instance, prctl-related things or signals, aio ring, etc.
The idea is simple - all actions about dump/restore which can be done externally we do from the CRIU processes, but if some syscall or action has to be done from the victim process context when we use parasite/restorer blobs.
The code in criu/pie is mostly generic but it has some arch-specific parts (see criu/arch directory).
@mihalicyn Hello Alex :) Thank you so much for detailed explanation and extending support.
To answer your question - Yes as of now i will test it on virtualised environment. But I do have provision to get FPGA with RISCV so we can test there as well once, it runs in virtual environment.
& Thanks for this tip let me try to play with examples in compel :). Yesterday night I was looking at the flow but I think this makes more sense to make test examples run first. I will update you, once I try to run this examples with findings I get.
And would love to know if any other things I should be trying or reading organically with understanding to get this thing running on RISCV as soon as we can.
you are welcome :)
offtopic: Ah, I'm also thought about the RISC-V motherboard but it's sooo expensive (and mostly just not in stock) right now... :(
I would like to join the effort too, I have a sifive unmatched board for testing if it may help. I will try to build compel with modifications from aarch64.
I also quite newbie to system programming, but i will try to do my best.
Opened a MR for traceability and reviewing : https://github.com/checkpoint-restore/criu/pull/1713
@nirousseau Hey thanks for joining the effort. I had branch in my fork and had done some changes already. But i didnt open the PR, do u mind joining this one ?MR : https://github.com/checkpoint-restore/criu/pull/1714 As I already have all the changes, in your PR along with modifications ?
@mihalicyn Hello Alex,
So i was trying to run compel as you suggested before and did some changes.
While compiling am facing the below issue (Have attached detailed logs below)
After reading the log with my knowledge, one strange thing am noticing is, when function : fini_sigreturn is called as per the logs : https://github.com/rushi47/criu/blob/mod_criu/compel/plugins/std/infect.c#L84
It's supposed to go to ARCH specific : ARCH_RT_SIGRETURN definition, for this case from my assumption it should call this macro : https://github.com/rushi47/criu/blob/mod_criu/compel/arch/riscv64/src/lib/include/uapi/asm/sigframe.h#L36-L43
But from log it looks to call : ARCH_RT_SIGRETURN_NATIVE & ARCH_RT_SIGRETURN_COMPAT . After searching, i found that it only exists in x86 arch (https://github.com/rushi47/criu/blob/mod_criu/compel/arch/x86/src/lib/include/uapi/asm/sigframe.h#L180-L212).
So am not sure why, it is executing code related to x86 arch.
I am not sure if am doing anything wrong while compiling, your help will be much appreciated.
Below are the logs :
Note: Building without setproctitle() and strlcpy() support.
To enable these features, please install libbsd-devel (RPM) / libbsd-dev (DEB).
sh: 1: pkg-config: not found
sh: 1: pkg-config: not found
sh: 1: pkg-config: not found
Note: Building without GnuTLS support
sh: 1: pkg-config: not found
Makefile.config:45: Warn: you have no libnftables installed
Makefile.config:46: Warn: Building without nftables support
DEP compel/arch/riscv64/plugins/std/syscalls/syscalls.d
DEP compel/arch/riscv64/plugins/std/parasite-head.d
DEP compel/plugins/std/infect.d
DEP compel/plugins/std/string.d
DEP compel/plugins/std/log.d
DEP compel/plugins/std/fds.d
DEP compel/plugins/std/std.d
DEP compel/plugins/shmem/shmem.d
DEP compel/plugins/fds/fds.d
CC compel/plugins/std/std.o
CC compel/plugins/std/fds.o
CC compel/plugins/std/log.o
CC compel/plugins/std/string.o
CC compel/plugins/std/infect.o
In file included from compel/plugins/std/infect.c:14:
compel/plugins/std/infect.c: In function 'fini_sigreturn':
compel/include/uapi/compel/asm/sigframe.h:181:9: error: unknown register name 'rax' in 'asm'
181 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:209:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_NATIVE'
209 | ARCH_RT_SIGRETURN_NATIVE(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r11' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r10' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r9' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r8' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'eax' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
make[1]: *** [/root/criu/scripts/nmk/scripts/build.mk:215: compel/plugins/std/infect.o] Error 1
make: *** [Makefile.compel:56: compel/plugins/std.lib.a] Error 2
pkg-config: not found
@rushi47 you need to install pkg-config. All build dependencies are described in https://criu.org/Installation.
@rst0git Thank you for writing it out :) , i thought its unrelated but i did install it as you suggested. But still the same error :(
root@debian:~/criu# dpkg -l | grep 'pkg-config'
ii pkg-config 0.29.2-1 riscv64 manage compile and link flags for libraries
I am trying to figure out, why it's going on the same unrelated method.
root@debian:~/criu# make install-compel
Note: Building without setproctitle() and strlcpy() support.
To enable these features, please install libbsd-devel (RPM) / libbsd-dev (DEB).
Note: Building without GnuTLS support
Makefile.config:45: Warn: you have no libnftables installed
Makefile.config:46: Warn: Building without nftables support
DEP compel/arch/riscv64/plugins/std/syscalls/syscalls.d
DEP compel/arch/riscv64/plugins/std/parasite-head.d
DEP compel/plugins/std/infect.d
DEP compel/plugins/std/string.d
DEP compel/plugins/std/log.d
DEP compel/plugins/std/fds.d
DEP compel/plugins/std/std.d
DEP compel/plugins/shmem/shmem.d
DEP compel/plugins/fds/fds.d
CC compel/plugins/std/std.o
CC compel/plugins/std/fds.o
CC compel/plugins/std/log.o
CC compel/plugins/std/string.o
CC compel/plugins/std/infect.o
In file included from compel/plugins/std/infect.c:14:
compel/plugins/std/infect.c: In function 'fini_sigreturn':
compel/include/uapi/compel/asm/sigframe.h:181:9: error: unknown register name 'rax' in 'asm'
181 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:209:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_NATIVE'
209 | ARCH_RT_SIGRETURN_NATIVE(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r11' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r10' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r9' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'r8' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
compel/include/uapi/compel/asm/sigframe.h:190:9: error: unknown register name 'eax' in 'asm'
190 | asm volatile( \
| ^~~
compel/include/uapi/compel/asm/sigframe.h:211:17: note: in expansion of macro 'ARCH_RT_SIGRETURN_COMPAT'
211 | ARCH_RT_SIGRETURN_COMPAT(new_sp); \
| ^~~~~~~~~~~~~~~~~~~~~~~~
compel/plugins/std/infect.c:84:9: note: in expansion of macro 'ARCH_RT_SIGRETURN'
84 | ARCH_RT_SIGRETURN(new_sp, sigframe);
| ^~~~~~~~~~~~~~~~~
make[1]: *** [/root/criu/scripts/nmk/scripts/build.mk:215: compel/plugins/std/infect.o] Error 1
make: *** [Makefile.compel:56: compel/plugins/std.lib.a] Error 2
@rushi47 x86 and RISC-V use different assembly registers. You can find more information about the RISC-V registers in the specifications.
@rst0git yes, that's correct. My doubt is I have copied code of aarch64 as riscv - in compel/arch & I am try to compile it for RISCV, from my understanding ideally it should go to riscv and should break there, am wondering why it's going to ARCH_RT_SIGRETURN_ COMPAT & NATIVE as this definition only exists in x86 arch like i pointed before, when am compiling for RISCV. Please correct me if am wrong with this.
I think am getting there, it hit the right spot now.
root@debian:~/criu# make install-compel
Note: Building without setproctitle() and strlcpy() support.
To enable these features, please install libbsd-devel (RPM) / libbsd-dev (DEB).
Note: Building without GnuTLS support
Makefile.config:45: Warn: you have no libnftables installed
Makefile.config:46: Warn: Building without nftables support
GEN compel/include/asm
GEN compel/include/version.h
touch .config
GEN include/common/config.h
GEN include/common/asm
GEN compel/plugins/include/uapi/std/asm/syscall-types.h
GEN compel/arch/riscv64/plugins/std/syscalls/syscalls.S
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_TERMINAL = "iTerm2",
LC_NUMERIC = "C",
LC_COLLATE = "C",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
DEP compel/arch/riscv64/plugins/std/syscalls/syscalls.d
DEP compel/arch/riscv64/plugins/std/parasite-head.d
DEP compel/plugins/std/infect.d
DEP compel/plugins/std/string.d
DEP compel/plugins/std/log.d
DEP compel/plugins/std/fds.d
DEP compel/plugins/std/std.d
DEP compel/plugins/shmem/shmem.d
DEP compel/plugins/fds/fds.d
CC compel/plugins/std/std.o
CC compel/plugins/std/fds.o
CC compel/plugins/std/log.o
CC compel/plugins/std/string.o
CC compel/plugins/std/infect.o
In file included from compel/include/uapi/compel/asm/sigframe.h:4,
from compel/plugins/std/infect.c:14:
/usr/include/riscv64-linux-gnu/asm/sigcontext.h:17:8: error: redefinition of 'struct sigcontext'
17 | struct sigcontext {
| ^~~~~~~~~~
In file included from /usr/include/signal.h:288,
from compel/include/uapi/compel/plugins/std/syscall-types.h:12,
from compel/include/uapi/compel/plugins/std/syscall.h:4,
from compel/include/uapi/compel/plugins/std.h:5,
from compel/plugins/std/infect.c:1:
/usr/include/riscv64-linux-gnu/bits/sigcontext.h:25:8: note: originally defined here
25 | struct sigcontext {
| ^~~~~~~~~~
In file included from compel/plugins/std/infect.c:14:
compel/include/uapi/compel/asm/sigframe.h:16:31: error: field 'fpsimd' has incomplete type
16 | struct fpsimd_context fpsimd;
| ^~~~~~
compel/include/uapi/compel/asm/sigframe.h:18:29: error: field 'end' has incomplete type
18 | struct _aarch64_ctx end;
| ^~~
In file included from compel/plugins/std/infect.c:14:
compel/plugins/std/infect.c: In function 'fini':
compel/include/uapi/compel/asm/sigframe.h:59:95: error: 'mcontext_t' has no member named 'pc'
59 | RT_SIGFRAME_REGIP(rt_sigframe) ((long unsigned int)(rt_sigframe)->uc.uc_mcontext.pc)
| ^
compel/plugins/std/infect.c:10:53: note: in expansion of macro 'RT_SIGFRAME_REGIP'
10 | #define pr_debug(fmt, ...) print_on_level(4, fmt, ##__VA_ARGS__)
| ^~~~~~~~~~~
compel/plugins/std/infect.c:94:9: note: in expansion of macro 'pr_debug'
94 | pr_debug("%ld: new_sp=%lx ip %lx\n", sys_gettid(), new_sp, RT_SIGFRAME_REGIP(sigframe));
| ^~~~~~~~
make[1]: *** [/root/criu/scripts/nmk/scripts/build.mk:215: compel/plugins/std/infect.o] Error 1
make: *** [Makefile.compel:56: compel/plugins/std.lib.a] Error 2
[Update] Reached till parasite code, trying to figure it out -
root@debian:~/criu# make install-compel
Note: Building without setproctitle() and strlcpy() support.
To enable these features, please install libbsd-devel (RPM) / libbsd-dev (DEB).
Note: Building without GnuTLS support
Makefile.config:45: Warn: you have no libnftables installed
Makefile.config:46: Warn: Building without nftables support
DEP compel/arch/riscv64/plugins/std/syscalls/syscalls.d
DEP compel/arch/riscv64/plugins/std/parasite-head.d
DEP compel/plugins/std/infect.d
DEP compel/plugins/std/string.d
DEP compel/plugins/std/log.d
DEP compel/plugins/std/fds.d
DEP compel/plugins/std/std.d
DEP compel/plugins/shmem/shmem.d
DEP compel/plugins/fds/fds.d
CC compel/plugins/std/std.o
CC compel/plugins/std/fds.o
CC compel/plugins/std/log.o
CC compel/plugins/std/string.o
CC compel/plugins/std/infect.o
CC compel/arch/riscv64/plugins/std/parasite-head.o
compel/arch/riscv64/plugins/std/parasite-head.S: Assembler messages:
compel/arch/riscv64/plugins/std/parasite-head.S:4: Error: unrecognized symbol type ""
compel/arch/riscv64/plugins/std/parasite-head.S:5: Error: unrecognized opcode `bl parasite_service'
compel/arch/riscv64/plugins/std/parasite-head.S:6: Error: unrecognized opcode `brk '
/tmp/ccJSDRmN.s: Error: .size expression for __export_parasite_head_start does not evaluate to a constant
@mihalicyn hey alex will you be able to help me, with assembly for parasite code injection. I am bit confused with, how it's done for riscv
A friendly reminder that this issue had no activity for 30 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
I think am able to compile compel part it looks to be successful [i think at least syntactically]. @mihalicyn @rst0git Will you be able to help me test ? https://criu.org/Compel I am trying to follow the instructions from here but couldnt follow much
@adrianreber @avagin will you be able to guys take a look ? I am trying to test it, ld is giving me some issue :
root@debian:~/criu/compel/test/fdspy# ld -r -z noexecstack -T ../../../compel/arch/riscv64/scripts/compel-pack.lds.S -o parasite.po parasite.o ../../../compel/plugins/std.lib.a ../../../compel/plugins/fds.lib.a
ld: cannot represent machine `riscv64'
Hi @rushi47
feel free to reach me on the Gitter [https://gitter.im/save-restore/CRIU]
-
Are you trying to run
ldfrom the RISC-V machine or are you trying to do cross-compilation? Please show you changes in compel.
@mihalicyn hey alex will you be able to help me, with assembly for parasite code injection. I am bit confused with, how it's done for riscv
if it's actual - ping me. I will try to help you.
@mihalicyn I am running it on RISC-V machine like native code. I was able to fix the linker issue by this commit : https://github.com/rushi47/criu/commit/95ba9153977fbe07a4f76936346fd9b4143702aa
Breaking here now, when am trying to run test: https://github.com/rushi47/criu/blob/mod_criu/compel/src/lib/handle-elf.c#L640
Ahh ok let me join jitter, looks like on jitter people are more responsive
@rushi47
I was able to fix the linker issue by this commit :
yep, this change to the linker script looks correct.
Breaking here now, when am trying to run test:
Yep, you need to support relocations for RISC-V. First of all, I suggest listing all relocation types that you've met here.
You can change goto err to break and collect all relocation types that the compiler uses in the final PIE. Then we will think about how to handle it properly.
- enumerate all relocation types
- assign proper constants for it like it done for other architectures
R_RISCV_JAL,R_RISCV_ADD32, etc it will be really useful to take a look on https://github.com/torvalds/linux/blob/5bfc75d92efd494db37f5c4c173d3639d4772966/arch/riscv/kernel/module.c#L312 - provide an implementation for needed relocation types. We can get some inspiration from the kernel. Because when the Linux kernel loads the module into the memory it also applies relocations (because LKM is also relocatable objects).
We've private text discussion with @rushi47 about relocation types and how to handle them.
I think at first we can go with a "naive way":
#ifdef ELF_RISCV
case R_RISCV_BRANCH:
ptrdiff_t offset = value64 + addend64 - place;
u32 imm12 = (offset & 0x1000) << (31 - 12);
u32 imm11 = (offset & 0x800) >> (11 - 7);
u32 imm10_5 = (offset & 0x7e0) << (30 - 10);
u32 imm4_1 = (offset & 0x1e) << (11 - 4);
*((int32_t *)where) = (*((int32_t *)where) & 0x1fff07f) | imm12 | imm11 | imm10_5 | imm4_1;
break;
case R_RISCV_JAL:
ptrdiff_t offset = value64 + addend64 - place;
u32 imm20 = (offset & 0x100000) << (31 - 20);
u32 imm19_12 = (offset & 0xff000);
u32 imm11 = (offset & 0x800) << (20 - 11);
u32 imm10_1 = (offset & 0x7fe) << (30 - 10);
*((int32_t *)where) = (*((int32_t *)where) & 0xfff) | imm20 | imm19_12 | imm11 | imm10_1;
break;
case R_RISCV_CALL_PLT:
ptrdiff_t offset = value64 + addend64 - place;
s32 fill_v = offset;
u32 hi20, lo12;
if (offset != fill_v) {
pr_err("Unsupported relocation of type R_RISCV_CALL_PLT with offset != fill_v\n");
goto err;
}
hi20 = (offset + 0x800) & 0xfffff000;
lo12 = (offset - hi20) & 0xfff;
*((int32_t *)where) = (*((int32_t *)where) & 0xfff) | hi20;
*((int32_t *)(where+4)) = (*((int32_t *)(where+4)) & 0xfffff) | (lo12 << 20);
break;
#endif
This code is not tested because I've no RISC-V machine yet. So @rushi47, please try to play with that. It's a "naive" translation of the kernel implementation for these relocation types.
@mihalicyn thanks ton for this. As per private conversation I tested this, so compel is compiling again fine but its still throwing error I think need to add more relocation types working on it
On other hand, this is docker image can be used as emulator to test RISCV: https://github.com/DavidBurela/riscv-emulator-docker-image
Hi @rushi47!
how things are going with RISC-V? I have some plan to play with that soon, as far as I'll get access to the real RISC-V device to test on it.
Please let me know if you have a serious plans to work on it.
Hi @rushi47!
how things are going with RISC-V? I have some plan to play with that soon, as far as I'll get access to the real RISC-V device to test on it.
Please let me know if you have a serious plans to work on it.
Hey,
- So compel was compiling fine syntactically, I would say. And I think majority of assembly code was in compel.
I had pause project for some time. I am quite keen about it & more than happy to work on it. So if you are starting working on it, I am more than happy to resume again. If someone can mentor me actively, I am more than happy to dedicatedly work for some hours on it daily.
Hello everyone,
I have been working on RISC-V recently, and was wondering if we could really advance on this feature. I would be happy to provide any RISC-V based simulations and testing.
Yep, you need to support relocations for RISC-V. First of all, I suggest listing all relocation types that you've met here. You can change
goto errtobreakand collect all relocation types that the compiler uses in the final PIE. Then we will think about how to handle it properly.
- enumerate all relocation types
- assign proper constants for it like it done for other architectures
R_RISCV_JAL,R_RISCV_ADD32, etc it will be really useful to take a look on https://github.com/torvalds/linux/blob/5bfc75d92efd494db37f5c4c173d3639d4772966/arch/riscv/kernel/module.c#L312- provide an implementation for needed relocation types. We can get some inspiration from the kernel. Because when the Linux kernel loads the module into the memory it also applies relocations (because LKM is also relocatable objects).
@mihalicyn I have followed this suggestion and added more relocation types I encountered (doing a break for now per your suggestion). By reusing and modifying @rushi47's previous code, I'm able to compile compel and run test in compel/test/infect. I'm working with cross compiling environment (X86 -> RISC-V) and have a RISC-V Qemu to test it. This is the result after running ./spy in compel/test/infect. The errors are expected since I haven't added all the RISC-V support yet, but wanted to see if we have an environment to test compel first. Looks like we do now! :) This runtime errors can give us a lot of insights to move forward. I'll keep working on it!
If other people are interested in this, @mihalicyn @rushi47 and I also have a group chat on Gitter to discuss the details. It's quite active :)
# ./spy
Checking the victim alive
1, want 1
42, want 42
Infecting the victim
Stopping task
Preparing parasite ctl
LC4: Preparing seqsk for 219
Configuring contexts
Infecting
LC4: ** delivering signal 4 si_code=1
LC1: Error (compel/src/lib/infect.c:609): Unexpected 219 task interruptiog
[ 377.932091] spy[218]: unhandled signal 11 code 0x1 at 0x0000000000000000 in li]
[ 377.933221] CPU: 5 PID: 218 Comm: spy Not tainted 5.15.0 #1
[ 377.933752] Hardware name: XXXXXXXXXXXXX
[ 377.934211] epc : 0000003fbb153344 ra : 0000002ae5bab716 sp : 0000003fde5d2a70
[ 377.934814] gp : 0000002ae5bb1800 tp : 0000003fbb2073c0 t0 : 0000000000000000
[ 377.935440] t1 : 0000002ae5ba857c t2 : 00000000000a66e6 s0 : 0000002aec7658a0
[ 377.936082] s1 : 0000000000006480 a0 : 0000000000000000 a1 : 0000002ae5bac3b0
[ 377.936698] a2 : 0000000000001dc0 a3 : 0000000000000200 a4 : 0000000000000000
LC3: Putting parasite blob into (nil)->(nil)
[ 377.939224] a5 : 0000000000000000 a6 : fefefefefefefeff a7 : 0000000000000040
[ 377.939967] s2 : 0000000000006480 s3 : 0000000000000001 s4 : 0000000000001fff
[ 377.940817] s5 : 0000000000002000 s6 : ffffffffffffffff s7 : 0000000000000007
[ 377.941479] s8 : ffffffffffffffda s9 : 00000000000000db s10: 0000003fde5d2a80
[ 377.942121] s11: 0000000000000001 t3 : 0000003fbb153334 t4 : 0000000000000000
[ 377.942803] t5 : 000000000000002a t6 : 0000003fbb1dcc40
[ 377.943292] status: 8000000200004620 badaddr: 0000000000000000 cause: 00000000f
Segmentation fault
Hello everyone, I have been working on RISC-V recently, and was wondering if we could really advance on this feature. I would be happy to provide any RISC-V based simulations and testing.
Hello @thesaadmemon We're actively working on this feature now. What simulation and testing environment are you referring to? Can you give us some more details? Thank you very much! :)
@felicitia Hello Yixue,I have two VisionFive2(4c8g) board and am currently building the riscv64 version of Proxmox VE.
@felicitia Hello Yixue,I have two VisionFive2(4c8g) board and am currently building the riscv64 version of Proxmox VE.
That's great @jiangcuo ! I actually found a way to test CRIU on riscv64 QEMU and wrote the instruction since simulation is a lot easier to get started. CRIU needs to be cross compiled for riscv64 target in this way and I included the instructions for cross-compiling too if you're interested :) @mihalicyn and I are actively working on the riscv64 support for CRIU! 💪