dynamorio

dynamorio copied to clipboard

Hi , as we all know , Risc-V popularity is rising more in time. Will Any support be avalible for Risc-V ?

There is already new soc released for risc-v

https://www.sifive.com/boards

Hi, we don't currently have plans for RISC-V support. However, we do support AArch32 and AArch64 which can be used as reference points for a RISC-V port: https://github.com/DynamoRIO/dynamorio/wiki/ARM-Port https://github.com/DynamoRIO/dynamorio/wiki/AArch64-Port

We are a small developer team but welcome and will support anyone or any group who want to add features such as RISC-V. Do you have specific use-cases in mind, i.e. what sort of problems would you hope to address with DynamoRIO if RISC-V was available?

As an aside, I would suggest posting your question on the users list https://groups.google.com/forum/#!forum/DynamoRIO-User

This will be seen by more people than this issues list and we'll get an idea of how many are interested in RISC-V support.

Hi,

Just FYI that the PLCT Lab is planning to port the DynamoRIO to RISC-V (RV64G)[1]. Not started yet, but we are optimistic that we are able to finish the porting project and contributing back to upstream before the end of 2021.

[1] https://github.com/plctlab/plctlab.github.io/blob/main/PLCT-Roadmap-2021.md#enable-dynamorio-running-on-rv64gc

That's exciting news!

I have a few suggestions:

• Please consider a workflow where code changes are sent for review early and often, keeping reviews small and incremental. This makes the review process much more manageable. (What we would like to avoid is building up a very large parallel forked code base with no review or merge until it is too large to practically review.) Initially, existing project members would perform reviews, but once PLCT developers are up to speed they can be added as project members and perform reviews.
• Naturally we would like to have automated testing of RISC-V added to the Continuous Integration suite: https://github.com/DynamoRIO/dynamorio/wiki/Test-Suite#automated-test-machines. This could be via Jenkins on a machine hosted somewhere else, or through a Github Actions-provided VM if that can be configured properly.

Hi @derekbruening

Thanks for the suggestions. We will send patch set for early review/comments once we got a basic framework. The process of porting is also open (on github, as a fork of the repo).

Naturally we would like to have automated testing of RISC-V added to the Continuous Integration suite: https://github.com/DynamoRIO/dynamorio/wiki/Test-Suite#automated-test-machines. This could be via Jenkins on a machine hosted somewhere else, or through a Github Actions-provided VM if that can be configured properly.

Thanks. I'll check it out. I've built some CI infrastructure before, so I am optimistic that automated testing of DynamoRIO might not be hard to me:)

Hi @lazyparser

Can I know your current progress? I am currently preparing to implement RISC-V's DBI tool base on dynamorio, I hope to communicate with you more in the future. I am currently a graduate student at iscas.

Thanks, Onezibo

Hi @lazyparser

Can I know your current progress? I am currently preparing to implement RISC-V's DBI tool base on dynamorio, I hope to communicate with you more in the future. I am currently a graduate student at iscas.

Thanks, Onezibo

Hi @onezibo

Glad to hear you are working on this. I had not start to coding yet. The plan is to finish the porting/support before the end of year 2021, and get started after April.

Feel free to drop me an email. :-)

The process of porting is also open (on github, as a fork of the repo).

Somewhat echoing the earlier suggestions, I would suggest not having a forked repo being the official port: it will diverge and we'll end up with a fractured state where RISC-V is only available with a stale and outdated rest of the system. A forked repo results in a tendency to commit only to that repo, to build up large differences which are impractical to review and merge, and combined with ongoing evolution of the other repo eventually results in permanent divergence. IMHO it is better to build new features directly in the community's existing repo from the beginning; it is always more work to try to merge later.

The process of porting is also open (on github, as a fork of the repo).

Somewhat echoing the earlier suggestions, I would suggest not having a forked repo being the official port: it will diverge and we'll end up with a fractured state where RISC-V is only available with a stale and outdated rest of the system. A forked repo results in a tendency to commit only to that repo, to build up large differences which are impractical to review and merge, and combined with ongoing evolution of the other repo eventually results in permanent divergence. IMHO it is better to build new features directly in the community's existing repo from the beginning; it is always more work to try to merge later.

@derekbruening I totally agree with you. The fork under the github/plctlab account is only used for sending pull requests to upstream. My long term goal is to make the RISC-V as a tier-1 platform support in upstream.

@derekbruening I totally agree with you. The fork under the github/plctlab account is only used for sending pull requests to upstream. My long term goal is to make the RISC-V as a tier-1 platform support in upstream.

+1, SGTM!

Hi, I started porting DynamoRIO to RISC-V and it's just starting. At present, I directly use the relevant code of AArch64 to make a general framework, and then I will solve the ISA-related problems first, and make the overall RISC-V framework clearer and easier for more developers to participate.

All work will be displayed under the default branch of this repository: https://github.com/bekcpear/dynamorio

I will synchronize the follow-up progress here in time, and I also very much hope that you can give me some suggestions on porting or codes.

Hi everyone. What a lovely coincidence as I have recently started working on DynamoRIO on RISC-V as well. @bekcpear it would be awesome if we could synchronize our work and collaborate.

So far I have a codebase that can be fully compiled and linked (both cross and native) which contains mostly non-functional stubs and some trivial bits (some possibly wrong). The biggest missing piece is the instruction encoder/decoder, which is next on my TODO list.

The code can be found in experimental-riscv branch of the following repo: https://github.com/Semihalf/dynamorio. This will also be my main development branch so You can expect updates.

At this point I would appreciate any feedback on the porting approach I've taken (which is also heavily based on the Aarch64) port. I think we shuold start discussing on a possible changes to architecture specific code separation as the current #ifdef may lead to unreadable code.

Additionally there are two commits where I've tried to separate specific questions that I have:

1. 7992b2e7 RFC: Use semantic register names to simplify code - Postulate to designate semantic register names to simplify IF_<arch>_ELSE(...) ladders which will complicate code layout once a new architecture is added.
2. 38e3e239 RFC: Add comments for upstream discussion - Several questions on implementation details such as flags register assumptions, use of semantic register names or streamlining some debug tracing.

Please see our suggestions and requests above, especially https://github.com/DynamoRIO/dynamorio/issues/3544#issuecomment-822003110 and https://github.com/DynamoRIO/dynamorio/issues/3544#issuecomment-766147223: we do not recommend separate forks.

Building up code in separate repositories makes it much harder to merge later. It happens all too often that it is so much work to merge a huge amount of code that was developed separately that it never happens. I am afraid we will end up with multiple conflicting ports that will all drift from the main repository and never be merged...it is much less work in the long run to work in the main repository from the start and most importantly send small incremental code reviews early and often.

+1 to Derek's suggestion. Additionally, forked repositories do not automatically get the bug fixes and features that we regularly add to the main repository, which accumulate over time. It is also easier to report/debug issues and get feedback on new code if it is merged with the main repository.

Echoing https://github.com/DynamoRIO/dynamorio/issues/3544#issuecomment-766147223, new developers can be provided more privileges when they are up to speed, so that they can help perform code reviews for their groups. We also have a Triager rotation in place to provide better support to our community, for code reviews and dynamorio-users list discussions.

Hi, all. I'm glad to receive your attention. I really want to merge RISC-V support into this repository, not separate development in my forked repo (the task of my fork is to make PR). I'm still familiar with the overall structure, and I will submit an appropriate PR as soon as possible.

I am also very much looking forward to collaborating with you ;) @semihalf-kardach-stanislaw. If you get some work done, hopefully it can be updated instantly (new commits on your fork or make a new PR to this repo) to avoid duplication, so do I.

I think there I've caused some confusion by being brief. The reason I've posted my code is to start the integration process as soon as possible (hence the fork as a pull request source).

I haven't posted anything earlier because achieving full compilation would help finding places that need to be stubbed/implemented and at the same time it would provide some semblance of validation (even though not a runtime one) that what I'm doing makes sense. @derekbruening Do you consider such pull request too big and therefore I should split it?

Second reason is that I want to keep my development process open and visible, especially in-between pull requests. This is so that I can discuss with You whether my understanding of a given module is proper or not, all without spamming RFC pull requests. In essence I want to shorten the feedback loop and allow a broader community to chime in.

So my current plan is the following:

1. Sync with @bekcpear to prepare the initial pull request content.
2. Start a discussion on dynamorio-users about potential re-factoring of the code to better accommodate 3 concurrent architectures.
3. Keep working on adding new features incrementally.
4. Figure out how to add RISC-V to CI - Qemu or physical U740 machines, how to connect them securely, etc.

Please let me know if that makes sense to everyone.

Figure out how to add RISC-V to CI - Qemu or physical U740 machines, how to connect them securely, etc.

I'd like to contribute here: either Cloud VM (amd64, hosting QEMU/RISC-V) or physical Unmatched are available, provided by the PLCT Lab. Collaborating with RISC-V International, the PLCT Lab, ISCAS has built a large RISC-V Lab for open source communities.

Feel free to send a PR to https://github.com/plctlab/riscv-lab-access , appending your github-id to the developers.list (the system will grab your pubkeys from github).

I haven't posted anything earlier because achieving full compilation would help finding places that need to be stubbed/implemented and at the same time it would provide some semblance of validation (even though not a runtime one) that what I'm doing makes sense.

This does seem reasonable, getting far enough to know the approach is going to work and to find the scope of what needs to be done.

@derekbruening Do you consider such pull request too big and therefore I should split it?

It is large. We normally suggest <1500 lines (https://dynamorio.org/page_code_reviews.html#autotoc_md110). Looks like:

https://github.com/DynamoRIO/dynamorio/compare/master...Semihalf:dynamorio:experimental-riscv Showing 96 changed files with 5,227 additions and 153 deletions.

If most changes are simple a few thousand might be ok -- but smaller pieces are always easier to find time to review. I would suggest breaking it into at least 2 pieces, even if there's no clear logical split, just to have manageable sizes for people to look at in one sitting.

Figure out how to add RISC-V to CI - Qemu or physical U740 machines, how to connect them securely, etc.

I'd like to contribute here: either Cloud VM (amd64, hosting QEMU/RISC-V) or physical Unmatched are available, provided by the PLCT Lab. Collaborating with RISC-V International, the PLCT Lab, ISCAS has built a large RISC-V Lab for open source communities.

Feel free to send a PR to https://github.com/plctlab/riscv-lab-access , appending your github-id to the developers.list (the system will grab your pubkeys from github).

Thank you for the offer! For connecting to Github, Jenkins may be a good fit: we're currently using that to run the test suite on an AArch64 machine for pull requests.

Hi everyone. What a lovely coincidence as I have recently started working on DynamoRIO on RISC-V as well. @bekcpear it would be awesome if we could synchronize our work and collaborate.

So far I have a codebase that can be fully compiled and linked (both cross and native) which contains mostly non-functional stubs and some trivial bits (some possibly wrong). The biggest missing piece is the instruction encoder/decoder, which is next on my TODO list.

The code can be found in experimental-riscv branch of the following repo: https://github.com/Semihalf/dynamorio. This will also be my main development branch so You can expect updates.

At this point I would appreciate any feedback on the porting approach I've taken (which is also heavily based on the Aarch64) port. I think we shuold start discussing on a possible changes to architecture specific code separation as the current #ifdef may lead to unreadable code.

Additionally there are two commits where I've tried to separate specific questions that I have:

1. [7992b2e7 RFC: Use semantic register names to simplify code](https://github.com/Semihalf/dynamorio/commit/7992b2e7d4ac84f0d769ce87a5f8a58c0c01ef59) - Postulate to designate semantic register names to simplify `IF_<arch>_ELSE(...)` ladders which will complicate code layout once a new architecture is added.

2. [38e3e239 RFC: Add comments for upstream discussion](https://github.com/Semihalf/dynamorio/commit/38e3e239f60dfa8d8a94318cc6221dd97fb15d4b) - Several questions on implementation details such as flags register assumptions, use of semantic register names or streamlining some debug tracing.

Amazing news! I have started a similar process to add support for RISC-V for DynamoRIO, but it makes a lot more sense to join forces and not duplicate effort! I am excited that more people are looking into this, and look forward to participate in the effort! On my side, I have access to some RISC-V boards, which unfortunately can't make accessible remotely. But I can certainly run and test things.

Thank you for the offer! For connecting to Github, Jenkins may be a good fit: we're currently using that to run the test suite on an AArch64 machine for pull requests.

Is there any examples or docs (URL) for this steps? the machines are ready, and hopefully I have enough time to deploy the CI this weekend.

Thank you for the offer! For connecting to Github, Jenkins may be a good fit: we're currently using that to run the test suite on an AArch64 machine for pull requests.

Is there any examples or docs (URL) for this steps? the machines are ready, and hopefully I have enough time to deploy the CI this weekend.

@AssadHashmi set up the AArch64 Jenkins: Assad, were there some docs that you followed?

@AssadHashmi set up the AArch64 Jenkins: Assad, were there some docs that you followed?

I don't have any formal documentation but this is what I did, in summary:

1. Installed Jenkins as described here https://www.jenkins.io/doc/book/installing/linux/
2. Installed 'GitHub Pull Request Builder' plugin, see https://plugins.jenkins.io/ghprb/
3. Created a pipeline on Jenkins server and made it run our pre-commit tests. When creating 'New item', (i.e. job) I used 'Freestyle project' not 'Pipeline'.
4. Created credentials for GitHub user drbot0.
5. Tested connection from Jenkins to GitHub using the 'Test Credentials' feature of the 'GitHub Pull Request Builder' section in Jenkins system configuration. This uses the existing drbot0 user and its Personal Access Token (PAT).
6. Configured GitHub to trigger our Jenkins tests by adding a Webhook URL pointing to our Jenkins 'GitHub Pull Request Builder' (ghprbhook), see https://github.com/DynamoRIO/dynamorio/settings/hooks
7. Tested manually from a GitHub PR using a comment phrase which can be configured in Jenkins instance. We use run arm tests. Got someone else to do the same making sure that person's user name was in the whitelist of our Jenkins pre-commit pipleine.

@shiptux @dolanzhao @derekbruening @semihalf-kardach-stanislaw

Hi! What is the current state of the RISC-V support? I've tried to build it under the Qemu for riscv64, but drrun failed on the simpliest hello world program. The debug version printed this: ERROR: unable to inject: exec of |(null)| failed

@shiptux @dolanzhao @derekbruening @semihalf-kardach-stanislaw

Hi! What is the current state of the RISC-V support? I've tried to build it under the Qemu for riscv64, but drrun failed on the simpliest hello world program. The debug version printed this: ERROR: unable to inject: exec of |(null)| failed

Hi thank you for your attention. The current status of DynamoRIO RISC-V port is stage 1 (could be compiled but couldn't run). Many instruction encoder are not-implemented. This also the reason why the simplest hello world program failed.

For your question, we will need instruction encoder to get it execute. I made the DynamoRIO RISC-V units test compile by disabled them all last month, and then we should start the most basic instructions support.

Thanks

Got it

Thanks!

There is a race in the RISC-V build config: #6140

Hi, If anyone is interested, I have successfully run a simple hello world program using drrun on a RISC-V SBC.

debian@lpi4a:/media/debian/dynamorio/build$ cat hello.S
.global _start
_start: li      a0, 1
        la      a1, helloworld
        li      a2, 14
        li      a7, 64
        ecall

        li      a0, 0
        li      a7, 93
        ecall

.data
helloworld:     .ascii "Hello, World!\n"

debian@lpi4a:/media/debian/dynamorio/build$ ./hello
Hello, World!
debian@lpi4a:/media/debian/dynamorio/build$ bin64/drrun hello
<Starting application /media/debian/dynamorio/build/hello (36030)>
<Initial options = -no_dynamic_options -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct >
<Paste into GDB to debug DynamoRIO clients:
set confirm off
add-symbol-file '/media/debian/dynamorio/build/lib64/debug/libdynamorio.so' 0x0000003fee3c70f0
>
Hello, World!
<Stopping application /media/debian/dynamorio/build/hello (36030)>
debian@lpi4a:/media/debian/dynamorio/build$

Next, I will split this work into smaller patches and submit them upstream.

Update the newest progress on DynamoRIO RISCV64:

Single-threaded Linux common commands can be run using drrun!

Also, before merged into upstream (which takes time), you can check out the progress at: https://github.com/ruyisdk/dynamorio