Add JShell Support for playing around with the API

[5c4lar]

Describe the solution you'd like Add a Window running JShell that can interact with Ghidra just like Jython. This may ease the development of new scripts written in Java.

Additional context JShell (Java REPL) is available for Java 9+.

[ryanmkurtz]

I attempted to do this a couple of years ago. The interpreter piece worked fine, but I could not find a way to get it to integrate with the Ghidra API like Jython does...it seemed to be running in a different JVM. I didn't have time to dig much deeper so I ultimately scrapped it.

[5c4lar]

For Kotlin, there is a repo ghidra-jupyter-kotlin, where the Ghdira API can be accessed with Kotlin. Maybe that is a workaround, or the Java version can be implemented the same way?

[ryanmkurtz]

I was hoping to have a more integrated (same-process) solution. I took a quick look at that plugin, and if I understand it correctly, it looks like it's doing interprocess communication over a socket. This can be a security risk when the capability you are enabling is executing arbitrary java commands.

[astrelsky]

What about cling?

Edit: This was a joke, it would be awful. Please don't.

[dev747368]

I did a proof-of-concept several years ago that is in no state to be shared or even 'thrown-over-the-wall'.

Ryan is partially right in that in some modes the jshell api will execute the statements in a separate JVM, but that is under the control of the caller of the jshell api. (I even think I showed it to Ryan way-back-then and he just forgot.)

I wasn't able to overcome the syntactic separation of the jshell source code parsing with the expected GhidraScript development comforts, but if I remember correctly I injected an empty GhidraScript instance called "api" into the jshell execution environment that the user could reference like "api.askString(....);"

JShell is great (I use it all the time), it would be good to see this implemented.

[fmagin]

(Context: I'm one of the authors of the ghidra-jupyter-kotlin plugin)

@ryanmkurtz Not sure how familiar you are with the general idea of Jupyter, but that kind of inter process communication is standard in that ecosystem. That is the main appeal of the Jupyter kernel embedding because it allows using the various interfaces (jupyter-qtconsole, jupyter-console, jupyter-notebook, jupyter-lab). The actual connection to the kernel (which allows running arbitrary code), requires the data from a connection file which looks like this:

{
  "control_port": 44305,
  "shell_port": 37641,
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "stdin_port": 38681,
  "hb_port": 45645,
  "ip": "127.0.0.1",
  "iopub_port": 48771,
  "key": "1bba3c7d-29e2-4848-9f4f-d76a618533e4"
}

i.e. there is a pre-shared key generated by the kernel, which is needed to connect to it. I guess in theory this could be setup in a way that this information never touches the disk and is instead just directly passed to the console client.
There is also a Java Jupyter Kernel https://github.com/SpencerPark/IJava which can probably be made to work in the same way. The only tricky part for me with the Kotlin Kernel was to change the underlying kernel project to support embedding (i.e. that it inherits the Ghidra classloader and classpath). The large amount of work needed is to implement the actual kernel for the language, but this was already taken care of by https://github.com/Kotlin/kotlin-jupyter in my case, and would be handled by IJava in case of Java. One of the initial considerations for my ghidra-jupyter-kotlin plugin was to support a variety of languages as kernels, like IJava, or almond (Scala), but I consider Kotlin to be a near ideal language for scripting in a JVM environment because it's a nice mix of Java and Python. So I don't have serious future plans to integrate other languages.

Concerning the original topic of this issue: @goatshriek has JShell Support for their plugin on their roadmap, maybe they can tell us more about the kind of issues with it.

[goatshriek]

I didn't realize anyone was waiting on that particular feature on the roadmap, sorry about that! I started implementing it upon seeing this conversation. It shouldn't take more than a few weeks if all goes well, I have a working Proof of Concept after just a few hours, interested folks can follow along on the jshell branch.

It does run in the same JVM instance, and I am able to inject variables for things like currentAddress and currentProgram, which is a necessity for this to be useful. I haven't done that just yet (just using some String instances I pass around instead), but I'm confident that the experimenting I've done so far shows that is possible. For the GhidraScript functions, I will probably have to inject an instance as dev747368 referenced above; I've had to do similar things for other languages.

At the end I should have something that could serve as a reference for a feature directly within Ghidra, for what that's worth. My test framework is a little different from Ghidra's (I don't do any integration testing, just unit tests) so there would certainly be extra work to be done there. But the core interpreter code and likely the documentation as well could probably be a pretty direct transfer.

[goatshriek]

I've created a release candidate with a functioning JShell interpreter. I encourage anyone interested to install it and kick the tires. Non-abusive feedback is welcome, either here or in the RubyDragon repo. Here's a little taster:

I'll flip this to a final release in a few weeks, incorporating any feedback. I also have some refactoring to do as well, though it shouldn't change any visible functionality.

There isn't currently a GhidraScript instance provided in the interpreter automatically. I haven't done this in any other language interpreters, as I see scripts and interactive sessions as independent things. If there is a strong desire for it, I will add a script variable as suggested (and probably to my other interpreters too).

[astrelsky]

There isn't currently a GhidraScript instance provided in the interpreter automatically. I haven't done this in any other language interpreters, as I see scripts and interactive sessions as independent things. If there is a strong desire for it, I will add a script variable as suggested (and probably to my other interpreters too).

Oh this looks nice. From the use of currentAddress and currentProgram I assume that there is an underlying FlatProgramAPI instance even if it may not be a GhidraScript instance? Unless you have only provided the currentXYZ values. It is unclear which.

[goatshriek]

Currently I am explicitly providing those variables only. I hadn't considered an instance of the flat api though, and that would make a lot more sense to me to provide automatically than a script instance.

I'll make that change in a few days, worst case scenario as an api variable or something similar. I'm not confident that I can get the interpreter to run in the context of the instance itself, though that would probably feel more natural.

Edit: this has been done now in the JShell interpreter as currentAPI. @astrelsky, I will get to work on that C++ REPL right away.