Enigma
Enigma copied to clipboard
FabricMC
fix NoSuchFileException when exporting sources (not jar)
error example: java.nio.file.NoSuchFileException: /Users/taewonkim/Develop/Java/Minecraft/net/minecraft/client/gui/chat/ChatListener.java at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218) at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:484) at java.base/java.nio.file.Files.newOutputStream(Files.java:228) at java.base/java.nio.file.Files.newBufferedWriter(Files.java:3008) at java.base/java.nio.file.Files.newBufferedWriter(Files.java:3056) at cuchaz.enigma.EnigmaProject$ClassSource.writeTo(EnigmaProject.java:347) at cuchaz.enigma.gui.GuiController.lambda$exportSource$6(GuiController.java:269) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Thanks for accepting the suggestion! I think it's worth adding that this fix was necessary for me to successfully export 1.19.3 sources using the CLI. Without it, exporting doesn't work at all, as explained in the linked issue.
Here's how to patch the jar release in case you need it working now:
- Download the file from the PR: https://github.com/FabricMC/Enigma/blob/0f1e72926a17c561ace2936c091c0c195424744c/enigma/src/main/java/cuchaz/enigma/EnigmaProject.java
- Download the jar release: https://maven.fabricmc.net/cuchaz/enigma-swing/2.3.3/
- Compile the .java that has the patch against the .jar:
javac -cp enigma-swing-2.3.3-all.jar EnigmaProject.java - Move the resulting "EnigmaProject$ClassSource.class" to a folder called
cuchaz/enigma - Update the jar file:
jar uf enigma-swing-2.3.3-all.jar cuchaz/enigma/EnigmaProject*.class
If no error happened you should have a patched working enigma (did that because i don't know how to recompile the entire project but it would be nice to just have a working release)
@Guigui220D The project uses Gradle for dependency and build management, which is a fairly standard setup. A fat distribution JAR can be built locally by running ./gradlew shadowJar, which will place the output at enigma-swing/build/libs/enigma-swing-2.3.3+local-all.jar.
Anyway, from an end-user perspective, I'd recommend against using Enigma for mass source export or automated CLI usage. Clearly, it has not seen much maintenance in this area lately, some CLI commands have always been extremely memory-intensive, and frankly, there are better ways to read and tinker with Minecraft's code, thanks to higher-level modding frameworks like Fabric itself.
@AlexTMjugador Okay thanks! I was using Enigma as it was recommended for older versions of minecraft, but I had better luck with RetroMCP
Anyway, from an end-user perspective, I'd recommend against using Enigma for mass source export or automated CLI usage. Clearly, it has not seen much maintenance in this area lately, some CLI commands have always been extremely memory-intensive, and frankly, there are better ways to read and tinker with Minecraft's code, thanks to higher-level modding frameworks like Fabric itself.
hi, I maintain a fork of Enigma for the Quilt project (https://GitHub.com/quiltmc/enigma). I was wondering if you have any feedback on how to improve Enigma that I could implement over there. I'd recommend checking it out via cloning the project and running ./gradlew completeTestGui first, but I would love to hear your thoughts either way
Hey @ix0rai, please excuse that it took me a bit to get back to you!
I'm happy to share some feedback about what could be improved with respect to Enigma's CLI memory usage efficiency, as I think those would be very substantial for UX. Quilt's Enigma is pretty similar to upstream in the aspects I'm going to touch on, so I hope you find them useful.
For the discussion below, I've used Enigma to decompile and deobfuscate the Minecraft 1.20.2 client JAR into readable Java code, via its decompile command, after some initial mapping set up. I use the CFR decompiler, as it provided the overall best output when I initially set Enigma up.
Something that's obvious right of the bat is that decompilation with CFR can be very memory intensive, to the point that it can get unusable for computers with 8 GiB of or less RAM. With the help of JDK Mission Control and the built-in Java Flight Recorder, I've measured that the maximum Java heap size used during decompilation on an example run took up to 4.41 GiB, which rivals much more sophisticated LLVM compiler toolchains, known for being resource-hungry. Anecdotally, I've seen even higher heap memory usages, in the ballpark of 6 GiB.
Decompiler resource usage overview
To dig deeper into why Enigma is taking so much memory while decompiling, I've inspected the object classes responsible for most heap allocations, and the stack traces of the methods that led to such allocations. A summary of the results is attached below.
Decompiler allocation breakdown summary
Most byte array and string allocations are due to the decompilation process itself constructing readable source code, so at a surface it looks like there is not much optimization potential there. But such an intense HashMap usage is interesting: conceptually, we should not need to keep so many mappings in memory to decompile class trees.
Working backwards through the stack trace, I found that the EnigmaProject.JarExport#decompileStream and EnigmaProject.JarExport#exportRemappedJar methods are the root of the whole operation, and those methods are implemented in... rather memory-unconscious ways. They fetch the entire list of 7538 classes as of MC 1.20.2, and create several remapped and decompiled maps and lists of those in memory.
I believe re-engineering the decompiler to export classes in a streaming fashion, instead of by collecting them all into maps and lists, could go a long way to improve Enigma in this regard. Memory usage patterns are similar when using other decompilers, such as Procyon, so this is not an issue specific to CFR either.
Sorry for the pretty long-wided comment! Please feel free to contact me via e-mail or Discord if you have any further questions or thoughts you want to share.
thank you for the detail, I've brought it up to the rest of the devs! do note that in quilt's enigma Vineflower is the default decompiler and we're focusing our efforts there