Byte Buddy agent not initialized or unavailable on paths with special characters

[DrFair]

Hey, first I just wanted to say that it's an awesome project you have going, and thank you for making it available for free!

I am running and developing a game in Java, which uses Byte Buddy to allow mods to transform the game's base classes on runtime. Everything is running great, but today I got a crash report which generated this stack trace:

java.lang.IllegalStateException: The Byte Buddy agent is not initialized or unavailable
at net.bytebuddy.agent.ByteBuddyAgent.getlnstrumentation(ByteBuddyAgent.java:225)
at net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:612)
at net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:563)
at net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:540)
at necesse.engine.util.ComputedValue.get(ComputedValue.java:16)
at necesse.engine.modLoader.LoadedMod.loadClasses(LoadedMod.java:114)
at necesse.engine.modLoader.ModLoader.loadMods(ModLoader.java:200)
at necesse.engine.GlobalData.loadAll(GlobalData.java:286)
at necesse.engine.Screen.init(Screen.java:1000)
at necesse.engine.Screen.<init>(Screen.java:264)
at StartClient.main(StartClientjava:112)

I should say that the game is currently using Byte Buddy version 1.12.8. I have also tested it with the latest version (1.14.3) and it seems to have the same problem.

I was able to hunt it down, and it was because the user had their game installed in a folder path with special characters. Like this: E:\==STEAM==\steamapps\common\Necesse.

I have run into this problem before when using Java's URI class for handling the game's zip archives as saves. I haven't found a way so far to encode the URI using one of Java's own URLEncoder method, since that will also encode things like spaces and caused it to not work. The way I have done it so far is hard code the encoding of some of the special characters that are causing issues and taking it case by case every time someone reports a new character is causing issues. These are characters like %^´#!= an also the character `.

Admittedly, I don't know much about Java reflection and what Byte Buddy is doing in general. So far I am just telling users to move the game's install directory to another folder and I just wanted to give a heads-up.

Thanks, again!

[raphw]

Likely the execution of this command fails: https://github.com/raphw/byte-buddy/blob/master/byte-buddy-agent/src/main/java/net/bytebuddy/agent/ByteBuddyAgent.java#L689

I do not have a Windows machine to test this. Would you be able to build Byte Buddy locally and try some options?

[DrFair]

It looks like that command is never reached. Instead, it is going into this: https://github.com/raphw/byte-buddy/blob/master/byte-buddy-agent/src/main/java/net/bytebuddy/agent/ByteBuddyAgent.java#L633

[dogourd]

I helped to do some debugging on Windows, and I found that there was an issue when the path of byte-buddy-agent.jar contained an = sign. In my case, the path was D:\downloads\==STEAM==\project\untitled\target\lib\byte-buddy-agent-1.14.3.jar, and the error message was: "Error opening zip file or JAR manifest missing: D:\downloads". It seems that the problem lies in the = parsing strategy in the InvocationAdapter.c file of OpenJDK.

Fortunately, we can bypass this problem at the Byte Buddy level by making some adjustments in the AgentProvider#resolve method. When we detect the presence of the = sign in the path, we can create and return a new JAR file using net.bytebuddy.agent.ByteBuddyAgent.AgentProvider.ForByteBuddyAgent#createJarFile. I was able to debug the code and modify the variable values to make the logic go through this branch, and it worked well!

[raphw]

You are right, that explains it. Glad you found a solution!

[dogourd]

Do you think ByteBuddy should be responsible for doing some extra processing for the special character '=' either after or within the implementation of the method? ForByteBuddyAgent#trySelfResolve

[raphw]

Certainly. Did you manage to escape this somehow?

[dogourd]

Yes, during the debugging phase, I found that the File path returned from trySelfResolve method contained the '=' character. So, I set the return value to null, allowing the logic to continue executing the net.bytebuddy.agent.ByteBuddyAgent.AgentProvider.ForByteBuddyAgent#createJarFile method, which completed the replacement. This resolved the issue.

I think ByteBuddy should consider adding some checks here, to return CANNOT_SELF_RESOLVE if '=' is found in the full path during the execution of trySelfResolve, so that it can try the subsequent createJarFile action, which seems reasonable because we have confirmed that this file path cannot be successfully processed later.

Of course, since createJarFile tries to output the jar file to java.io.tmpdir, the worst-case scenario is that java.io.tmpdir points to a path containing the '=' character:worried:. But in this case, I think ByteBuddy is not responsible for it, as it has already made the maximum effort.

[raphw]

That's a good idea actually. I'll do that!

[dogourd]

I noticed a commit (Avoid use of location if agent argument separator is contained.) that seems to be attempting to fix this issue, but it may still not be working correctly. The error discussed in this issue does not come from installExternal, but rather from this part of the code Attacher::install where agentProvider.resolve().getAbsolutePath() produces a bad argument.

[raphw]

It should fix the issue as it is implemented like this:

public File resolve() throws IOException {
  try {
    File agentJar = trySelfResolve();
    return agentJar == null
       ? createJarFile()
       : agentJar;
     } catch (Exception ignored) {
       return createJarFile();
     }
}

If the path of the current application is invalid as an agent path (trySelfResolve) null is returned and a new temporary file is created instead. Of course, if this still contains =, the actual attach would still fail, but then there is no good way around.

[dogourd]

There are two definitions of the method trySelfResolve in the ByteBuddyAgent.java file. One is in the ByteBuddyAgent class, and the other is hidden in the ByteBuddyAgent.AgentProvider.ForByteBuddyAgent.