Provide a plugin for JMXMP support with jmx.remote.profiles = TLS SASL/PLAIN

[vroyer]

JMX over RMI over k8s port-forwarding is not possible. JMX over TCP with JMXMP is fine, but you need to manually add the jmxremote_optional.jar and VisualVM does allow to configure a jmx.remote.profiles to support TLS SASL plain text authentication.

Describe the solution you'd like A JMXMP plugin with the jmxremote_optional jar could add the jmx.remote.profiles=TLS SASL/PLAIN in the env to properly connect to a JVM over TLS with authentication. See oracle doc for client JMXMP connection with TLS+SASL plain text authentication.

Describe alternatives you've considered It currently works with no TLS and no authentication, but it's no suitable in a production environnement of JVM running in Kubernetes.

Additional context See this blog

[thurka]

Thanks for enhancement. Indeed JMX over TCP should be supported by VisualVM.

[thurka]

Unfortunately it looks like JMXMP is not actively developed. :-( Where is the latest version and how old it is?

[vroyer]

Yes, it has been developed by Oracle/Sun an open source project, but we did not found the source code. Jar is available on maven https://repo1.maven.org/maven2/org/glassfish/main/external/jmxremote_optional-repackaged/5.0/ https://repo1.maven.org/maven2/org/glassfish/main/external/jmxremote_optional-repackaged/5.0/

On 31 Oct 2019, at 20:31, Tomas Hurka [email protected] wrote:

Unfortunately it looks like JMXMP is not actively developed. :-( Where is the latest version and how old it is?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/oracle/visualvm/issues/162?email_source=notifications&email_token=ACOMPGKRKMVQ65DJY2FRCKTQRMXADA5CNFSM4I7P6MYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECY7M3Q#issuecomment-548533870, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOMPGNAUZ3QDJEYAQYBSBDQRMXADANCNFSM4I7P6MYA.

[jmborer]

I have such a plugin if you are interested

[vroyer]

Hi @jmborer, Yes, this plugin would be useful for the java community running java under kubernetes. Is it something you could share ?

[jmborer]

I am just investigating currently how JMX "environment" variables could be set in VisualVM. I have to look at the code.

[vroyer]

Ok thanks. You just need to add something like that in the JMX context:

Security.addProvider(new com.sun.security.sasl.Provider()); env.put("jmx.remote.profiles", "TLS SASL/PLAIN”);

On 6 Jan 2020, at 16:07, Jean-Marc Borer [email protected] wrote:

I am just investigating currently how JMX "environment" variables could be set in VisualVM. I have to look at the code.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/oracle/visualvm/issues/162?email_source=notifications&email_token=ACOMPGOR7U7CIFVHE4TIMGTQ4NCJXA5CNFSM4I7P6MYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIFWRQI#issuecomment-571173057, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOMPGKS76E4L5UUAINDDP3Q4NCJXANCNFSM4I7P6MYA.

[jmborer]

Sure, but then it is hardcoded in the implementation of the library where it should be configurable, isn't it? If you want I can first try this quick solution.

[vroyer]

Yes, ideally, the SASL profile should be configurable with a checkbox from the plugin panel for each new connection.

On 6 Jan 2020, at 16:48, Jean-Marc Borer [email protected] wrote:

Sure, but then it is hardcoded in the implementation of the library where it should be configurable, isn't it? If you want I can first try this quick solution.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/oracle/visualvm/issues/162?email_source=notifications&email_token=ACOMPGJ6QKZ2ATA64OGU6MTQ4NHFFA5CNFSM4I7P6MYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIF242Q#issuecomment-571190890, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOMPGM3Z3EIY4CEV3F5MGTQ4NHFFANCNFSM4I7P6MYA.

[jmborer]

First I need to update my plugin project. It is so old that I can no longer compile it (missing dependencies). I need to migrate it to Maven and then starting to explore with JMX customizers from VisualVM. At the same time, I will push everything on Github.

[jmborer]

I finally managed to build the plugin. You can find it here https://drive.google.com/open?id=1i7VliEs5pIDn_tHH1Spx147S184JdQDN

I forced SASL. Let me know if it works. If yes I'll start investigating how to improve the VisualVM plugin.

[vroyer]

Ok fine, it works with Elassandra with JMXMP and JMXMP over TLS with SASL login/password auth.

[jmborer]

So as it, it works with or without SASL, if I understand correctly your sentence?

[leleueri]

Hi,

I'm working with @vroyer. We tried your plugin to connect to our server :

• through JMXMP without security (no TLS/SSL link, no JMX login/pwd by checking the "do not required ssl connection" checkbox)
• through JMXMP with TLS/SSL and end user authentication using jmx login/pwd

Both tests succeeded.

the command line used to launch visualvm 1.4.4 is : visualvm -J-Djavax.net.ssl.trustStore=truststore.p12 -J-Djavax.net.ssl.trustStorePassword=xxxxx -J-Djavax.net.ssl.trustStoreType=PKCS12 -J-Dcom.sun.management.jmxremote.registry.ssl=true

[leleueri]

Small update.

Because you force the SASL profile, a non secured JMXMP connection doesn't work with your plugin. In the previous comment I said it was OK but for the non secured link, the successful connection was done without your plugin (I append the opendmk jar in the classpath : ./bin/visualvm -cp:p jmxremote_optional-repackaged-5.0.jar)

So our test results are

• connection through JMXMP without security (no TLS/SSL link, no JMX login/pwd by checking the "do not required ssl connection" checkbox)
  • KO with your plugin
  • OK without your plugin and including the OpenDMK jar in the visualvm classpath
• connection through JMXMP with TLS/SSL and end user authentication using jmx login/pwd
  • OK with your plugin
  • KO without your plugin and including the OpenDMK jar in the visualvm classpath

[jmborer]

OK thank you for your feedback. I actually was more expecting such behavior to be honest. I'll start working on the VisualVM plugin to support more connection options and get rid off hard coded settings. This was just a quick and dirty implementation to test the SASL feature. Question for you as you have the experience: how can I setup a simple test environment to check both connection types? Help would be appreciated here to accelerate the development.

[leleueri]

Hi,

I realise I didn't provide to you a way to test your plugin. I'm sorry about that.

To test your plugin, you can download elassandra (tar.gz file) : https://github.com/strapdata/elassandra/releases/download/v6.2.3.24/elassandra-6.2.3.24.tar.gz Extract the content and go into the elassandra directory.

Define the cassandra home : export CASSANDRA_HOME=$PWD

to start the server with JMXMP and without authentication, you can run elassandra with the commande :

./bin/cassandra -Dcassandra.jmxmp. to start the server with JMXMP and authentication, you have to follow the documentation about setting SSL on JMX for cassandra (https://docs.datastax.com/en/ddacsecurity/doc/ddacsecurity/secureSSLCertificates.html && https://docs.datastax.com/en/ddacsecurity/doc/ddacsecurity/secureDDACNodetoolSSL.html) and restart elassandra with the commande :

./bin/cassandra -Dcassandra.jmxmp. Beware to redefine the jmxremote.password file location in the cassandra-env.sh file.

Let me know if you encounter some issue.

[vroyer]

Hi @jmborer I did not succeed in using your VisualVM plugin with a TLS connection and SASL login/password. Your code is not setting the SASL profile to work properly in the JMX connection env.

So, i have fixed that in this plugin https://github.com/strapdata/visualvm-jmxmp and this works properly over a k8s port-forwarded connection.

Nevertheless, I had to remove the META-INF/services/javax.management.remote.JMXConnectorProvider from the jmxremote_optional-repackaged-5.0.jar to avoid a conflict with one i provided. It's a packaging issue that i did not succeed to solve properly ....

[jmborer]

~~May I ask if you guys were ever able to build the latest version of VisualVM from the sources? I am using Zulu 8 version and the build fails complaining that any source < 1.7 is no longer supported.~~ Forget that question. Environment setup issue on my side. I can finally start to work on the VVM plugin, if I manage to publish the NBMs as Maven artefacts ;)

[jmborer]

Hello guys.

Just to let you know that I am progressing with the implementation of the plugin. Setuping the project was not easy since it required several fixes, I discovered in both in VisualVM and the Maven NBM build plugin. After fixing and improving those projects, I now I have solid project skeleton base that works with the latest VisualVM (2.0 or 20.1) and a simple running example.

[vroyer]

Hi Jean-Marc, Thanks for feedback. Do you known why Oracle does not fix JMXMP serialization vulnerability, see acunetix ? JMXMP source code is not available, there is no way to fix that without violating the license ?

Graal VM won't support RMI, so how will it support JMX without JMXMP ? Remote monitoring won't be possible with GraalVM ?

[jmborer]

Hi Eric, I have no idea about Oracle. The JMX version I am using and that I patched for bugs is a fork of the DMK project that no longer exists. You can find the code here:

https://github.com/javawerks/jmxmp

If you know to to fix the issue describe https://www.acunetix.com/blog/web-security-zone/old-java-libraries feel free to clone my repo and fix it. If you look at the root there are Classe diagrams I have reverse engineered when I was looking for the JMX issues.

All I can tell you it that this library is used in a critical production environment since many years without any issue. We monitor several dozens of applications and use heavily bean notifications without any issue. The library is used on both on the server and clients.

It is still written to support Java 1.3. However I have to admit that when I looked at the code, I was thinking that it could benefit for a more modern implementation (especially on multitasking side), but had never the courage to start this work. You know what you have, you don't know what you will get... but it could be a very interesting exercise that would require a lot of unit and sanity tests.

I would suggest you contact me directly for any further discussions out of scope of this issue.