Feature request: allow sampling snapshot recording from command line only in headless environment

[kevinmilner]

Is your feature request related to a problem? Please describe. I would like to be able to record snapshots of sampled CPU usage from the command line in a headless environment, and then later open that snapshot in the VisualVM GUI. The command line docs suggest that it's possible to start and stop sampling from a command line, but there doesn't seem to be a working option to dump that information to a snapshot file.

Describe the solution you'd like New command line option to dump CPU (or memory) sampling results to a file that can be loaded in the GUI later

Describe alternatives you've considered I have tried the following series of commands, hoping that it might work, but it doesn't seem to do anything:

[kmilner@e19-12 ~]$ mkdir /tmp/visualvm_temp/
[kmilner@e19-12 ~]$ ~/visualvm_21/bin/visualvm -J-Dvisualvm.tmpdir=/tmp/visualvm_temp/ --start-cpu-sampler 31650
/home1/kmilner/visualvm_21/platform/lib/nbexec: WARNING: environment variable DISPLAY is not set
^Z
[1]+  Stopped                 ~/visualvm_21/bin/visualvm -J-Dvisualvm.tmpdir=/tmp/visualvm_temp/ --start-cpu-sampler 31650
[kmilner@e19-12 ~]$ bg
[1]+ ~/visualvm_21/bin/visualvm -J-Dvisualvm.tmpdir=/tmp/visualvm_temp/ --start-cpu-sampler 31650 &
[kmilner@e19-12 ~]$ ~/visualvm_21/bin/visualvm -J-Dvisualvm.tmpdir=/tmp/visualvm_temp/ --snapshot-sampler 31650
/home1/kmilner/visualvm_21/platform/lib/nbexec: WARNING: environment variable DISPLAY is not set
[kmilner@e19-12 ~]$ ls /tmp/visualvm_temp/
[kmilner@e19-12 ~]$

Additional context Add any other context or screenshots about the feature request here.

[ronosaurus]

Have you tried capturing with JFR? Its produces data very similar to VisualVM's sampling and is designed to be invoked from the command line. JFR files can be opened with VisualVM. Having said all that, I prefer capturing with VisualVM because the Hit Counts are much more accurate in nps files than jfr files and VisualVM's support for exclude/include classes removes a lot of noise.

[thurka]

The command line docs suggest that it's possible to start and stop sampling from a command line

You are right, but they are used for IDE integrations, so they start VisualVM and then invoke action start/stop sampling of particular process without user interaction. Can you describe your use case in more details? Maybe you can use remote monitoring via JMX.

[thurka]

@kevinmilner Any update? Can you describe your use case in more details?

[ronosaurus]

VisualVM CPU sampling is often better than JFR because it allows package filtering and hit counts are much more accurate.

In our environments we have to restrict who has access to the GUI for capturing because they could install the MBeans plugin and access database connection strings. If things were scriptable like this we could automate ad-hoc and scheduled captures:

visualvm --start-cpu-sampler <pid@exclude-classes=<filter>,sampling-rate=<rate_ms>>
sleep 10000
visualvm --snapshot-sampler <pid>
sleep 5000
visualvm --snapshot-sampler <pid>
sleep 5000
visualvm --snapshot-sampler <pid>
visualvm --stop-sampler

Seems like --snapshot-sampler needs an option to specify a path of the output file.

I don't know what the correct behavior would be to handle the (edge?) case where there are overlapping requests to --start-cpu-sampler.

[thurka]

@ronosaurus As mentioned above the commandline options are used for IDE integration with VisualVM, so they just simulate appropriate user action in running VisualVM. One thing, I didn't get an answer to, is why it is not possible to use remote JMX connection to your application? You can do sampling over the JMX connection and you can also control who can access the application with various JMX settings.

[thurka]

One more idea - you can use <visualvm_dir>/platform/modules/org-netbeans-modules-sampler.jar to create CPU sampling snapshot without visualvm and without GUI. See below:

java -jar platform/modules/org-netbeans-modules-sampler.jar 
Usage: <port> <snapshot.npss>

First of all start your application with following parameters:
  -Dcom.sun.management.jmxremote.authenticate=false
  -Dcom.sun.management.jmxremote.ssl=false
  -Dcom.sun.management.jmxremote.port=<port>
Then you can start this sampler with correct port and file to write snapshot to.

This uses local JMX connection to sampled application. If you do not want to enable JMX with -Dcom.sun.management.jmxremote properties, you can use jcmd to enable local management agent with:

<java_home>/bin/jcmd <pid>  ManagementAgent.start_local
<java_home>/bin/jcmd <pid>  ManagementAgent.status

use URL from ManagementAgent.status command as connection string:

java -jar platform/modules/org-netbeans-modules-sampler.jar <URL> <path_to_CPU_snapsht>

[ronosaurus]

Yes, I understand the intent of "visualvm --start-cpu-sampler" is to help automate the UI from IDEs. My understanding of this thread was to gather use cases for an enhancement so interacting with a UI is not required. If a UI is not required this could be automated into a Jenkins job:

visualvm --start-cpu-sampler <pid@exclude-classes=<filter>,sampling-rate=<rate_ms>>
sleep 10000
visualvm --snapshot-sampler <pid>
sleep 5000
visualvm --snapshot-sampler <pid>
sleep 5000
visualvm --snapshot-sampler <pid>
visualvm --stop-sampler

I use VisualVM to remotely connect to a host via JMX ports but it requires a human to manually start CPU profiling and take snapshots. There are safe guards in place so only a small number of authorized users can connect to the JMX ports. I want to take that to the next level and encapsulate it into a Jenkins job so more users can press a button without having to interact with the VisualVM UI. They just launch a Jenkins job and nps files appear a few minutes later.

I've automated JFR from the command line with jcmd but I prefer VisualVM because of its package filtering and accurate hit counts.

I'm not familiar with org-netbeans-modules-sampler.jar so I'll study that. I think you're explaining that I could use this example code to control capturing remote samples via ManagementFactory.THREAD_MXBEAN_NAME and save them to a nps file:

https://github.com/apache/netbeans/blob/14/platform/sampler/src/org/netbeans/modules/sampler/CLISampler.java

final ThreadMXBean threadMXBean = ManagementFactory.newPlatformMXBeanProxy(
	server, ManagementFactory.THREAD_MXBEAN_NAME, ThreadMXBean.class);
final File output = new File(args[1]);
CLISampler s = new CLISampler(threadMXBean, output);

I quickly glanced at the code but didn't see a way to exclude-classes or set sampling-rate??

Also, I'm not an expert on the inner workings of JMX, I'm just a happy user of VisualVM :)

[thurka]

Yes, I understand the intent of "visualvm --start-cpu-sampler" is to help automate the UI from IDEs. My understanding of this thread was to gather use cases for an enhancement so interacting with a UI is not required.

Yes, you are right. I was not sure that you are aware of the usage of --start-cpu-sampler command. Using it in your examples look strange for me. There have to be different set of commands for what you are trying to accomplish.

I use VisualVM to remotely connect to a host via JMX ports but it requires a human to manually start CPU profiling and take snapshots. There are safe guards in place so only a small number of authorized users can connect to the JMX ports. I want to take that to the next level and encapsulate it into a Jenkins job so more users can press a button without having to interact with the VisualVM UI. They just launch a Jenkins job and nps files appear a few minutes later.

Well, the original request was to do CPU sampling in headless environment and this can be also accomplished via remote JMX connection from VisualVM - so this is the reason I was talking about it. Of course if you want to automate it, this is different story.

I'm not familiar with org-netbeans-modules-sampler.jar so I'll study that. I think you're explaining that I could use this example code to control capturing remote samples via ManagementFactory.THREAD_MXBEAN_NAME and save them to a nps file:

Right, expect that the file is npss. This is different format from nps, which does not require any VisualVM code for its creation. Therefor you need just this one jar to be able to create snapshot of sampled CPU.

I quickly glanced at the code but didn't see a way to exclude-classes or set sampling-rate??

It captures everything, you can exclude classes once you open the snapshot in VisualVM. Sampling rate is 10 ms and cannot be changed from commandline.

Also, I'm not an expert on the inner workings of JMX

Me neither and this expertise is not required to use org-netbeans-modules-sampler.jar.

I'm just a happy user of VisualVM :)

Thank you! :-)

[AntoineLange]

@ronosaurus Did you manage to accomplish what you wanted to do?