homebrew-core icon indicating copy to clipboard operation
homebrew-core copied to clipboard
verified publisher icon Homebrew

async-profiler 4.0 (new formula)

Open bric3 opened this issue 11 months ago • 13 comments

  • [x] Have you followed the guidelines for contributing?
  • [x] Have you ensured that your commits follow the commit style guide?
  • [x] Have you checked that there aren't other open pull requests for the same formula update/change?
  • [x] Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • [x] Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • [x] Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

Also : brew style --formula async-profiler

Contributing my formula for async-profiler/async-profiler (8.1 K stars).

Note this formula is only for version 4.0, however my original formulas started with 2.8 (which was mostly a script and and lib until 3.0).

bric3 avatar May 02 '25 14:05 bric3

Here's how I'd create this formula:

class AsyncProfiler < Formula
  desc "Sampling CPU and HEAP profiler for Java"
  homepage "https://github.com/async-profiler/async-profiler/releases"
  url "https://github.com/async-profiler/async-profiler/archive/refs/tags/v4.0.tar.gz"
  sha256 "7beb736868af485d6b0b624e42141f78df0ca8403188adc17965b7153261aa55"
  license "Apache-2.0"

  depends_on "openjdk"

  def install
    system "make", "all"
    bin.install Dir["build/bin/*"]
    lib.install Dir["build/lib/*"]
    libexec.install Dir["build/jar/*"]
  end

  test do
    assert_match "Async-profiler #{version}", shell_output("#{bin}/asprof --version")
    # TODO: add better test
    # https://docs.brew.sh/Formula-Cookbook#add-a-test-to-the-formula
  end
end

botantony avatar May 02 '25 15:05 botantony

Hi, also what would be the strategy (if it's even possible) to make my formula redirect to this one?

bric3 avatar May 05 '25 08:05 bric3

You can add a tap_migrations.json, it marks a formula as migrated.

SMillerDev avatar May 05 '25 10:05 SMillerDev

So the tests are failing https://github.com/Homebrew/homebrew-core/actions/runs/14833693346/job/41640450001?pr=222217#step:3:57

The error : Could not start attach mechanism: No such file or directory

Is there something specific to know about macOs tests, how is the JVM run ?

Error log 
  /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.4.3/bin/bundle clean
  ==> Testing async-profiler
  ==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof --version
  ==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-14530-kx2x51/test-profile-via-attach.html jps
  Could not start attach mechanism: No such file or directory
  Error: async-profiler: failed
  An exception occurred within a child process:
    BuildError: Failed executing: /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-14530-kx2x51/test-profile-via-attach.html jps
  /opt/homebrew/Library/Homebrew/formula.rb:3119:in 'block in Formula#system'
  /opt/homebrew/Library/Homebrew/formula.rb:3055:in 'IO.open'

Locally on macOs 15.4

brew test --verbose async-profiler
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.4.3/bin/bundle clean
==> Testing bric3/tap/async-profiler
==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof --version
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-66666-ak7y1g/test-profile-via-attach.html jps
Profiling for 2 seconds
Done
==> /opt/homebrew/opt/openjdk/bin/java -agentpath:/opt/homebrew/Cellar/async-profiler/4.0/lib/libasyncProfiler.dylib=start,event=cpu,lock=10ms,file=test-profile-via-lib.jfr /private/tmp/async-profiler-test-20250505-66666-ak7y1g/Main.java 2
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
Profiling started
==> /opt/homebrew/Cellar/async-profiler/4.0/bin/jfrconv -o pprof /private/tmp/async-profiler-test-20250505-66666-ak7y1g/test-profile-via-lib.jfr /private/tmp/async-profiler-test-20250505-66666-ak7y1g/test-profile-via-lib.pprof
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
Converting test-profile-via-lib.jfr -> test-profile-via-lib.pprof # 0.017 s

FYI here's what async-profiler says on this error.

https://github.com/async-profiler/async-profiler/blob/73f04869464df9dce09fe4755e7a1c05712ae480/docs/Troubleshooting.md#could-not-start-attach-mechanism-no-such-file-or-directory

Could not start attach mechanism: No such file or directory

The profiler cannot establish communication with the target JVM through UNIX domain socket. Usually this happens in one of the following cases:

  1. Attach socket /tmp/.java_pidNNN has been deleted. It is a common practice to clean /tmp automatically with some scheduled script. Configure the cleanup software to exclude .java_pid* files from deletion.

    • How to check: run lsof -p PID | grep java_pid. If it lists a socket file, but the file does not exist, then this is exactly the described problem.

  2. JVM is started with -XX:+DisableAttachMechanism option.

  3. /tmp directory of Java process is not physically the same directory as /tmp of your shell, because Java is running in a container or in chroot environment. asprof attempts to solve this automatically, but it might lack the required permissions to do so.

    • Check strace asprof PID jcmd

  4. JVM is busy and cannot reach a safepoint. For instance, JVM is in the middle of long-running garbage collection.

    • How to check: run kill -3 PID. Healthy JVM process should print a thread dump and heap info in its console.

bric3 avatar May 05 '25 13:05 bric3

Attach socket /tmp/.java_pidNNN has been deleted.

You can't just write anywhere in the tests, brew usage, especially for tests, is heavily sandboxed. It might be best to set the tmp directory to testpath/"tmp" in the test.

SMillerDev avatar May 05 '25 13:05 SMillerDev

Unfortunately the JVM doesn't work that way, the JVM temp directory is hardcoded to /tmp on linux, and to the the user temp folder on OSX

In order for the attach mechanism to work a socket FD is needed to be present.

Locally I have the following file descriptor opened.

==> bash -c 'lsof -p 84540'
Picked up _JAVA_OPTIONS: -Duser.home=/Users/brice.dutheil/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
==> COMMAND   PID          USER   FD   TYPE             DEVICE  SIZE/OFF                NODE NAME
java    84540 brice.dutheil  cwd    DIR               1,15       128           251759721 /private/tmp/async-profiler-test-20250505-84530-cz8bev
java    84540 brice.dutheil  txt    REG               1,15     69856           225322662 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/bin/java
java    84540 brice.dutheil  txt    REG               1,15    130416           225323381 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib
java    84540 brice.dutheil  txt    REG               1,15     32768           251759730 /private/var/folders/5_/gltgq4390wx07yvypgl1mc400000gq/T/hsperfdata_brice.dutheil/84540
java    84540 brice.dutheil  txt    REG               1,15     76144           225323377 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjimage.dylib
java    84540 brice.dutheil  txt    REG               1,15     66964           250200055 /Library/Preferences/Logging/.plist-cache.NXoAOdnc
java    84540 brice.dutheil  txt    REG               1,15       110 1152921500312087311 /System/Library/CoreServices/SystemVersion.bundle/English.lproj/SystemVersion.strings
java    84540 brice.dutheil  txt    REG               1,15    187024           225323364 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjava.dylib
java    84540 brice.dutheil  txt    REG               1,15   2288832 1152921500312522782 /usr/lib/dyld
java    84540 brice.dutheil  txt    REG               1,15  15633904           225323448 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/libjvm.dylib
java    84540 brice.dutheil  txt    REG               1,15 143206280           225323256 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
java    84540 brice.dutheil  txt    REG               1,15  33979312 1152921500312533288 /usr/share/icu/icudt76l.dat
java    84540 brice.dutheil  txt    REG               1,15  15630336           225323267 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/classes.jsa
java    84540 brice.dutheil    0u   CHR               16,8     0t471                1043 /dev/ttys008
java    84540 brice.dutheil    1u   CHR               16,8     0t471                1043 /dev/ttys008
java    84540 brice.dutheil    2u   CHR               16,8     0t471                1043 /dev/ttys008
java    84540 brice.dutheil    3r   REG               1,15 143206280           225323256 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
java    84540 brice.dutheil    4u  unix 0x71c935f907907507       0t0                     /var/folders/5_/gltgq4390wx07yvypgl1mc400000gq/T//.java_pid84540.tmp

Notice the .java_pid84540 file, while on the CI it is not present:

  ==> bash -c 'lsof -p 14483'
  Picked up _JAVA_OPTIONS: -Duser.home=/Users/brew/Library/Caches/Homebrew/java_cache -Djava.io.tmpdir=/private/tmp
  ==> COMMAND   PID USER   FD    TYPE             DEVICE  SIZE/OFF    NODE NAME
  java    14483 brew  cwd     DIR               1,15       128 4021049 /private/tmp/async-profiler-test-20250505-14459-udwwfr
  java    14483 brew  txt     REG               1,15     69856 4020034 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/bin/java
  java    14483 brew  txt     REG               1,15    131136 4020734 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib
  java    14483 brew  txt     REG               1,15     52392 3957305 /Library/Preferences/Logging/.plist-cache.0qxpMgpi
  java    14483 brew  txt     REG               1,15     32768 4021072 /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T/hsperfdata_brew/14483
  java    14483 brew  txt     REG               1,15     76864 4020733 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjimage.dylib
  java    14483 brew  txt     REG               1,15    187744 4020729 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjava.dylib
  java    14483 brew  txt     REG               1,15  15987024 4020757 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/libjvm.dylib
  java    14483 brew  txt     REG               1,15 143205520 4020628 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
  java    14483 brew  txt     REG               1,15  15630336 4020639 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/server/classes.jsa
  java    14483 brew    0u    CHR               16,0     0t453     703 /dev/ttys000
  java    14483 brew    1u    CHR               16,0     0t453     703 /dev/ttys000
  java    14483 brew    2u    CHR               16,0     0t453     703 /dev/ttys000
  java    14483 brew    3r    REG               1,15 143205520 4020628 /opt/homebrew/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home/lib/modules
  java    14483 brew  136u  systm 0x47efe4af3f450a85       0t0         [ctl com.apple.netsrc id 6 unit 34]
  java    14483 brew  140u  systm 0x47efe4af3f450005       0t0         [ctl com.apple.netsrc id 6 unit 33]
  java    14483 brew  149    PIPE 0x4d5c62a40ea9cfc0     65536         ->0x235b49b07fa21557
  java    14483 brew  152    PIPE 0x669a58636e94ab49     16384         ->0x2778f7377120f743

What is strange is that the hsperfdata also generated by the JVM is correctly created in what seems like to be an usual temp folder /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T. Not sure what's going on there.

Note I used the -XX:+StartAttachListener which creates this file upon JVM start.

I wonder if there might be another thing at play (maybe due to this sandboxing), because this test works locally (brew test --verbose async-profiler).

bric3 avatar May 05 '25 17:05 bric3

an usual temp folder /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T

That looks like the default tmp location for macOS.

SMillerDev avatar May 05 '25 17:05 SMillerDev

The error is pretty clear, it's not finding openjdk.

  ==> /opt/homebrew/Cellar/async-profiler/4.0/bin/asprof -d 2 -f /private/tmp/async-profiler-test-20250505-14459-udwwfr/test-profile-via-attach.html jps
  The operation couldn’t be completed. Unable to locate a Java Runtime.

SMillerDev avatar May 05 '25 17:05 SMillerDev

That's the last run when I introduced the lsof on java to debug. But in previous runs, Async profiler had the error described in my earlier comment.

See : https://github.com/Homebrew/homebrew-core/actions/runs/14833693346/job/41640450001#step:3:64

bric3 avatar May 05 '25 22:05 bric3

By any chance is there a way to reproduce in some kind of macos container what happens in the this workflow?

bric3 avatar May 07 '25 19:05 bric3

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

github-actions[bot] avatar May 28 '25 23:05 github-actions[bot]

By any chance is there a way to reproduce in some kind of macos container what happens in the this workflow?

I usually just leverage the macos github action workflow to do such debugging https://github.com/chenrui333/github-action-test/blob/main/.github/workflows/brew-debug.yml

chenrui333 avatar May 29 '25 16:05 chenrui333

Thanks @chenrui333 I'll take a look

bric3 avatar May 30 '25 18:05 bric3

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

github-actions[bot] avatar Jul 20 '25 03:07 github-actions[bot]