Installation via brew fails on M1 Macs

[the-kenny]

Trying to install the latest bloop via Homebrew on a M1 MBP fails:

➜ brew install scalacenter/bloop/bloop
==> Installing bloop from scalacenter/bloop
==> Downloading https://github.com/scalacenter/bloop/releases/download/v1.4.8/ba
Already downloaded: /Users/moritz/Library/Caches/Homebrew/downloads/15b656d402cb87c13fcd6b5ed9a4fa4844dc7a8bfd053e3dd8436e081d2827d3--bash-completions
==> Downloading https://github.com/scalacenter/bloop/releases/download/v1.4.8/zs
Already downloaded: /Users/moritz/Library/Caches/Homebrew/downloads/8097faaab156e49489f66095321e47efcbf6eb834af7fb6812db66aa24445af4--zsh-completions
==> Downloading https://github.com/scalacenter/bloop/releases/download/v1.4.8/fi
Already downloaded: /Users/moritz/Library/Caches/Homebrew/downloads/0f402fb7013984ff25db8a5a4ee5a8ac7030ae895220f5a49af243352a97e603--fish-completions
==> Downloading https://github.com/scalacenter/bloop/releases/download/v1.4.8/bl
Already downloaded: /Users/moritz/Library/Caches/Homebrew/downloads/34257445775368b19ca13cd407ed295836d269ffbfdf750d1492bd60cbffeac0--bloop-coursier.json
==> coursier install --install-dir bin --default-channels=false --channel channe
Last 15 lines from /Users/moritz/Library/Logs/Homebrew/bloop/01.coursier:
Downloading https://github.com/shyiko/jabba/raw/master/index.json
Downloaded https://github.com/shyiko/jabba/raw/master/index.json
Exception in thread "main" coursier.jvm.JvmCache$JvmNotFoundInIndex: JVM graalvm:19.3 not found in index: No JVM found for OS darwin and CPU architecture arm64
	at coursier.jvm.JvmCache.$anonfun$getIfInstalled$2(JvmCache.scala:101)
	at coursier.jvm.JvmCache.$anonfun$getIfInstalled$2$adapted(JvmCache.scala:100)
	at coursier.util.Task$.$anonfun$flatMap$extension$1(Task.scala:14)
	at coursier.util.Task$.$anonfun$flatMap$extension$1$adapted(Task.scala:14)
	at coursier.util.Task$.wrap(Task.scala:84)
	at coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
	at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:831)

[tgodzik]

Thanks for reporting! It looks like graalvm is not support in jabba, which is used underneath in coursier:

https://github.com/shyiko/jabba/blob/master/index.json

arm64 architecture only has jdk currently.

It also looks like it's graalvm is not supported on arm64? https://github.com/oracle/graal/issues/2666

Probably it would be best to use normal jdk currently, though I am not up to date on specifics of the new apple chipsets.

[carlosedp]

Bloop also does not install on arm64 Linux hosts using AdoptOpenJDK 8:

❯ cs install --jvm 8 bloop
https://repo1.maven.org/maven2/io/get-coursier/apps/maven-metadata.xml
  No new update since 2021-03-02 07:43:10
https://repo1.maven.org/maven2/ch/epfl/scala/bloopgun_2.12/maven-metadata.xml
  No new update since 2021-03-02 06:50:20
Exception in thread "main" coursier.jvm.JvmCache$JvmNotFoundInIndex: JVM graalvm:19.3 not found in index: JVM graalvm not found
	at coursier.jvm.JvmCache.$anonfun$getIfInstalled$2(JvmCache.scala:101)
	at coursier.jvm.JvmCache.$anonfun$getIfInstalled$2$adapted(JvmCache.scala:100)
	at coursier.util.Task$.$anonfun$flatMap$extension$1(Task.scala:14)
	at coursier.util.Task$.$anonfun$flatMap$extension$1$adapted(Task.scala:14)
	at coursier.util.Task$.wrap(Task.scala:84)
	at coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
	at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(Thread.java:834)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:517)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)

My environment:

❯ uname -a
Linux Batuu 4.9.201-tegra #1 SMP PREEMPT Fri Feb 19 08:42:04 PST 2021 aarch64 aarch64 aarch64 GNU/Linux

❯ cs java-home
/home/carlosedp/.cache/coursier/jvm/[email protected]
❯ cs java --jvm 8 -version
openjdk version "1.8.0_272"
OpenJDK Runtime Environment (build 1.8.0_272-b10)
OpenJDK 64-Bit Server VM (build 25.272-b10, mixed mode)
❯ cs java -version
openjdk version "1.8.0_272"
OpenJDK Runtime Environment (build 1.8.0_272-b10)
OpenJDK 64-Bit Server VM (build 25.272-b10, mixed mode)

[tgodzik]

I think I know what's going on. Homebrew install uses the graalvm-native-image by default, which is not possible to use on arm. We should probably detect it an install jar based launcher in this case.

The workaround currently is to use coursier to install it: coursier install bloop-jvm

It would be best to fix it in https://github.com/scalacenter/homebrew-bloop/blob/master/bloop.rb to detect the system architecture and run the different installer in that case. I don't have a Mac to test it out, so any help would be appreciated.

[carlosedp]

Just adding that I haven't used Homebrew to install Coursier on Linux. Used the native binary directly and then cs install cs.

[tgodzik]

Just adding that I haven't used Homebrew to install Coursier on Linux. Used the native binary directly and then cs install cs.

The default bloop app is using the native image. That's why you need bloop-jvm

[fernandoracca]

Reproduced this issues on a Mac M1 via Homebrew installation.

This file is been used as an index for the JVM to pull

https://raw.githubusercontent.com/shyiko/jabba/master/index.json

however, there is no such version 19.3, instead it has a minor version

    "19.3.0": "zip+https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-19.3.0.2/graalvm-ce-java8-windows-amd64-19.3.0.2.zip",

==> Installing scalacenter/bloop/bloop
==> coursier install --install-dir bin --default-channels=false --channel channel bloop
Last 15 lines from /Users/fer/Library/Logs/Homebrew/bloop/01.coursier:
Downloading https://github.com/shyiko/jabba/raw/master/index.json
Downloaded https://github.com/shyiko/jabba/raw/master/index.json
Exception in thread "main" coursier.jvm.JvmCache$JvmNotFoundInIndex: JVM graalvm:19.3 not found in index: JVM graalvm not found
	at coursier.jvm.JvmCache.$anonfun$getIfInstalled$2(JvmCache.scala:101)
	at coursier.jvm.JvmCache.$anonfun$getIfInstalled$2$adapted(JvmCache.scala:100)

this breaks the installation. any suggestion?

[fernandoracca]

this PR looks related

https://github.com/scalacenter/bloop/pull/1278

[kraktus]

Installation is still failing on M1 (same error)

[tgodzik]

Graal still doesn't seem to work for M1 (https://github.com/oracle/graal/issues/2666) so best to use bloop-jvm instead

[kraktus]

Thank you for the workaround will try. What are the differences between bloop and bloop-jvm? If there is none for the end user, I might as well try to update brew formula.

[tgodzik]

bloop is native image based (which doesn't work on M1) while bloop-jvm is just a jar.

[kammoh]

Hi! Any updates since https://github.com/oracle/graal/issues/2666#issuecomment-1074884020? What's the recommended way to compile bloop with an existing GraalVM installation?

[tgodzik]

Most likely we would only need to update once the release of GraalVM is stable, is it out yet?

It should be possible use coursier to boostrap bloop or bloop-jvm - https://get-coursier.io/docs/cli-bootstrap#graalvm-native-image

Though I have not tried myself.

[arixmkii]

GraalVM 22.1.0 with Apple Silicon native image (still preview) support was recently released.

[tgodzik]

Interesting! I think we would require a github runner for Apple Silicon to create binary, but maybe it's possible to produce it also using coursier?

There is no runner currently available from what I know, but if the demand is high we can try and set something up with some help.

[arixmkii]

I think we would require a github runner for Apple Silicon to create binary

Unfortunately this is very true and it is a blocker for numerous OSS projects as the only solution right now seems to be selfhosted runner.

It could be possible to bootstrap native version with coursier on M1 macs, just need to adjust the command line, so, that coursier will be able to find the version for darwin-aarch64 in JVM index. I will try it myself, when I find a time window for this.

[arixmkii]

For now coursier is a no go, at least because of this https://github.com/oracle/graal/issues/4526 There were also issues that VM resolution in coursier bootstrap is not working correctly on M1 (~for me it detected arch as amd64~ this is because native version in homebrew is compiled for Rosetta2) and that it uses incorrect coordinates in jvm index. Will try to formulate issue in coursier github.

For one other application coursier had no such issues, when used from sbt and sbt-native-image plugin. It is bootstrap cli exclusive thing.

[arixmkii]

And update on another issue, that JDK can't be located in index. It is because for 22.1.0 default graalvm tag is not populated, only these are available in index:

      "jdk@graalvm-java11": {
        "22.1.0": "tgz+https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.1.0/graalvm-ce-java11-darwin-aarch64-22.1.0.tar.gz"
      },
      "jdk@graalvm-java17": {
        "22.1.0": "tgz+https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.1.0/graalvm-ce-java17-darwin-aarch64-22.1.0.tar.gz"
      },

And here is coursier issue, that states, that currently tag is hardcoded https://github.com/coursier/coursier/issues/1789#issuecomment-658185317

So, I needed a workaround of my own index.json with content:

{
  "darwin": {
    "arm64": {
      "jdk@graalvm": {
        "22.1.0": "tgz+https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.1.0/graalvm-ce-java17-darwin-aarch64-22.1.0.tar.gz"
      }
    }
  }
}

[rosysalt]

I was able to install bloop a couple of days ago on Apple M1 chip. Below are the steps, in case it's helpful for someone:

1. Install GraalVM CE 22.2.0-dev build
2. Install bloop with Coursier

Step 1: Install GraalVM CE 22.2.0-dev build

Graal introduced a support for Apple M1 in version 22.2. As of today, that version is still in the dev status. Hence I downloaded that version from the graalvm latest dev releases

I downloaded graalvm-ce-java11-darwin-aarch64-dev.tar.gz, but I think java17 should work as well.

Then follow the Installation on macOS Platforms

After that, I should see:

 >> java -version
openjdk version "11.0.16" 2022-07-19
OpenJDK Runtime Environment GraalVM CE 22.2.0-dev (build 11.0.16+5-jvmci-22.2-b03)
OpenJDK 64-Bit Server VM GraalVM CE 22.2.0-dev (build 11.0.16+5-jvmci-22.2-b03, mixed mode)

Step 2: Install bloop

cs install bloop

And then I have this:

 >> bloop about
bloop v1.5.0-18-003e6c7b

Using Scala v2.12.15 and Zinc v1.6.0
Running on Java JDK v11.0.16 (/Library/Java/JavaVirtualMachines/graalvm-ce-java11-22.2.0-dev/Contents/Home)
  -> Supports debugging user code, Java Debug Interface (JDI) is available.
Maintained by the Scala Center and the community.

[lloydmeta]

Interesting! I think we would require a github runner for Apple Silicon to create binary, but maybe it's possible to produce it also using coursier?

Looks like this is now possible?

https://github.blog/changelog/2023-10-02-github-actions-apple-silicon-m1-macos-runners-are-now-available-in-public-beta/

[tgodzik]

Looks like it's only for large runners, can we even use them in open source projects?

[lloydmeta]

I guess it ultimately depends on the team/Org

https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners

Larger runners are only available for organizations and enterprises using the GitHub Team or GitHub Enterprise Cloud plans.

[tgodzik]

Looks like we would need to wait still a bit to enable it for Bloop or try to generate the image in another org where we have a self hosted runner.

[kraktus]

Imho a more reliable way would be to push for https://github.com/coursier/coursier/issues/2301 (moving coursier to brew instead of a tap), allowing bloop to be installed via brew without tap.

Brew infrastructure has plenty of Mac runners to bottle formulas (build binaries), and is very trusted and comptent to maintain package in working condition.