Builds on Mac M1 imposible

[tawalaya]

Environment details:

• Macbook Pro M1 (Apple Silicon)
• Idea: 2021.3.2
• docker: 20.10.12
• jdk: OpenJDK 11.0.14
• gralde: using bundled

Steps to reproduce the issue:

1. check out brand new git copy
2. ./gradlew compileScala
3. 🔥

Provide the actual results and outputs:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':core:scheduler:generateProto'.
> Could not resolve all files for configuration ':core:scheduler:protobufToolsLocator_protoc'.
   > Could not find protoc-osx-aarch_64.exe (com.google.protobuf:protoc:3.4.0).
     Searched in the following locations:
         https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.4.0/protoc-3.4.0-osx-aarch_64.exe

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.5.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 2m 21s

Additional information you deem important:

• Tried to use local protoc binary instead using protocCommand=... did not work.
• Changed build.gradle (:core:sscheduler) by adding:

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.19.4'
    }
}

This causes an already "mitigated" bug (https://github.com/akka/akka-grpc/issues/786) to reappear, adding a build/akka-grpc-gradle-plugin.lo in the working directory of the gradle daemon fixes that but that does not seem to be a reasonable solution?

Ultimately, the version of grpc:akka-grpc-gradle-plugin should be updated but that looks to require a major update of gralde.

[style95]

Today, I met this too. Obviously, this needs to be addressed. I confirmed newer version of gradle is required to upgrade the akka-grpc-gradle-plugin. And I feel it would require another significant change on codes along with changes on akka-grpc dependencies families.

[style95]

I could build OpenWhisk with an ugly workaround. This is not a good solution and this issue must be addressed, but anyone who suffers from this might take this approach.

Summary

In short, what I did to build OpenWhisk is as follows.

$ export GRADLE_OPTS=-Dorg.gradle.project.buildDir=/tmp/gradle-build
$ ./gradlew distDocker
$ mv /tmp/gradle-build core/scheduler/build
$ unset GRADLE_OPTS
$ ./gradlew distDocker -x :core:scheduler:generateProto

---

Details

The main issue was the gradle plugin could not properly recognize M1 macs. So I added the following to the core/scheduler/build.gradle. https://gist.github.com/style95/ac7bb101c64b1308e7e92830ff0a74c0/revisions#diff-49a96e7eea8a94af862798a45174e6ac43eb4f8b4bd40759b5da63ba31ec3ef7R87

It looks for protoc_platform so I added the following to the ~/.gradle/gradle.properties

protoc_platform=osx-x86_64

This is basically the same thing as what is described in the issue but the protoc version should be 3.4.0.

After then it will complain about the not found log.

* What went wrong:
Execution failed for task ':core:scheduler:generateProto'.
> protoc: stdout: . stderr: Exception in thread "main" java.io.FileNotFoundException: build/akka-grpc-gradle-plugin.log (No such file or directory)
  	at java.io.FileOutputStream.open0(Native Method)
  	at java.io.FileOutputStream.open(FileOutputStream.java:270)
  	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
  	at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
  	at java.io.PrintWriter.<init>(PrintWriter.java:192)
  	at java.io.PrintWriter.<init>(PrintWriter.java:232)
  	at akka.grpc.gen.FileLogger.<init>(Logging.scala:35)
  	at akka.grpc.gen.Main$.$anonfun$logger$1(Main.scala:48)
  	at scala.Option.map(Option.scala:163)
  	at akka.grpc.gen.Main$.delayedEndpoint$akka$grpc$gen$Main$1(Main.scala:48)
  	at akka.grpc.gen.Main$delayedInit$body.apply(Main.scala:14)
  	at scala.Function0.apply$mcV$sp(Function0.scala:39)
  	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
  	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
  	at scala.App.$anonfun$main$1$adapted(App.scala:80)
  	at scala.collection.immutable.List.foreach(List.scala:392)
  	at scala.App.main(App.scala:80)
  	at scala.App.main$(App.scala:78)
  	at akka.grpc.gen.Main$.main(Main.scala:14)
  	at akka.grpc.gen.Main.main(Main.scala)
  --akkaGrpc_out: protoc-gen-akkaGrpc: Plugin failed with status code 1.

The plugin sets the log location based on the buildDir of the project. https://github.com/akka/akka-grpc/blob/df0bd8fd9e545aadc6d0e7543a39bda61eb6a133/gradle-plugin/src/main/groovy/akka/grpc/gradle/AkkaGrpcPlugin.groovy#L66

So I changed its location to somewhere else like below and run ./gradlew distDocker.

$ export GRADLE_OPTS=-Dorg.gradle.project.buildDir=/tmp/gradle-build
$ ./gradlew distDocker

The build itself will still fail because dependency resolution is failed, but it will pass the :core:scheduler:generateProto task. And the built codes will be placed in /tmp/gradle-build. So I just move the built results to core/scheduler/build.

After then unset the GRADLE_OPTS and rerun distDocker without the core:scheduler:generateProto task.

$ mv /tmp/gradle-build core/scheduler/build
$ unset GRADLE_OPTS
$ ./gradlew distDocker -x :core:scheduler:generateProto

What I am curious about is I tried setting the buildDir as the core/scheduler/build directly, but it didn't work. It couldn't find the log file even if there was a log file. Not quite sure the reason.

Anyway, I hope this helps anyone who tries to build OpenWhisk on M1 Mac until we address this issue.

[savaskoc]

I built successfully with the modification @tawalaya did. Instead of giving another version of protobuf, I just used x86-64 artifact.

So, in core/scheduler/build.gradle

protobuf {
  protoc {
    artifact = 'com.google.protobuf:protoc:3.4.0:osx-x86_64'
  }
}