datafusion-comet Bad CPU type in executable protoc-jar

Describe the bug

I am trying to build Comet on a new Macbook Pro M3 Max and I am running into this error:

Cannot run program "/Users/andy/protoc5660975561245775818.exe": error=86, Bad CPU type in executable

Steps to reproduce

I installed openjdk@17 and protoc using homebrew. I am using the following command to build the project.

make release PROFILES="-Pspark-3.4"

Here is some of the log output:

Downloaded from central: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.19.6/protoc-3.19.6-osx-aarch_64.exe (12 MB at 1.6 MB/s)
protoc-jar: executing: [/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe, --version]
protoc-jar: caught exception, retrying: java.io.IOException: Cannot run program "/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe": error=86, Bad CPU type in executable

Expected behavior

No response

Additional context

No response

Mar 23 '24 01:03 andygrove

Hmm, I'm also using M3 Max, but I don't see the issue.

java --version
openjdk 11.0.22 2024-01-16
OpenJDK Runtime Environment Homebrew (build 11.0.22+0)
OpenJDK 64-Bit Server VM Homebrew (build 11.0.22+0, mixed mode)

Is it possible that you installed x86 JDK?

And, from the error message,

Downloaded from central: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.19.6/protoc-3.19.6-osx-aarch_64.exe (12 MB at 1.6 MB/s)
protoc-jar: executing: [/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe, --version]
protoc-jar: caught exception, retrying: java.io.IOException: Cannot run program "/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe": error=86, Bad CPU type in executable

Looks like it tries to run x86 protoc?

Mar 23 '24 01:03 viirya

And do you have Xcode installed?

Mar 23 '24 01:03 viirya

I do have xcode installed. I tried with openjdk 11 as well.

% java --version 
openjdk 11.0.22 2024-01-16
OpenJDK Runtime Environment Homebrew (build 11.0.22+0)
OpenJDK 64-Bit Server VM Homebrew (build 11.0.22+0, mixed mode)

It seems to be arm64 build:

% file /opt/homebrew/Cellar/openjdk@11/11.0.22/bin/javac
/opt/homebrew/Cellar/openjdk@11/11.0.22/bin/javac: Mach-O 64-bit executable arm64

I also have protoc installed and seems to be arm64 version:

 % file /opt/homebrew/bin/protoc
/opt/homebrew/bin/protoc: Mach-O 64-bit executable arm64

I still see the same issue 😕

Mar 23 '24 05:03 andygrove

Is it possible due to this https://github.com/protocolbuffers/protobuf/pull/8557?

Mar 23 '24 05:03 viirya

Do you install protobuf using Homebrew? This version protoc should be used instead of the downloaded one "/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe".

Mar 23 '24 05:03 viirya

I tried installing protobuf with homebrew as well as manually. In either case, maven runs the downloaded one, which is not really an arm64 build even though it pretends to be.

As a workaround, I ran the following comment to enable x86 compatibility mode (as suggested in https://github.com/protocolbuffers/protobuf/pull/8557) and the build now works.

softwareupdate --install-rosetta

Mar 23 '24 16:03 andygrove

Hmm, it's weird. I don't remember I've installed rosetta. @sunchao Do you have it?

Mar 23 '24 17:03 viirya

I don't think I installed Rosetta either.

Mar 23 '24 18:03 sunchao

This is no longer an issue for me and we have not had other reports of this happening, so will close this

May 24 '24 14:05 andygrove