Install elasticsearch with Homebrew on OSX Ventura

[pioz]

Elasticsearch Version

7.17.4

Installed Plugins

No response

Java Version

bundled

OS Version

Mac OS X Ventura

Problem Description

I'm trying to install Elasticsearch using this guide.

When I start Elasticsearch with the command brew services start elasticsearch-full I see this window alert:

https://i.imgur.com/JRHt2xm.png

and in the log file I see this message:

/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: 10827 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker

Elasticsearch is not running.

Steps to Reproduce

Run these commands:

brew tap elastic/tap
brew install elastic/tap/elasticsearch-full
brew services start elasticsearch-full

Logs (if relevant)

No response

[francoisdelage]

+1

[nishant04412]

+1

[mitzanoo]

same here - tried different methods but still getting the same issue + now I am getting: warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME

[elasticsearchmachine]

Pinging @elastic/es-delivery (Team:Delivery)

[cyanicr]

@pioz As a temporary fix, I've managed to install it via Docker.

If you need old data from homebrew, this is the command I ran after giving Docker permissions to access the /opt folder (Settings -> Resources -> File sharing).

docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms1g -Xmx1g" --name elastic --mount type=bind,source=/opt/homebrew/var/lib/elasticsearch,target=/usr/share/elasticsearch/data elasticsearch:7.17.4

[pioz]

@pioz I've managed to install it via Docker.

If you need old data from homebrew, this is the command I ran after giving Docker permissions to access the /opt folder (Settings -> Resources -> File sharing).

docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms1g -Xmx1g" --name elastic --mount type=bind,source=/opt/homebrew/var/lib/elasticsearch,target=/usr/share/elasticsearch/data elasticsearch:7.17.4

If possible, I'd rather not use Docker.

[cyanicr]

I've tried all day to make it work via homebrew, but with no luck. That's why I chose Docker instead. You could also try using the archives if you don't prefer Docker.

[cmoen]

I'm also having problems installing and I'm getting this error when I do brew install elasticsearch-full:

Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).

==> Downloading https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.4-darwin-x86_64.tar.gz?
Already downloaded: /Users/cm/Library/Caches/Homebrew/downloads/69a42f80a7a14f892ddcb7d63a53c1ccaecce67dc8ffa0ac125be8f31241b01e--elasticsearch-7.17.4-darwin-x86_64.tar.gz
==> Installing elasticsearch-full from elastic/tap
Error: Your Xcode (14.0.1) is too outdated.
Please update to Xcode 14.1 (or delete it).
Xcode can be updated from the App Store.

Error: Your Command Line Tools (CLT) does not support macOS 13.
It is either outdated or was modified.
Please update your Command Line Tools (CLT) or delete it if no updates are available.
Update them from Software Update in System Preferences or run:
  softwareupdate --all --install --force

If that doesn't show you any updates, run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/all/.
You should download the Command Line Tools for Xcode 14.1.

Xcode 14.1 hasn't been released yet (GA). Is there some of dependency error here? Is there any reason Xcode 14.1 should be required?

[gauravcastingnetworks]

+1 I have MacBook Pro with macOS Ventura 13.0 (22A380) after installing the elastic search full I found the same issue.

When I try to start the elastic search it shows jd.app is damaged and can't be opened.
I allowed jdk.app from Privacy & Security setting but still same issue persist.

➜  ~ java -version
java version "19.0.1" 2022-10-18
Java(TM) SE Runtime Environment (build 19.0.1+10-21)
Java HotSpot(TM) 64-Bit Server VM (build 19.0.1+10-21, mixed mode, sharing)
➜  ~

➜  ~ brew services list
Name                  Status     User         File
apm-server-full       none
elasticsearch-full    error  256 gauravsharma ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch-full.plist
filebeat-full         none
kibana-full           none

xcode version ➜ ~ xcode-select -version xcode-select version 2396.

[swar30]

Having same issue.

can't start services when using brew services start elastic/tap/elasticsearch-full it shows "jdk" is damaged ... error and clicking "Open anyway" on the warning in "Privacy & Security" settings screen does not help.

Looking in /opt/homebrew/var/log/elasticsearch.log I see some warnings about JDK, I presume related to the system message, which I assume blocks access to JDK.

/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83:  2094 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker
/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java: No such file or directory
could not find java in bundled JDK at /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java
could not find java in bundled JDK at /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java
could not find java in bundled JDK at /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java
/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: 13819 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker
/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: 14175 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker
/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: 14791 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker
could not find java in bundled JDK at /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java
could not find java in bundled JDK at /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java
/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: 21346 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker
/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: 22293 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker
/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/bin/elasticsearch-env: line 83: 22454 Killed: 9               "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.java_version_checker.JavaVersionChecker

As work around I see I can run it not as services rather directly from command line by running elasticsearch, I assume that when I do this it will run with my locally installed java instead of the one bundled with the elastic deployment which most likely bypasses the JDK issue all together but will require default JDK to align with one used by ES.

[LMCom]

When I try to start the elastic search it shows jd.app is damaged and can't be opened. I allowed jdk.app from Privacy & Security setting but still same issue persist.

Same problem here. I also have the same messages in the log. I looked at line 83 of elasticearch-env and logged the command that is executed there. On my system it is:

/opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java -Xshare:auto -cp /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/lib/* org.elasticsearch.tools.java_version_checker.JavaVersionChecker

[jeanschdev]

+1

[harshaanNihal]

+1

[redouanhajjari]

I have also issues like could not find java in bundled JDK at /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app/Contents/Home/bin/java I have uninstall elasticsearch-full and now i cant install it any more because he version of MacOs is to High (13).

[LMCom]

It would be reasonable to give this bug a high priority. Developers need to be able to run Elasticsearch to continue developing applications that are using it. This bug is currently blocking our production releases, too. I just want to point this out because people here seem to be quite relaxed when panic would be an appropriate reaction.

[LeashFox]

I'm also affected by this and have a few comments to add to this thread:

Upgrading to MacOS 13 Ventura broke Xcode Command Line Tools, thus breaking git, forcing me to install Xcode 14.1 Release Candidate 2 as suddenly my Macbook was incompatible with the publicly available Xcode 14.

I've reinstalled "elasticsearch-full" via homebrew a variety of times along with many versions of Java & openJDK. I'm also unable to add "jdk.app" to the Developer Tools under System Settings > Privacy & Security. In addition, I tried adding export PATH=[location/of/java/or/openjdk] to my ~/.zshrc profile, yet I later found out this has no affect as it's trying to load jdk.app located in /opt/homebrew/Cellar/elasticsearch-full/7.17.4/libexec/jdk.app (or at least, for me it was located there.

Furthermore, another odd issue I had with this was that Rosetta 2 was automatically removed from my Macbook; I'm hoping this is an anomaly, yet annoying none-the-less.

The only thing I'm able to do to combat this issue is to use Elasticsearch via Docker, or use the all-forbidden adversary "OpenSearch."

Fingers crossed this will be sorted soon; I don't want to use an alternative to Elasticsearch as I'd rather not spend time figuring out why functionality on my sites are suddenly borked.

[DutchPrince]

+1

[DavidAmyot]

+1

[Cryptunas]

+1

[andrewobrien]

I've had success installing from an archive as a temporary workaround - you may need to modify the elasticsearch.yml security settings to work for you locally https://www.elastic.co/guide/en/elasticsearch/reference/current/targz.html

[redouanhajjari]

I've had success installing from an archive as a temporary workaround - you may need to modify the elasticsearch.yml security settings to work for you locally https://www.elastic.co/guide/en/elasticsearch/reference/current/targz.html

there are different solutions to fix this. but the problem is with elastic/tap/elasticsearch-full.

[skinnynpale]

+1

[LMCom]

Why have I never heard about OpenSearch before? It is a totally free and fully compatible fork of Elasticsearch. It has a Homebrew formula which is verified to run on macOS Ventura: https://formulae.brew.sh/formula/opensearch. Did anyone try it out? I might do soon, since I cannot afford stopping development for so long.

And here's a migration guide from existing Elasticsearch to OpenSearch which seems pretty simple: https://opensearch.org/docs/latest/upgrade-to/upgrade-to/

[LeashFox]

@LMCom, I'm currently using this as a last resort until Elasticsearch is fixed. In fairness, I've had no issues with OpenSearch at the moment.

Personally, I'm unable to replace Elasticsearch with OpenSearch from here-on-out in my staging/live environments as my web host requires Elasticsearch (something to do with compliance.)

In addition, I'm worried that I'll encounter issues that could arise from using OpenSearch which would lead me in the direction of trying to fix something only present on my local environment; I prefer matching the configuration between my dev environment and staging/live environments.

Regardless, if you're able to use OpenSearch in your environment(s) then go for it 🤙🏼

[ur001]

Install OpenJDK

brew install openjdk

Find path to java home

>> /usr/libexec/java_home
/Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home

Edit elasticsearch/kibana .plists to use OpenJDK instead of bundled JDK

/usr/local/Cellar/kibana-full/7.17.4/homebrew.mxcl.elasticsearch-full.plist
/usr/local/Cellar/kibana-full/7.17.4/homebrew.mxcl.kibana-full.plist

add lines:

<key>EnvironmentVariables</key>
<dict>
  <key>ES_JAVA_HOME</key>
  <string>/Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home</string>
</dict>

Work! :)

[LMCom]

@ur001 Thank you, that is a big part of the solution! First, my Elasticsearch still showed an error and crashed:

uncaught exception in thread [main] ElasticsearchException[Failure running machine learning native code. This could be due to running on an unsupported OS or distribution, missing OS libraries, or a problem with the temp directory. To bypass this problem by running Elasticsearch without machine learning functionality set [xpack.ml.enabled: false].]

So I set xpack.ml.enabled: false in my config /opt/homebrew/etc/elasticsearch/elasticsearch.yml. Then I restarted with brew services restart elasticsearch-full and it works. :)

[yavuz]

thanks @ur001 and @LMCom.

• Add this lines to /opt/homebrew/Cellar/elasticsearch-full/7.17.4/homebrew.mxcl.elasticsearch-full.plist

   <key>EnvironmentVariables</key>
    <dict>
      <key>ES_JAVA_HOME</key>
      <string>/opt/homebrew/Cellar/openjdk/20.0.1/libexec/openjdk.jdk/Contents/Home</string>
    </dict>

• add this line to /opt/homebrew/etc/elasticsearch/elasticsearch.yml

xpack.ml.enabled: false

• optional: add this line to ~/.bash_profile

export ES_JAVA_HOME=$(/usr/libexec/java_home)

• for test curl -s http://localhost:9200

---

note: if works elasticsearch, kill process elasticsearch. may be you should reboot.

ps -ax | grep "elasticsearch"

[laurnts]

Following @yavuz post

Just to be clear the file that needed to be updated and the out come /opt/homebrew/Cellar/elasticsearch-full/7.17.4/homebrew.mxcl.elasticsearch-full.plist

   <key>EnvironmentVariables</key>
    <dict>
      <key>ES_JAVA_HOME</key>
      <string>/opt/homebrew/Cellar/openjdk/19/libexec/openjdk.jdk/Contents/Home</string>
    </dict>

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>KeepAlive</key>
    <false/>
    <key>Label</key>
    <string>homebrew.mxcl.elasticsearch-full</string>
    <key>ProgramArguments</key>
    <array>
      <string>/opt/homebrew/opt/elasticsearch-full/bin/elasticsearch</string>
    </array>
    <key>EnvironmentVariables</key>
    <dict>
    </dict>
    <key>RunAtLoad</key>
    <true/>
    <key>WorkingDirectory</key>
    <string>/opt/homebrew/var</string>
    <key>StandardErrorPath</key>
    <string>/opt/homebrew/var/log/elasticsearch.log</string>
    <key>StandardOutPath</key>
    <string>/opt/homebrew/var/log/elasticsearch.log</string>
    <key>EnvironmentVariables</key>
    <dict>
      <key>ES_JAVA_HOME</key>
      <string>/opt/homebrew/Cellar/openjdk/19/libexec/openjdk.jdk/Contents/Home</string>
    </dict>
  </dict>
</plist>

And also add this line to /opt/homebrew/etc/elasticsearch/elasticsearch.yml xpack.ml.enabled: false

Works !

[redouanhajjari]

openjdk thanks @ur001 and @LMCom.

• Add this lines

   <key>EnvironmentVariables</key>
    <dict>
      <key>ES_JAVA_HOME</key>
      <string>/opt/homebrew/Cellar/openjdk/19/libexec/openjdk.jdk/Contents/Home</string>
    </dict>

to file

/opt/homebrew/Cellar/elasticsearch-full/7.17.4/homebrew.mxcl.elasticsearch-full.plist

• add this line to /opt/homebrew/etc/elasticsearch/elasticsearch.yml

xpack.ml.enabled: false

• optional

~./bash_profile

add this line

export ES_JAVA_HOME=$(/usr/libexec/java_home)

+1

Works!! You saved my day!

[jbutton]

Thanks @yavuz I had a slightly different file locations though, for the reference:

1. add this to /usr/local/Cellar/elasticsearch-full/7.17.4/homebrew.mxcl.elasticsearch-full.plist

<key>EnvironmentVariables</key>
    <dict>
       <key>ES_JAVA_HOME</key>
        <string>/usr/local/Cellar/openjdk/19/libexec/openjdk.jdk/Contents/Home</string>
        ...

2. add this line to /usr/local/etc/elasticsearch/elasticsearch.yml xpack.ml.enabled: false
3. reboot

Works!