Puppet for elastic stack 7.1

[tamama]

Greetings,

May I ask if this puppet module is compatibile with Elastic stack 7 (.1) ?

Many thanks.

Kind regards, Tama Ma

[bryanpearson]

Also waiting on a compatible ES 7.* version.

[devinmatte]

Is this currently compatible with 7.x? Has anyone tried setting the version to => 7.0 with any success?

[edupr91]

I tried with 7.2 but the puppet module is focused on setting up multiple elastic search instances. But the last package (at least after installing from RPM) only works with one instance. Or at least the script that runs elastic won't retrieve correctly the specific conf path that the module setup.

[bryanpearson]

The problem we had is the module is outputting some config options that aren't compatible with the Kibana 7.0 Upgrade Assistant, https://www.elastic.co/guide/en/elasticsearch/reference/7.0/breaking-changes-7.0.html#tls-setting-fallback

Specifically in the instance.pp file: 'xpack.ssl.keystore.path' => $_keystore_path, 'xpack.ssl.keystore.password' => $keystore_password,

[sferry]

I managed to deploy one cluster 7.1.1 with 2 nodes (i.e. 2instances) without xpack nor searchguard. I had to override pretty much every default lines using jvm_options => [ ], with entries provided by the rpm. One of the 2 nodes is also a kibana endpoint and was deployed with elastic/puppet-kibana

[sticky-note]

@sferry Thanks for sharing. Which vars had you yo change ? I've seen already datadir environment var name not correct but had no time to dig into. I'll test that tomorrow on three 7.2 nodes, 3 master including 2 data and will edit this messages after my experience. Maybe we can PR some early stage modification ?

[sferry]

@sticky-note from the rpm see below [1] I tried modifying one by one variable, looking at errors; pretty much every '*GC*' variables and "Print*" But I gave up ( I don't know exactly all problematic variable). So in the end I just copied all of them. For a PR it would be in template/etc/elasticsearch/jvm.options.erb But it should be backward compatible.

[1] -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=${ES_TMPDIR} -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log 8:-XX:+PrintGCDetails 8:-XX:+PrintGCDateStamps 8:-XX:+PrintTenuringDistribution 8:-XX:+PrintGCApplicationStoppedTime 8:-Xloggc:logs/gc.log 8:-XX:+UseGCLogFileRotation 8:-XX:NumberOfGCLogFiles=32 8:-XX:GCLogFileSize=64m 9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m 9-:-Djava.locale.providers=COMPAT

[smasa90]

Thanks for you help, I ended up with this PR to provision the bare minimum to have a cluster running with ES 7.x and not break backward compatibility, I Think.. https://github.com/elastic/puppet-elasticsearch/pull/1040

[initharrington]

Any update on this or if #1040 will be merged?

[pw64]

Please, a working puppet module for ES7 (just one; not multiple ES instances) would be great, together with an working SSL implementation. Without SSL, the current version it works.

[rwaffen]

Any updates here? puppet-elastic-stack is already at 7.x ... so what's going on? :) I'm eagerly waiting to use 7.x, but not without puppet support.

[uberjew666]

I have managed to get ELK 7.4.1 working using this module with xpack security running. I've updated my fork with all the requirements. Due to xpack requiring encryption between all instances in a cluster, I've added a new parameter called elasticsearch::xpack. This will apply all xpack.security settings but requires ca_certificate, certificate and private_key parameters set.

[rwaffen]

@uberjew666 thanks for the great work! When will you do a PR?

[uberjew666]

@rwaffen Soon, just tidying my commits and confirming all the tests pass. Hopefully this week.

[fatmcgav]

Hi there,

Thank you for your patience on this issue. I just wanted to post a quick update to say that support for 7.x IS coming.

More details on how etc can be found here: https://github.com/elastic/puppet-elasticsearch/issues/1068

[fatmcgav]

Copying the update from https://github.com/elastic/puppet-elasticsearch/issues/1068#issuecomment-750602680:

Apologies again for the lack of progress here... Other priorities took over...

With that being said, I'd like to announce that version 7.0.0 of the module has just been published ot the Puppet Forge. As always, as this is a major version bump with a number of breaking changes, please PLEASE test this before using it in production.

I'd also like to call out the Warning from the README - This module DOES NOT support upgrading or migrating existing multi-instance deployments! I say again, if you run this against an existing installation, it will most likely BREAK THINGS!!!

Our current recommendation is to deploy Elasticsearch to new nodes and migrate the data over. You might be able to deploy Elasticsearch using the new version of this module and point it at an existing data directory; however that hasn't been tested as yet by us.

We're planning on trying to make this story better, however I'm honestly not sure when that will happen.

Feel free to reach out with any issues or questions, and thanks again for everyone's patience and comments on this issue.

Hope everyone has a good festive period and the computer gods are kind!