hiera-eyaml
hiera-eyaml copied to clipboard
Puppet-Server 4.1.0: Cannot load backend eyaml: cannot load such file
Im trying to install the latest opensource puppet and am struggling with getting eyaml to work getting this error when i debug the puppetserver:
Notice: hiera(): Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend
Im using RHEL 7.1 ive installed hiera-eyaml using both the puppetserver gem installer and the default system gem installer
any help would be appreciated
/opt/puppetlabs/server/bin/puppetserver gem list
*** LOCAL GEMS ***
deep_merge (1.0.1) hiera-eyaml (2.0.8) highline (1.6.21) jar-dependencies (0.1.13) jruby-openssl (0.9.7 java) json (1.8.0 java) rake (10.1.0) rdoc (4.1.2) trollop (2.1.2)
[root@.....]# gem list
*** LOCAL GEMS ***
bigdecimal (1.2.0) colored (1.2) cri (2.6.1) deep_merge (1.0.1) faraday (0.9.1) faraday_middleware (0.9.1) faraday_middleware-multi_json (0.0.6) hiera-eyaml (2.0.8) highline (1.6.21) io-console (0.4.2) json (1.7.7) json_pure (1.8.2) log4r (1.1.10) multi_json (1.11.0) multipart-post (2.0.0) psych (2.0.0) puppetclassify (0.1.0) r10k (1.5.1) rdoc (4.0.0) rgen (0.7.0, 0.6.6) semantic_puppet (0.1.1) trollop (2.1.2)
Do you mean the eyaml tool?
In order to use eyaml your best bet is to run a 'dual' installation. Install once outside of puppetserver (using a standard installation of Ruby) to support the editing of files (typically on your workstation). Install a second time inside puppetserver for enabling hiera to lookup and decrypt eyaml files.
I am using both the system ruby gem installer and the contained puppetserver version eg
- /bin/gem install hiera-eyaml
- /opt/puppetlabs/bin/puppetserver gem install hiera-eyaml
Packages installed
[root@puppet puppet]# rpm -qa|grep -e 'puppet\|hiera'
puppet-agent-1.1.0-1.el7.x86_64
puppetdb-terminus-2.3.4-1.el7.noarch
puppetdb-2.3.4-1.el7.noarch
puppetlabs-release-pc1-0.9.2-1.el7.noarch
puppetserver-2.1.0-1.el7.noarch
hiera version
[root@puppet puppet]# /opt/puppetlabs/bin/hiera --version
2.0.1
Hiera.yaml
:backends:
- eyaml
#- yaml
:hierarchy:
- ... removed
:eyaml:
:datadir: /etc/puppetlabs/puppet/environments/%{::environment}/hieradata
:pkcs7_private_key: /etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
:pkcs7_public_key: /etc/puppetlabs/puppet/keys/public_key.pkcs7.pem
:extension: 'yaml'
:merge_behavior: deeper
Command run
[root@puppet puppet]# /bin/gem install hiera-eyaml
Successfully installed hiera-eyaml-2.0.8
Parsing documentation for hiera-eyaml-2.0.8
1 gem installed
[root@puppet puppet]# /opt/puppetlabs/bin/puppetserver gem install hiera-eyaml
Successfully installed hiera-eyaml-2.0.8
1 gem installed
also note I am able to successfully edit eyaml files; its puppetserver which is having issues
puppet master --no-daemonize --verbose --debug
..
.. lots of output ..
..
Notice: hiera(): Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend
ok i got it working in a hacky way ... synlinking the directories
there is probably a new way to install gems in puppetserver 4
Workaround
cd /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/hiera/backend
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/hiera-eyaml-2.0.8/lib/hiera/backend/eyaml_backend.rb eyaml_backend.rb
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/hiera-eyaml-2.0.8/lib/hiera/backend/eyaml eyaml
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/hiera-eyaml-2.0.8/lib/hiera/backend/eyaml.rb eyaml.rb
cd /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/highline-1.6.21/lib/highline.rb highline.rb
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/highline-1.6.21/lib/highline highline
Results
Debug: hiera(): Using Hiera 1.x backend API to access instance of class Hiera::Backend::Eyaml_backend. Lookup recursion will not be detected
Debug: hiera(): [eyaml_backend]: Set option: datadir = /etc/puppetlabs/puppet/environments/feature_base_development/hieradata
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_private_key = /etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_public_key = /etc/puppetlabs/puppet/keys/public_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Set option: extension = yaml
Debug: hiera(): [eyaml_backend]: Looking up classes in eYAML backend
Debug: hiera(): [eyaml_backend]: Looking for data source nodes/puppet.management
there is probably a new way to install gems in puppetserver 4
Not that I'm aware of. It's interesting that this worked.
What works for me, running puppetserver v4.1 under CentOS 7, is to confine encrypted hiera files to default *.eyaml extension, unencrypted hiera in *.yaml files.
Here is my /etc/hiera.yaml, which is a symlink to /etc/puppetlabs/code/hiera.yaml:
---
:backends:
- yaml
- eyaml
:logger: console
:yaml:
:datadir: /etc/puppetlabs/code/environments/%{::environment}/hiera
:eyaml:
:datadir: /etc/puppetlabs/code/environments/%{::environment}/hiera
:pkcs7_private_key: /etc/puppetlabs/secure/private_key.pkcs7.pem
:pkcs7_public_key: /etc/puppetlabs/secure/public_key.pkcs7.pem
:hierarchy:
# load encrypted parameters first to ensure their priority
- secure
- "fqdn/%{::fqdn}"
- "virtual/%{::virtual}"
- "operatingsystem/%{::operatingsystem}"
- common
:merge_behavior: deeper
I put keys that need priority in secure.eyaml, as mentioned in the comment above. Also, I only installed hiera-eyaml into puppet's ruby stack, but then symlinked it to /opt/puppetlabs/bin (which is in the system path):
# /opt/puppetlabs/puppet/bin/gem install hiera-eyaml
# ln -s /opt/puppetlabs/puppet/bin/eyaml /opt/puppetlabs/bin/eyaml
Actually, to follow-up, the gem installation method in my previous comment apparently only works with the Apache Passenger puppetserver (and/or with manual invocations of "puppet apply").
On a JVM-based puppetserver I just stood up under CentOS, I saw the same problem NetworkBytes describes above. It looks like the appropriate way to install gems into a JVM-based puppetserver is indeed as the README says:
# puppetserver gem install hiera-eyaml
At least, this is what works for me under CentOS 7.0 running puppetserver-2.1.1-1.el7.noarch.
Hey @NetworkBytes, is this still an issue for you? It's strange that puppet server is installing the gems in /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems
but looking for them in /opt/puppetlabs/puppet/lib/ruby/vendor_ruby
Hi @TomPoulton There have been a few reports of this issue in IRC over the last couple weeks too, from dwittenberg (PE) and djstorm (puppetserver) last night. Looks like this is still an issue.
I had this issue with puppet 4 on centos 7.1, I installed the gem as follows:
/opt/puppetlabs/opt/puppetlabs/puppet/bin/gem install hiera-eyaml
Using the below worked for me:
/opt/puppetlabs/bin/puppetserver gem install -p http://proxy.co.za:8080 hiera-eyaml
I was having this issue with puppetserver 2.2.1 on Ubuntu 14.04. My issue was that:
/opt/puppetlabs/bin/puppetserver gem install hiera-eyaml
did not install the hiera-eyaml
dependencies. I had to manually install trollop
/opt/puppetlabs/bin/puppetserver gem install trollop
and a specific version of highline
/opt/puppetlabs/bin/puppetserver gem install highline -v 1.6.19
Before I installed those two gems manually, I was seeing errors in the puppetserver logs:
[puppet-server] Puppet hiera(): Cannot load backend eyaml: Could not find 'highline' (~> 1.6.19) - did find: [highline-1.7.8]
I'm still running into other issues with version 2.2.1, but installing those two gems at least got me past the "Cannot load backend eyaml" errors. I'm going to revert to 2.1.2 for now. I'll report back here whether or not it works.
Looks like my issue was due to the way I was scripting the installation of the gems. Now that I've got that sorted both 2.2.1 and 2.1.2 are working without issue.
I'm running into this same issue, though it only occurred after a reboot. Last night I installed hiera-eyaml, as suggested, both as gem and through puppetserver. It worked swimmingly! This morning I booted up my VMs again and now it fails with the error as indicated.
Error: Evaluation Error: Error while evaluating a Function Call, Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend at /etc/puppetlabs/code/environments/production/modules/boks/manifests/config.pp:21:16 on node client2.broehaha.nl
Error: Evaluation Error: Error while evaluating a Function Call, Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend at /etc/puppetlabs/code/environments/production/modules/boks/manifests/config.pp:21:16 on node client2.broehaha.nl
Error: Failed to compile catalog for node client2.broehaha.nl: Evaluation Error: Error while evaluating a Function Call, Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend at /etc/puppetlabs/code/environments/production/modules/boks/manifests/config.pp:21:16 on node client2.broehaha.nl
Mind you, I'm running this on the Learning Puppet VM as provided by Puppet Labs.
Unlike NetworkBytes' case (108646608), adding those symlinks did not alleviate the issue for me. Interestingly, Puppet installed an older version for me (2.0.6) when I asked it to get the gem. That's interesting.
In the end, I found my solution in another thread, in reply 196825646 by dogfish182. It turns out that a simple "gem install hiera-eyaml" doesn't cut it on the Learning Puppet VM! Because there is yet another "gem" command over there! I did "gem install hiera-eyaml", while running "/opt/puppetlabs/puppet/bin/gem install hiera-eyaml" seemingly fixed it for me. Funny then, that I have to run it three times! "gem install", "puppet/bin/gem install" and "puppetserver gem install" :)
If I'm not mistaken you do it 3 times because you have 3 use cases:
- gem install hiera-eyaml: get it to work just on your system
- /opt/puppetlabs/puppet/bin/gem install hiera-eyaml to get it to work with the puppet agent, so e.g.' puppet lookup works.
- puppetserver gem install to get it to work in the puppetserver
I have it working with just hiera-eyaml installed on my local system (to do the encryption of vars) and via puppetserver gem install. puppet-agent does not need it as the yaml is decoded before it's sent to the client when it's needed (as determined by puppetserver)
@rgevaert everyonce in a while i come across a life changing tip. This was one of them. THANK YOU