Puppet-Server 4.1.0: Cannot load backend eyaml: cannot load such file

[NetworkBytes]

Im trying to install the latest opensource puppet and am struggling with getting eyaml to work getting this error when i debug the puppetserver:

Notice: hiera(): Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend

Im using RHEL 7.1 ive installed hiera-eyaml using both the puppetserver gem installer and the default system gem installer

any help would be appreciated

/opt/puppetlabs/server/bin/puppetserver gem list

*** LOCAL GEMS ***

deep_merge (1.0.1) hiera-eyaml (2.0.8) highline (1.6.21) jar-dependencies (0.1.13) jruby-openssl (0.9.7 java) json (1.8.0 java) rake (10.1.0) rdoc (4.1.2) trollop (2.1.2)

[root@.....]# gem list

*** LOCAL GEMS ***

bigdecimal (1.2.0) colored (1.2) cri (2.6.1) deep_merge (1.0.1) faraday (0.9.1) faraday_middleware (0.9.1) faraday_middleware-multi_json (0.0.6) hiera-eyaml (2.0.8) highline (1.6.21) io-console (0.4.2) json (1.7.7) json_pure (1.8.2) log4r (1.1.10) multi_json (1.11.0) multipart-post (2.0.0) psych (2.0.0) puppetclassify (0.1.0) r10k (1.5.1) rdoc (4.0.0) rgen (0.7.0, 0.6.6) semantic_puppet (0.1.1) trollop (2.1.2)

[sihil]

Do you mean the eyaml tool?

In order to use eyaml your best bet is to run a 'dual' installation. Install once outside of puppetserver (using a standard installation of Ruby) to support the editing of files (typically on your workstation). Install a second time inside puppetserver for enabling hiera to lookup and decrypt eyaml files.

[NetworkBytes]

I am using both the system ruby gem installer and the contained puppetserver version eg

• /bin/gem install hiera-eyaml
• /opt/puppetlabs/bin/puppetserver gem install hiera-eyaml

Packages installed

[root@puppet puppet]# rpm -qa|grep -e 'puppet\|hiera'
puppet-agent-1.1.0-1.el7.x86_64
puppetdb-terminus-2.3.4-1.el7.noarch
puppetdb-2.3.4-1.el7.noarch
puppetlabs-release-pc1-0.9.2-1.el7.noarch
puppetserver-2.1.0-1.el7.noarch

hiera version

[root@puppet puppet]# /opt/puppetlabs/bin/hiera  --version
2.0.1

Hiera.yaml

:backends:
  - eyaml
  #- yaml

:hierarchy:
  - ... removed 

:eyaml:
   :datadir: /etc/puppetlabs/puppet/environments/%{::environment}/hieradata
   :pkcs7_private_key: /etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
   :pkcs7_public_key: /etc/puppetlabs/puppet/keys/public_key.pkcs7.pem
   :extension: 'yaml'

:merge_behavior: deeper

Command run

[root@puppet puppet]# /bin/gem install hiera-eyaml
Successfully installed hiera-eyaml-2.0.8
Parsing documentation for hiera-eyaml-2.0.8
1 gem installed

[root@puppet puppet]# /opt/puppetlabs/bin/puppetserver gem install hiera-eyaml
Successfully installed hiera-eyaml-2.0.8
1 gem installed

[NetworkBytes]

also note I am able to successfully edit eyaml files; its puppetserver which is having issues

puppet master --no-daemonize --verbose --debug
..
.. lots of output ..
..
Notice: hiera(): Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend

[NetworkBytes]

ok i got it working in a hacky way ... synlinking the directories

there is probably a new way to install gems in puppetserver 4

Workaround

 cd /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/hiera/backend

ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/hiera-eyaml-2.0.8/lib/hiera/backend/eyaml_backend.rb eyaml_backend.rb
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/hiera-eyaml-2.0.8/lib/hiera/backend/eyaml eyaml
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/hiera-eyaml-2.0.8/lib/hiera/backend/eyaml.rb eyaml.rb
cd /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/highline-1.6.21/lib/highline.rb highline.rb
ln -s /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems/highline-1.6.21/lib/highline highline

Results

Debug: hiera(): Using Hiera 1.x backend API to access instance of class Hiera::Backend::Eyaml_backend. Lookup recursion will not be detected
Debug: hiera(): [eyaml_backend]: Set option: datadir = /etc/puppetlabs/puppet/environments/feature_base_development/hieradata
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_private_key = /etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_public_key = /etc/puppetlabs/puppet/keys/public_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Set option: extension = yaml
Debug: hiera(): [eyaml_backend]: Looking up classes in eYAML backend
Debug: hiera(): [eyaml_backend]: Looking for data source nodes/puppet.management

[elyscape]

there is probably a new way to install gems in puppetserver 4

Not that I'm aware of. It's interesting that this worked.

[westbywest]

What works for me, running puppetserver v4.1 under CentOS 7, is to confine encrypted hiera files to default *.eyaml extension, unencrypted hiera in *.yaml files.

Here is my /etc/hiera.yaml, which is a symlink to /etc/puppetlabs/code/hiera.yaml:

---
:backends:
  - yaml
  - eyaml
:logger: console
:yaml:
  :datadir: /etc/puppetlabs/code/environments/%{::environment}/hiera
:eyaml:
  :datadir: /etc/puppetlabs/code/environments/%{::environment}/hiera
  :pkcs7_private_key: /etc/puppetlabs/secure/private_key.pkcs7.pem
  :pkcs7_public_key:  /etc/puppetlabs/secure/public_key.pkcs7.pem
:hierarchy:
# load encrypted parameters first to ensure their priority
  - secure
  - "fqdn/%{::fqdn}"
  - "virtual/%{::virtual}"
  - "operatingsystem/%{::operatingsystem}"
  - common
:merge_behavior: deeper

I put keys that need priority in secure.eyaml, as mentioned in the comment above. Also, I only installed hiera-eyaml into puppet's ruby stack, but then symlinked it to /opt/puppetlabs/bin (which is in the system path):

# /opt/puppetlabs/puppet/bin/gem install hiera-eyaml
# ln -s /opt/puppetlabs/puppet/bin/eyaml /opt/puppetlabs/bin/eyaml

[westbywest]

Actually, to follow-up, the gem installation method in my previous comment apparently only works with the Apache Passenger puppetserver (and/or with manual invocations of "puppet apply").

On a JVM-based puppetserver I just stood up under CentOS, I saw the same problem NetworkBytes describes above. It looks like the appropriate way to install gems into a JVM-based puppetserver is indeed as the README says:

# puppetserver gem install hiera-eyaml

At least, this is what works for me under CentOS 7.0 running puppetserver-2.1.1-1.el7.noarch.

[TomPoulton]

Hey @NetworkBytes, is this still an issue for you? It's strange that puppet server is installing the gems in /opt/puppetlabs/server/data/puppetserver/jruby-gems/gems but looking for them in /opt/puppetlabs/puppet/lib/ruby/vendor_ruby

[optiz0r]

Hi @TomPoulton There have been a few reports of this issue in IRC over the last couple weeks too, from dwittenberg (PE) and djstorm (puppetserver) last night. Looks like this is still an issue.

[mkrakowitzer]

I had this issue with puppet 4 on centos 7.1, I installed the gem as follows:

/opt/puppetlabs/opt/puppetlabs/puppet/bin/gem install hiera-eyaml

Using the below worked for me:

/opt/puppetlabs/bin/puppetserver gem install  -p http://proxy.co.za:8080 hiera-eyaml

[quixoten]

I was having this issue with puppetserver 2.2.1 on Ubuntu 14.04. My issue was that:

/opt/puppetlabs/bin/puppetserver gem install hiera-eyaml

did not install the hiera-eyaml dependencies. I had to manually install trollop

/opt/puppetlabs/bin/puppetserver gem install trollop

and a specific version of highline

/opt/puppetlabs/bin/puppetserver gem install highline -v 1.6.19

Before I installed those two gems manually, I was seeing errors in the puppetserver logs:

[puppet-server] Puppet hiera(): Cannot load backend eyaml: Could not find 'highline' (~> 1.6.19) - did find: [highline-1.7.8]

I'm still running into other issues with version 2.2.1, but installing those two gems at least got me past the "Cannot load backend eyaml" errors. I'm going to revert to 2.1.2 for now. I'll report back here whether or not it works.

[quixoten]

Looks like my issue was due to the way I was scripting the installation of the gems. Now that I've got that sorted both 2.2.1 and 2.1.2 are working without issue.

[tsluyter]

I'm running into this same issue, though it only occurred after a reboot. Last night I installed hiera-eyaml, as suggested, both as gem and through puppetserver. It worked swimmingly! This morning I booted up my VMs again and now it fails with the error as indicated.

Error: Evaluation Error: Error while evaluating a Function Call, Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend at /etc/puppetlabs/code/environments/production/modules/boks/manifests/config.pp:21:16 on node client2.broehaha.nl
Error: Evaluation Error: Error while evaluating a Function Call, Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend at /etc/puppetlabs/code/environments/production/modules/boks/manifests/config.pp:21:16 on node client2.broehaha.nl
Error: Failed to compile catalog for node client2.broehaha.nl: Evaluation Error: Error while evaluating a Function Call, Cannot load backend eyaml: cannot load such file -- hiera/backend/eyaml_backend at /etc/puppetlabs/code/environments/production/modules/boks/manifests/config.pp:21:16 on node client2.broehaha.nl

Mind you, I'm running this on the Learning Puppet VM as provided by Puppet Labs.

Unlike NetworkBytes' case (108646608), adding those symlinks did not alleviate the issue for me. Interestingly, Puppet installed an older version for me (2.0.6) when I asked it to get the gem. That's interesting.

In the end, I found my solution in another thread, in reply 196825646 by dogfish182. It turns out that a simple "gem install hiera-eyaml" doesn't cut it on the Learning Puppet VM! Because there is yet another "gem" command over there! I did "gem install hiera-eyaml", while running "/opt/puppetlabs/puppet/bin/gem install hiera-eyaml" seemingly fixed it for me. Funny then, that I have to run it three times! "gem install", "puppet/bin/gem install" and "puppetserver gem install" :)

[rgevaert]

If I'm not mistaken you do it 3 times because you have 3 use cases:

1. gem install hiera-eyaml: get it to work just on your system
2. /opt/puppetlabs/puppet/bin/gem install hiera-eyaml to get it to work with the puppet agent, so e.g.' puppet lookup works.
3. puppetserver gem install to get it to work in the puppetserver

[prometheanfire]

I have it working with just hiera-eyaml installed on my local system (to do the encryption of vars) and via puppetserver gem install. puppet-agent does not need it as the yaml is decoded before it's sent to the client when it's needed (as determined by puppetserver)

[devoncustard]

@rgevaert everyonce in a while i come across a life changing tip. This was one of them. THANK YOU