octocatalog-diff preserve-environments using r10k deployed environments not working

Description of problem

What did you do? I am trying to run octocatalog-diff on a puppetserver running puppet 4.10.12 with puppet environments managed by r10k in the following structure: `/etc/puppetlabs/puppet
- environments
  - production
    - environment.conf
    - manifests
      - site.pp
    - modules
    - site
  - staging
    - environment.conf
    - manifests
      - site.pp
    - modules
    - site
  - testing_branch
    - environment.conf
    - manifests
      - site.pp
    - modules
    - site` I am trying to run octocatalog-diff against the checked out code using the --preserve-environments option as follows:
What happened? octocatalog-diff exits with an error code 1, expecting a basedir configuration specified, which isn't required as per documentation on advanced environments
What did you expect to happen? I expected octocatalog-diff to create and compare catalogues created from the two r10k deployed environments. Puppet agent runs against both environments work as expected.
How can someone reproduce the problem? Create two different working puppet environments and run octocatalog-diff as explained above

Command used and debugging output

octocatalog-diff -n [nodename] --bootstrapped-from-dir /etc/puppetlabs/puppet --bootstrapped-to-dir /etc/puppetlabs/puppet --from-environment production --to-environment testing_branch --preserve-environments D, [2018-06-13T17:07:41.649263 #24328] DEBUG -- : Running octocatalog-diff 1.5.3 with ruby 2.1.5 D, [2018-06-13T17:07:41.649465 #24328] DEBUG -- : Command line arguments: ["-n", "node_name", "--bootstrapped-from-dir", "/etc/puppetlabs/puppet", "--bootstrapped-to-dir", "/etc/puppetlabs/puppet", "--from-environment", "production", "--to-environment", "testing_branch", "--preserve-environments", "-d"] D, [2018-06-13T17:07:41.649552 #24328] DEBUG -- : Running on host puppetserver_name (x86_64-linux-gnu) /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in fetch': key not found: :basedir (KeyError) from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in branch_sha' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:112:in git_repo_checkout_bootstrap' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:49:in run' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/cli.rb:106:in cli' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/bin/octocatalog-diff:34:in <top (required)>' from /usr/local/bin/octocatalog-diff:23:in load' from /usr/local/bin/octocatalog-diff:23:in <main>'

Platform and version information

Your OS: Debian Linux 8 (Jessie)
Your Ruby version: ruby 2.1.5p273 (2014-11-13) [x86_64-linux-gnu]
Your version of Puppet: puppet 4.10.12 OSE
Your version of octocatalog-diff: octocatalog-diff 1.5.3

Do the tests pass from a clean checkout? yes

Anything else to add that you think will be helpful? When I add the -f and -t options, the catalog compiles, but then I get errors about custom providers not found: command: octocatalog-diff -n node_name --bootstrapped-from-dir /etc/puppetlabs/puppet/environments/production --bootstrapped-to-dir /etc/puppetlabs/puppet/environments/testing_branch --from-environment production --to-environment testing_branch -f production -t testing_branch -d catalog error: Error: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper Error: Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper at /tmp/ocd-ipc-20180613-28056-1iximb4/ocd-builddir-20180613-28061-30pfww/environments/testing_branch/modules/monolith/quotaman/manifests/config.pp:80:3 on node quotaman-staging3.cpt3.host-h.net Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper at /tmp/ocd-ipc-20180613-28056-1iximb4/ocd-builddir-20180613-28061-30pfww/environments/testing_branch/modules/monolith/quotaman/manifests/config.pp:80:3 on node quotaman-staging3.cpt3.host-h.net Error: Failed to compile catalog for node quotaman-staging3.cpt3.host-h.net: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper at /tmp/ocd-ipc-20180613-28056-1iximb4/ocd-builddir-20180613-28061-30pfww/environments/testing_branch/modules/monolith/quotaman/manifests/config.pp:80:3 on node node_name

Jun 13 '18 15:06 walkleyn

@walkleyn can you paste the complete commandline you ouse and the complete output of octocatalog-diff?

Jun 13 '18 18:06 vinzent

@vinzent here you go: octocatalog-diff -n [nodename] --bootstrapped-from-dir /etc/puppetlabs/puppet --bootstrapped-to-dir /etc/puppetlabs/puppet --from-environment production --to-environment testing_branch --preserve-environments D, [2018-06-13T17:07:41.649263 #24328] DEBUG -- : Running octocatalog-diff 1.5.3 with ruby 2.1.5 D, [2018-06-13T17:07:41.649465 #24328] DEBUG -- : Command line arguments: ["-n", "node_name", "--bootstrapped-from-dir", "/etc/puppetlabs/puppet", "--bootstrapped-to-dir", "/etc/puppetlabs/puppet", "--from-environment", "production", "--to-environment", "testing_branch", "--preserve-environments", "-d"] D, [2018-06-13T17:07:41.649552 #24328] DEBUG -- : Running on host puppetserver_name (x86_64-linux-gnu) /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in fetch': key not found: :basedir (KeyError) from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in branch_sha' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:112:in git_repo_checkout_bootstrap' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:49:in run' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/cli.rb:106:in cli' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/bin/octocatalog-diff:34:in <top (required)>' from /usr/local/bin/octocatalog-diff:23:in load' from /usr/local/bin/octocatalog-diff:23:in <main>'

Jun 13 '18 19:06 walkleyn

Hello !

We are facing the same issue with the elastic_search module (https://github.com/elastic/puppet-elasticsearch) v6.3.0 ..

Error: Failed to compile catalog for node xxx9-8.domain.io: 
Evaluation Error: Error while evaluating a Virtual Query, Could not autoload puppet/type/elasticsearch_user_file: 
Could not autoload puppet/provider/elasticsearch_user_file/xpack: 
cannot load such file -- puppet/provider/elastic_parsedfile at /tmp/ocd-ipc-20180910-186-kdxjnr/ocd-builddir-20180910-191-g6wb0/environments/production/modules/elasticsearch/manifests/init.pp:545:6 on node xxx9-8.domain.io

CMD

cmd: octocatalog-diff \
  --retry-failed-catalog 1 -o /dev/null \
  --hostnamexxx9-8.domain.io \
  --command_line --certname\=xxxx \
  --command_line --confdir\=/home/xxxx/.puppetlabs/etc/puppet \
  --catalog-only --debug

Platform and version information

Your OS: Debian Linux 8 (Jessie) Your Ruby version: ruby 2.4.4 Your version of Puppet: puppet 4.10.12 OSE Your version of octocatalog-diff: octocatalog-diff 1.5.3

I had to do ugly things to work around:

$ git diff modules/elasticsearch/
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb
index 2acf877369..af6af98375 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_user_command'
+#require 'puppet/provider/elastic_user_command'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_user_command')
 
 Puppet::Type.type(:elasticsearch_user).provide(
   :elasticsearch_users,
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb
index f900f51cfd..983629f402 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_user_command'
+#require 'puppet/provider/elastic_user_command'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_user_command')
 
 Puppet::Type.type(:elasticsearch_user).provide(
   :esusers,
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb
index d078a48cee..e6122eae76 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_user_command'
+#require 'puppet/provider/elastic_user_command'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_user_command')
 
 Puppet::Type.type(:elasticsearch_user).provide(
   :users,
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb
index a160bea7de..23fb0f30f7 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_parsedfile'
+#require 'puppet/provider/elastic_parsedfile'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_parsedfile')
...

Sep 10 '18 16:09 xakraz