librarian stopped working with latest minitar v1 release

[der-eismann]

It turns out that the puppet_forge gem has a dependency on minitar without a version requirement on it (https://github.com/puppetlabs/forge-ruby/blob/main/puppet_forge.gemspec#L26) and is still using the old archive/tar/minitar namespace. However a few days ago minitar had it's v1 major release, abandoning the old namespace, causing issues with librarian:

$ librarian-puppet install --clean
<internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require': cannot load such file -- archive/tar/minitar (LoadError)
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/puppet_forge-4.1.0/lib/puppet_forge/tar/mini.rb:2:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/puppet_forge-4.1.0/lib/puppet_forge/tar.rb:4:in `<class:Tar>'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/puppet_forge-4.1.0/lib/puppet_forge/tar.rb:3:in `<module:PuppetForge>'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/puppet_forge-4.1.0/lib/puppet_forge/tar.rb:2:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/puppet_forge-4.1.0/lib/puppet_forge.rb:21:in `<module:PuppetForge>'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/puppet_forge-4.1.0/lib/puppet_forge.rb:3:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet/source/forge/repo_v3.rb:2:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet/source/forge.rb:4:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet/source.rb:3:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet/dsl.rb:3:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet/environment.rb:2:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet/extension.rb:1:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet.rb:4:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/lib/librarian/puppet/cli.rb:4:in `<top (required)>'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from <internal:/opt/puppetlabs/puppet/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
	from /opt/puppetlabs/puppet/lib/ruby/gems/3.2.0/gems/librarian-puppet-5.0.0/bin/librarian-puppet:6:in `<top (required)>'
	from /opt/puppetlabs/puppet/bin/librarian-puppet:25:in `load'
	from /opt/puppetlabs/puppet/bin/librarian-puppet:25:in `<main>'

I opened https://github.com/puppetlabs/forge-ruby/pull/119 to fix this temporarily, however librarian isn't even using forge-ruby v5 and I'm not sure if/when this change will land. Maybe this helps someone else or somebody at puppet can speed things up a bit, it feels like the whole ecosystem is pretty outdated unfortunately.

My workaround so far:

gem install --no-document librarian-puppet -v '~> 5'
gem uninstall minitar -I
gem install --no-document minitar -v '0.12'

[ekohl]

I opened puppetlabs/forge-ruby#119 to fix this temporarily

This was merged and released

however librarian isn't even using forge-ruby v5 and I'm not sure if/when this change will land

There was 4bf250398bcc95c01c5c20fed8b47e45bf0431ba (which has a wrong summary). Let's create a new release that includes it.

[teluq-pbrideau]

Any chance for this new release of librarian-puppet on rubygems.org soon?

[parkr]

+1, I am having this issue as well. To work around it, I downgraded to minitar 0.12.1.

[der-eismann]

@bastelfreak can you tag a new release that includes 4bf2503? It's been over 3 months now where librarian is straight up broken without a workaround.

[bastelfreak]

Hi, as you can see in https://github.com/voxpupuli/librarian-puppet/pull/123, we currently don't have a working CI. And I don't have the time right now to fix this. If you can take a look, we can make a new release afterwards.

[parkr]

@der-eismann FWIW I downgraded minitar to pre-1.0 and this worked for me to get librarian working great again.

[hurricanehrndz]

I will see if I can fix #123

[bastelfreak]

I raised https://github.com/voxpupuli/librarian-puppet/pull/129 which fixes it

[parkr]

Thank you, @bastelfreak !

[bastelfreak]

5.1.0 release is out

[der-eismann]

Thank you very much for the release! Unfortunately puppet 8.9.0 itself seems to still rely on the old minitar though, resulting in this:

$ gem install --no-document librarian-puppet -v '~> 5'
Fetching minitar-1.0.2.gem
Fetching net-http-0.6.0.gem
Fetching rsync-1.0.9.gem
Fetching faraday-net_http-3.4.0.gem
Fetching puppet_forge-6.0.0.gem
Fetching faraday-2.12.2.gem
Fetching faraday-follow_redirects-0.3.0.gem
Fetching librarianp-1.1.2.gem
Fetching librarian-puppet-5.1.0.gem
Successfully installed rsync-1.0.9
Successfully installed minitar-1.0.2
Successfully installed net-http-0.6.0
Successfully installed faraday-net_http-3.4.0
Successfully installed faraday-2.12.2
Successfully installed faraday-follow_redirects-0.3.0
Successfully installed puppet_forge-6.0.0
Successfully installed librarianp-1.1.2
Successfully installed librarian-puppet-5.1.0
9 gems installed
$ cd /opt/puppetlabs/puppet
$ librarian-puppet install --clean
Error executing puppet module install. Check that this command succeeds:
puppet module install --version 6.6.0 \
  --target-dir /opt/puppetlabs/puppet/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/6.6.0 \
  --module_repository https://forgeapi.puppetlabs.com \
  --modulepath /opt/puppetlabs/puppet/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/6.6.0 \
  --module_working_dir /opt/puppetlabs/puppet/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/6.6.0 \
  --ignore-dependencies puppetlabs-stdlib
Error:
Error: No suitable tar implementation found
Error: Try 'puppet help module install' for usage

When I execute the mentioned command with --debug I get this:

Info: Resolving dependencies ...
Info: Preparing to install ...
Debug: Using cached connection for https://forgeapi.puppetlabs.com:443
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-6.6.0.tar.gz returned 200 OK
Debug: Caching connection for https://forgeapi.puppetlabs.com:443
Debug: Facter: Resolving facts sequentially
Debug: Facter: resolving fact with user_query: fips_enabled
Debug: Facter: Searching fact: fips_enabled in file: fips_enabled.rb
Debug: Facter: Searching fact: fips_enabled in core facts and external facts
Debug: Facter: Loading all internal facts
Debug: Facter: List of resolvable facts: [#<Facter::SearchedFact:0x00007f432cd38f80 @name="fips_enabled", @fact_class=Facts::Linux::FipsEnabled, @user_query="fips_enabled", @type=:core, @file=nil>]
Debug: Facter: Loading external facts
Debug: Facter: fact "fips_enabled" has resolved to: false
Debug: Could not find library 'archive/tar/minitar' required to enable feature 'minitar'

After installing the old minitar as well with /opt/puppetlabs/puppet/bin/gem install --no-document minitar -v '0.12' it works fine. What a dependency nightmare.

[ekohl]

Honestly I think the puppet module install face should have been extracted to its own gem, just like puppet-modulebuilder to build the releases. Then you don't actually need Puppet itself installed to run librarian. It'd also be useful in other tools. Given Perforce's closed source plans I don't see it being resolved in Puppet itself, but it's something we could implement in the OpenVox context.

[bastelfreak]

yes I totally agree