bolt icon indicating copy to clipboard operation
bolt copied to clipboard
verified publisher icon puppetlabs

BoltSpec::BoltContext.in_bolt_context should be able to find publicly documented Bolt data types

Open Sharpie opened this issue 2 years ago • 1 comments

Describe the Bug

The BoltSpec::BoltContext.in_bolt_context helper added in commit 7aad50e aims to enable testing of functions that use the built-in bolt datatypes. However, the testcase and examples given use the "internal" name of Boltlib::TargetSpec where the publicly documented name of this data type is TargetSpec:

https://www.puppet.com/docs/bolt/latest/bolt_types_reference.html#targetspec

Spec tests for functions written to use the publicly documented type name fail.

Steps to Reproduce

Clone the bolt repository and patch the fixture used by the with_datatype_spec.rb test to use the TargetSpec data type:

git clone https://github.com/puppetlabs/bolt
cd bolt
git checkout 3.26.2

patch -p1 <<'EOF'
diff --git a/bolt_spec_spec/functions/with_datatype.pp b/bolt_spec_spec/functions/with_datatype.pp
index d70cdb34..affd3cab 100644
--- a/bolt_spec_spec/functions/with_datatype.pp
+++ b/bolt_spec_spec/functions/with_datatype.pp
@@ -1,5 +1,5 @@
 function bolt_spec_spec::with_datatype (
-  Boltlib::TargetSpec $target
+  TargetSpec $target
 ) {
   out::message("Loaded TargetSpec ${$target}")
   $target
EOF

The test fails with an error indicating the TargetSpec data type cannot be found:

bundle install
cd bolt_spec_spec

bundle exec rake spec
I, [2023-01-25T16:26:20.053353 #71136]  INFO -- : Creating symlink from spec/fixtures/modules/bolt_spec_spec to /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/bolt_spec_spec
/Users/charlie.sharpsteen/.rbenv/versions/2.7.6/bin/ruby -I/Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib:/Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-support-3.12.0/lib /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/exe/rspec --pattern spec/\{aliases,classes,defines,functions,hosts,integration,plans,tasks,type_aliases,types,unit\}/\*\*/\*_spec.rb
F

Failures:

  1) bolt_spec_spec::with_datatype bolt_context runs a Puppet function with Bolt datatypes
     Failure/Error: is_expected.to run.with_params('localhost').and_return('localhost')

       expected bolt_spec_spec::with_datatype("localhost") to have returned "localhost" instead of raising Puppet::PreformattedError(Evaluation Error: Resource type not found: TargetSpec (file: /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/bolt_spec_spec/spec/fixtures/modules/bolt_spec_spec/functions/with_datatype.pp, line: 2, column: 3))
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/runtime3_support.rb:39:in `optionally_fail'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/runtime3_support.rb:21:in `fail'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/evaluator_impl.rb:324:in `eval_QualifiedReference'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/visitor.rb:49:in `block in visit_this'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/visitor.rb:43:in `each'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/visitor.rb:43:in `visit_this'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/visitor.rb:96:in `visit_this_1'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/closure.rb:341:in `create_param_type'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/closure.rb:299:in `block in create_callable_type'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/closure.rb:298:in `each'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/closure.rb:298:in `create_callable_type'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/evaluator/closure.rb:146:in `type'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/dispatcher.rb:23:in `block in find_matching_dispatcher'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/dispatcher.rb:23:in `each'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/dispatcher.rb:23:in `find'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/dispatcher.rb:23:in `find_matching_dispatcher'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/dispatcher.rb:36:in `dispatch'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/function.rb:46:in `block in call'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/function.rb:45:in `catch'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/pops/functions/function.rb:45:in `call'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-puppet-2.12.0/lib/rspec-puppet/example/function_example_group.rb:19:in `block in execute'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/context.rb:62:in `override'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet.rb:289:in `override'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-puppet-2.12.0/lib/rspec-puppet/example/function_example_group.rb:18:in `execute'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-puppet-2.12.0/lib/rspec-puppet/matchers/run.rb:10:in `matches?'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-expectations-3.12.2/lib/rspec/expectations/handler.rb:51:in `block in handle_matcher'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-expectations-3.12.2/lib/rspec/expectations/handler.rb:27:in `with_matcher'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-expectations-3.12.2/lib/rspec/expectations/handler.rb:48:in `handle_matcher'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-expectations-3.12.2/lib/rspec/expectations/expectation_target.rb:65:in `to'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-expectations-3.12.2/lib/rspec/expectations/expectation_target.rb:101:in `to'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/bolt_spec_spec/spec/functions/with_datatype_spec.rb:17:in `block (2 levels) in <top (required)>'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:263:in `instance_exec'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:263:in `block in run'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:511:in `block in with_around_and_singleton_context_hooks'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:468:in `block in with_around_example_hooks'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/hooks.rb:486:in `block in run'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/hooks.rb:626:in `block in run_around_example_hooks_for'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:352:in `call'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/bolt_spec_spec/spec/functions/with_datatype_spec.rb:11:in `block (3 levels) in <top (required)>'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/context.rb:62:in `override'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet.rb:289:in `override'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/lib/bolt_spec/bolt_context.rb:117:in `in_bolt_context'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/bolt_spec_spec/spec/functions/with_datatype_spec.rb:10:in `block (2 levels) in <top (required)>'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:457:in `instance_exec'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:457:in `instance_exec'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/hooks.rb:390:in `execute_with'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:352:in `call'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/hooks.rb:629:in `run_around_example_hooks_for'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/hooks.rb:486:in `run'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:468:in `with_around_example_hooks'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:511:in `with_around_and_singleton_context_hooks'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:259:in `run'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:646:in `block in run_examples'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:642:in `map'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:642:in `run_examples'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:607:in `run'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `map'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2070:in `with_suite_hooks'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:116:in `block in run_specs'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:74:in `report'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:115:in `run_specs'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:89:in `run'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
       /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/exe/rspec:4:in `<main>'
     # ./spec/functions/with_datatype_spec.rb:17:in `block (2 levels) in <top (required)>'
     # ./spec/functions/with_datatype_spec.rb:11:in `block (3 levels) in <top (required)>'
     # /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet/context.rb:62:in `override'
     # /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/puppet-7.22.0/lib/puppet.rb:289:in `override'
     # /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/lib/bolt_spec/bolt_context.rb:117:in `in_bolt_context'
     # ./spec/functions/with_datatype_spec.rb:10:in `block (2 levels) in <top (required)>'

Finished in 0.673 seconds (files took 2.07 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/functions/with_datatype_spec.rb:15 # bolt_spec_spec::with_datatype bolt_context runs a Puppet function with Bolt datatypes

/Users/charlie.sharpsteen/.rbenv/versions/2.7.6/bin/ruby -I/Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/lib:/Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-support-3.12.0/lib /Users/charlie.sharpsteen/Projects/puppetlabs/bolt/.bundle/ruby/2.7.0/gems/rspec-core-3.12.0/exe/rspec --pattern spec/\{aliases,classes,defines,functions,hosts,integration,plans,tasks,type_aliases,types,unit\}/\*\*/\*_spec.rb failed

Expected Behavior

The test should pass as TargetSpec is the publicly documented name of the type.

Environment

  • Version 3.26.2
  • Platform macOS 13.2

Additional Context

I'm guessing this is due to the BoltSpec::Plans::MockExecutor not calling Bolt::PAL#alias_types:

https://github.com/puppetlabs/bolt/blob/3.26.2/lib/bolt/pal.rb#L124-L131

The effect of this function call can be replicated by adding the following precondition to the spec test:

  let(:pre_condition) do
    'type TargetSpec = Boltlib::TargetSpec'
  end

However, this workaround will cause the spec test to start failing with a duplicate type declaration error if this bug is resolved in BoltSpec.

Sharpie avatar Jan 25 '23 22:01 Sharpie

I'd like to kindly remind that this is still an issue today. The workaround above helped.

dionysius avatar Sep 07 '24 00:09 dionysius