Butcher failes to copy client.pem, when Chef-run is cancelled with Ctrl-C

[DracoAter]

I have a Vagrantfile that starts vagrant machine (VirtualBox) and provisions it with chef-client. I want the client and node to be removed when I run vagrant destroy. The problem is that when I run vagrant up and chef starts provisioning, if I press Ctrl-C twice to cancel process, butcher fails to copy the client.pem file. It's not something what I do very often, but the same thing takes place when the job is cancelled in Jenkins.

Log output:

[vbox-precise64] Importing base box 'precise64'...
[vbox-precise64] Matching MAC address for NAT networking...
[...]
[vbox-precise64] Mounting shared folders...
[vbox-precise64] -- /vagrant
[vbox-precise64] Running provisioner: chef_client...
Creating folder to hold client key...
Uploading chef client validation key...
Uploading chef encrypted data bag secret key...
Generating chef JSON and uploading...
Running chef-client...
stdin: is not a tty
[2013-12-11T12:03:29+00:00] INFO: *** Chef 10.14.4 ***
[2013-12-11T12:03:47+00:00] INFO: Client key /etc/chef/client.pem is not present - registering
[...]
[2013-12-11T12:03:48+00:00] INFO: Running start handlers
[2013-12-11T12:03:48+00:00] INFO: Start handlers complete.
[2013-12-11T12:03:48+00:00] INFO: Loading cookbooks [apache2]
[...]
[vbox-precise64] Waiting for cleanup before exiting...                        <-- First time Ctrl-C pressed.
[...]
[2013-12-11T12:03:53+00:00] INFO: Storing updated cookbooks/apache2/attributes/default.rb in the cache.
[...]
[vbox-precise64] Exiting immediately, without cleanup!                        <-- Second time Ctrl-C pressed.
[Butcher] Guest cache dir at /vagrant/.vagrant/butcher
[Butcher] Failed to copy /etc/chef/client.pem to /vagrant/.vagrant/butcher/vbox-precise64-client.pem
[Butcher] can't alloc thread
[Butcher] Failed to copy Chef client key from the guest: Vagrant::Butcher::Errors::KeyCopyFailure

And as there is no client key copied, vagrant destroy also fails to remove client and node from chef-server. (/etc/chef/client.pem exists inside VM.)

[c10l]

Interesting edge case. I'll have to investigate that and check that it's even possible to solve. It seems that when you hit control-C the second time you killed the possibility of Vagrant opening a thread for the plugin, but I'm just guessing at this point.

[benlangfeld]

I'm also seeing this, but not the Butcher output at the end. Perhaps the validator key could be used to delete the node/client if the client key cannot be found?

[c10l]

@benlangfeld I don't think the validator key can be used in this fashion without opening a whole new set of security issues. It would also introduce a new dependency -- to change permissions of the validator client on the Chef server.

I'll look into this issue again when I have the time, but I can't promise a date for a solution.

In the meantime, pull requests are welcome! :)

[smuth4]

Would it be possible to double-check if the client.pem is available on the client before attempting to butcher it? I.E. vagrant destroy -> butcher starts cleanup -> "I can't find a cached client.pem" -> fetch pem from client -> successfully remove client/node from chef server.

[c10l]

@smuth4 That can probably be done, but if the VM is down there's no way to retrieve the key. If everyone is happy with that I'll give it a go. @DracoAter and @benlangfeld what's your opinion?