Can't run kitchen-docker with Docker for Mac

[chilicheech]

I have the new Docker for Mac installed and I can run docker just fine. When I run kitchen docker though I get the following error:

)-> kitchen converge --log-level=debug
-----> Starting Kitchen (v1.7.3)
D      [kitchen::driver::docker command] BEGIN (sudo -E docker >> /dev/null 2>&1)
D      [kitchen::driver::docker command] END (0m0.01s)
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::UserError
>>>>>> Message: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

D      ------Exception-------
D      Class: Kitchen::UserError
D      Message: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/
D      ---Nested Exception---
D      Class: Kitchen::ShellOut::ShellCommandFailed
D      Message: Expected process to exit with [0], but received '1'
---- Begin output of sudo -E docker >> /dev/null 2>&1 ----
STDOUT:
STDERR:
---- End output of sudo -E docker >> /dev/null 2>&1 ----
Ran sudo -E docker >> /dev/null 2>&1 returned 1
D      ------Backtrace-------
D      /Users/thiagoo/.chefdk/gem/ruby/2.1.0/gems/kitchen-docker-2.4.0/lib/kitchen/driver/docker.rb:78:in `rescue in verify_dependencies'
D      /Users/thiagoo/.chefdk/gem/ruby/2.1.0/gems/kitchen-docker-2.4.0/lib/kitchen/driver/docker.rb:76:in `verify_dependencies'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/driver.rb:45:in `for_plugin'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:230:in `new_driver'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:243:in `new_instance'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:139:in `block in build_instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:138:in `map'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:138:in `with_index'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:138:in `build_instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:114:in `instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command.rb:115:in `filtered_instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command.rb:145:in `parse_subcommand'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command/action.rb:38:in `block in call'
D      /opt/chefdk/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command/action.rb:37:in `call'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/cli.rb:56:in `perform'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/cli.rb:180:in `block (2 levels) in <class:CLI>'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/cli.rb:321:in `invoke_task'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/bin/kitchen:13:in `block in <top (required)>'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/errors.rb:154:in `with_friendly_errors'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/bin/kitchen:13:in `<top (required)>'
D      /opt/chefdk/bin/kitchen:17:in `load'
D      /opt/chefdk/bin/kitchen:17:in `<main>'
D      ----------------------

)-> sudo -E docker >> /dev/null 2>&1

)-> echo $?
0

)->

[peterabbott]

You have to make sure that DOCKER_HOST is not set to the tcp address you'd use with Docker Machine or use the socket attribute to set to unix:///var/run/docker.sock

There is an existing issue that captured all the things around getting this driver to work with the native client: https://github.com/portertech/kitchen-docker/issues/201

[robertlabrie]

Just run eval (docker-machine env default) on the shell. This is an issue with the Mac, not the driver. If you can't docker ps then the driver doesn't stand a chance

[chilicheech]

i don't use docker-machine, though.. i'm using the native docker for mac. no docker environment variables need to be set for it.

[robertlabrie]

Sorry, I misunderstood. Good luck!

[chilicheech]

I tried the use_sudo: false workaround from #201 and it works. 😄 .. But it's just a workaround. It should work whether or not I use sudo.

[ozbillwang]

So the work around steps are:

• active port-forwarding

    pinata set network nat external-bind=true

• add use_sudo: false to your kitchen.yml driver config:

    driver:
      name: docker
      use_sudo: false

[erikogan]

I added a PR (#210) to add an environment variable to override the default value before I noticed this issue.

I don’t want to edit (all of) my .kitchen.yml file(s), because this will break kitchen-docker for others on my team for whom the default value is correct.

I had been editing the files I needed to work on, but it is cumbersome enough to do, and I’m worried I’ll accidentally check it in.

[jlruizmlg]

You can try the docker version 1.12 beta20 to me works

[davidcpell]

Ran into this issue today:

OSX 10.11.6 Docker version 1.12.1, build 6f9534c kitchen-docker 2.6.0

Had to add use_sudo: false to .kitchen.yml to get it to work.

[erikogan]

@davidcpell You can also add use_sudo: false to your ~/.kitchen/config.yml in the driver section.

This solution was mentioned on the PR referenced above, but never brought back here.

[davidcpell]

Thanks @erikogan! Came back here today because I ran into the issue again and forgot how I had gotten around it. That will help a bunch.

[walterdolce]

I'm having the very same issue on one of the machines I work from. The user from which I run bundle exec kitchen list (which fails) triggers sudo -E docker >> /dev/null 2>&1 under the hood (as seen from the debug output).

If I run the same command from the CLI I am prompted to enter the password. This is what I suspect it's causing the exception to be raised. Running the same command without sudo works as expected.

• Is the fact the user gets prompted to enter a password enough to fail the command and raise the exception? (which, by the way, in this instance reports the wrong message to the user given docker is installed and reachable from PATH)
• How do you think this scenario should be handled?
• Last but not least, wouldn't a simple which docker or where.exe docker be enough to verify the existence/accessibility of the docker executable, instead of running [sudo] docker >> {/dev/null|NIL} 2>&1?

[walterdolce]

FYI (Please note I purposefully removed >> {NIL|/dev/null})

$ bundle exec kitchen list -l debug
D      [local command] BEGIN (sudo -E docker 2>&1)
       sudo: no tty present and no askpass program specified
D      [local command] END (0m0.08s)
...

[docwhat]

See @erikogan's solution below... You have to tell kitchen not to use sudo.

[peterabbott]

The main issue is probably the check for remote_socket forcing sudo, https://github.com/test-kitchen/kitchen-docker/blob/master/lib/kitchen/driver/docker.rb#L60

When using Docker for Mac (or Windows) the DOCKER_HOST environment variable no longer needs to be set. So when the driver starts it sees that it is running locally and assumes that it needs to run sudo. This behaviours could perhaps be reviewed or the check made a bit smarter???

[coderanger]

Yeah, the right long-term fix is probably to use docker version or docker info to probe things a bit more.

[joegoggins]

Is there anything I could do to help address this issue?

Or perhaps there is a different recommendation for using Test Kitchen with Docker on a Mac?

I'm blocked right now and I'm not sure how to proceed.

If I use this use_sudo: false work-around, I can make the Class: Kitchen::UserError Message: You must first install the Docker CLI tool... error go away, however, instead I get a failure like this on kitchen converge:

            Installing chef 12.14.89
                 installing with dpkg...
                 Selecting previously unselected package chef.
          (Reading database ... 16043 files and directories currently installed.)
                 Preparing to unpack .../chef_12.14.89-1_amd64.deb ...
                 Unpacking chef (12.14.89-1) ...
                 Setting up chef (12.14.89-1) ...
                 Thank you for installing Chef!
                 mkdir: cannot create directory ‘/tmp/kitchen’: No space left on device

which makes sense. When you run kitchen login default as the kitchen user (not root) and run mkdir /tmp/kitchen you get this same error, it's because the provisioner now no longer has privileges to do this.

How do I give the provisioner the permissions it needs when use_sudo: false?

My .kitchen.yml

---
driver:
  name: docker
  use_sudo: false
  privileged: true
transport:
  max_ssh_sessions: 6
provisioner:
  name: chef_zero
  require_chef_omnibus: 12.21.4
  log_level: info
  data_bags_path: test/data_bags
  encrypted_data_bag_secret_key_path: test/data_bags/encrypted_data_bag_secret
verifier:
  name: inspec
  format: documentation
platforms:
  # We run this special image so we can keep upstart running
  # inside the container because
  # - runit services inside the container require it
  # - Gets around SSH connect issue: https://github.com/test-kitchen/kitchen-docker/issues/254
  # - runit cookbook recommends it: https://github.com/chef-cookbooks/runit#development
  - name: ubuntu-14.04
    driver_config:
      image: ubuntu-upstart:14.04
      platform: ubuntu
      disable_upstart: false
      run_command: /sbin/init

suites:
  - name: default
    named_run_list: default
    verifier:
      inspec_tests:
        - path: test/smoke/default
        - name: particle-inspec-foo-service
          git: [email protected]:spark/particle-inspec.git
    attributes:
      particle_foo_service:
        my_attribute: "foo"
      tags:
      - my_special_tag

[coderanger]

@joegoggins Please include your full config file. You might have disabled too much sudo.

[joegoggins]

@coderanger I added my failing .kitchen.yml to the above comment. Thanks for the ping asking for this. Let me know if I can provide anything else or what you'd recommend my next step to be!

[FilBot3]

I also have this issue, but not just for OSX. Even though the default is false for use_sudo, I still had to specify the use_sudo: false option otherwise I got this error:

$ chef exec kitchen converge
-----> Starting Kitchen (v1.17.0)
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::UserError
>>>>>> Message: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

[joegoggins]

On OSX, I was able to get around this error permanently by:

1. Clicking Docker Icon on the menu bar
2. Clicking preferences
3. Clicking "Reset"
4. Clicking "Remove all data"
5. Clicking "Reset to factory defaults"

I haven't seen it since.

Not everyone can do this, because it's highly destructive operation, but I suspect most can because they are using Docker purely for dev purposes on their workstation.

[Dispader]

Thanks, @joegoggins — that worked for me, here.

[JoshuaCWebDeveloper]

I faced the same error on Windows 10 and came across this issue just by Googling the underlying Ruby exception. Even though my error is of a different nature than what is described here, I wanted to chime in and add that the recommended solution of specifying use_sudo: false worked for me as well to get past this error.

In my case, the error was because I am running kitchen under MINGW64 on Git Bash for Windows, and there is no sudo command on this shell, so running sudo … anything results in an error.

[detj]

Looks like this is still an issue. Couldn't get this to work for the life of me. Running docker v18.06.1-ce on macOS 10.13.6.

.kitchen.yml looks like this

---
driver:
  name: docker
  use_sudo: false
  privileged: true

I tried @joegoggins's solution of restoring Docker to its factory settings, removing all data, but no luck. Tried all combinations of use_sudo and privileged and other settings too. Getting this error still

>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 2 actions failed.
>>>>>>     Failed to complete #create action: [Expected process to exit with [0], but received '125'
---- Begin output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultubuntu1604-detj-pinwheellocal-da42cs3y --privileged fa5c9ca31d03 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
STDOUT:
STDERR: docker: you are not authorized to perform this operation: server returned 401.
See 'docker run --help'.
---- End output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultubuntu1604-detj-pinwheellocal-da42cs3y --privileged fa5c9ca31d03 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
Ran docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultubuntu1604-detj-pinwheellocal-da42cs3y --privileged fa5c9ca31d03 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid returned 125] on default-ubuntu-1604
>>>>>>     Failed to complete #create action: [Expected process to exit with [0], but received '125'
---- Begin output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultcentos7-detj-pinwheellocal-b07r76rl --privileged aee8aabc8079 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
STDOUT:
STDERR: docker: you are not authorized to perform this operation: server returned 401.
See 'docker run --help'.
---- End output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultcentos7-detj-pinwheellocal-b07r76rl --privileged aee8aabc8079 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
Ran docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultcentos7-detj-pinwheellocal-b07r76rl --privileged aee8aabc8079 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid returned 125] on default-centos-7
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Running kitchen diagnose --all does pass though, which is all the more confusing.

Is there any progress with this issue ?

[martinm82]

@detj how does your docker run command looks like? In my case I got it working with Docker 18.6.1 (on macOS 10.13.6) with the following command:

docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):/app -w /app salt-testing sh

and the use_sudo: false setting in kitchen.yml.

In case your image does not run as root then you have to pass --group-add root in order to get read permissions on /var/run/docker.sock within the container, otherwise you will get Permission denied errors.

[darkn3rd]

I am able to use kitchen on Mac OS X with either Docker-For-Mac (hyperkit) or Docker-Machine (virtualbox).

I have noticed that sometime, the ssh key is set with bad permissions, which I have to run chmod to reset them so that kitchen login works.

[luisra51]

This is working for me sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock