chef-provisioning-docker icon indicating copy to clipboard operation
chef-provisioning-docker copied to clipboard

Docker::Error::TimeoutError

Open matiasdecarli opened this issue 10 years ago • 16 comments

Sometimes Im getting this error.

    Error executing action `converge` on resource 'machine[web27]'
    ================================================================================

    Docker::Error::TimeoutError
    ---------------------------
    read timeout reached

    Resource Declaration:
    ---------------------
    # In /home/vagrant/cookbooks/murally/recipes/create_container.rb

      4: machine "web27" do
      5:   recipe 'matiasdecarli::provision_container' 
      6: 
      7:   chef_server :chef_server_url=>'https://api.opscode.com/organizations/matiasdecarli',
      8:     :options =>{
      9:        :client_name=>"matiasdecarli",
     10:        :signing_key_filename=>"/etc/chef/client.pem"
     11:   }
     12: 
     13:   machine_options :docker_options => {
     14:       :base_image => {
     15:           :name => 'matiasdecarli/release-chef',
     16:           :repository => 'matiasdecarli',
     17:           :tag => 'latest'
     18:       },
     19:       :command => 'node',
     20:       :ports => 8080   
     21:   }
     22: end

Watching the log, I can see that this is happening after the Chef run

[2014-09-25T18:11:37+00:00] INFO: Chef Run complete in 347.802490812 seconds
[2014-09-25T18:11:37+00:00] INFO: Running report handlers
[2014-09-25T18:11:37+00:00] INFO: Report handlers complete
[2014-09-25T18:11:37+00:00] INFO: Sending resource update report (run-id: fb22fa40-e662-4193-b99b-71fe8db0cd1f)

Googleing this error, I can see that the solution is passing parameters (write_timeout, read_timeout) to docker-api. Is there any way to do this using the machine resource?

matiasdecarli avatar Sep 25 '14 18:09 matiasdecarli

Huh... Yeah we could make those passable options; what is it doing when it times out?

johnewart avatar Sep 26 '14 01:09 johnewart

I guess after the run, its updating the node in the server and for some reason it reach the time out. If I delete this recipe, the run finish successfully, which is kinda strange...

As I can see here, https://github.com/swipely/docker-api/issues/106 read && write timeout parameters could be the fix for this issue

matiasdecarli avatar Sep 26 '14 01:09 matiasdecarli

Can you gist your debug output so I can see where it comes from?

johnewart avatar Sep 26 '14 02:09 johnewart

Yes. Here is the log

https://gist.github.com/matiasdecarli/19fb00a24bdb6031c25b

matiasdecarli avatar Sep 26 '14 03:09 matiasdecarli

Ok so it's timing out when committing the image; I guess I could see that if you are making a large number of containers and your host's disk is under a lot of load or something (is that the case?). I don't think we should make blanket changes to the timeout through global config but I'm not certain off-hand if there's any way to change the timeout just for the commit call so that may be the only option. I'll do some digging in the morning.

johnewart avatar Sep 26 '14 04:09 johnewart

Now that you mention, maybe the image is a little too big (2GB). Either way, I don't think this should be a problem. Im going to do a little research as well. Please, keep me updated.

matiasdecarli avatar Sep 26 '14 11:09 matiasdecarli

Well... i've created a 1GB image, but im still having the same timeout error... I have only 1 imag, 0 containers and 0 nodes at the beginning of the run

matiasdecarli avatar Sep 26 '14 14:09 matiasdecarli

@johnewart Im still facing this issue.

Did you had the chance to see a little more about this? Thanks in advance

matiasdecarli avatar Oct 01 '14 15:10 matiasdecarli

@johnewart Im having this issue more and more lately. Is there any workaround for this?

matiasdecarli avatar Oct 06 '14 18:10 matiasdecarli

I am seeing this problem as well. I will try and do a little more debugging and see if I can figure out where and why, I have been wholly unable to get the chef-provisioning-docker to work at all though.

phillipbroberts avatar May 05 '15 03:05 phillipbroberts

Also seeing this issue with chef-provisioning-docker and the timeout.

Happens with the "wario" example here - https://github.com/chef/chef-provisioning-docker

Running OS X 10.10.4 docker-machine version 0.4.1 Chef 12.4.1

Has this been tested with docker-machine and not boot2docker? Hmm...

alecl avatar Sep 17 '15 20:09 alecl

I also have read timeout errors in execute(). This seems logical since sometime I have long commands running (like apt-get install). Putting this in DockerTransport#execute method seems to make things better:

Docker.options[:read_timeout] = 7200

yazgoo avatar Sep 29 '15 08:09 yazgoo

Hi, any news about this issue?

Sliim avatar Feb 10 '16 09:02 Sliim

+1 still happening

jgoulah avatar Mar 16 '16 01:03 jgoulah

I can also confirm this fixes Docker.options[:read_timeout] = 7200

jgoulah avatar Mar 16 '16 03:03 jgoulah

@jkeiser I believe this one is now fixed in latest master release and can be closed

jgoulah avatar Mar 30 '16 17:03 jgoulah