Open
jolly2
opened this issue 9 years ago
•
6 comments
Trying contiv volume plugin for ceph/rbd support for docker. Ceph/rbd works perfectly fine independent of docker. Once integrated with docker, after creating the global and tenant configurations, when a container is launched with contiv volume driver and the policy, it errors out with the message "rbd: map failed: (6) No such device or address". It looks like it is executing rbd map command without waiting for or checking if the volume got created. Detailed log from volplugin is provided below.
For some reason , it is not executing the rbd create volume command. May be it is timing out fast. I have set the Timeout option in global configuration to 10. The doc says it is in minutes. But, I do not see rbd create volume run in the background. Is there any other way to set the timeout driver option in minutes?
Using contiv volume plugin git clone with docker 1.12 with ceph jewel.
This is failing because the /sys/bus devices don’t exist. How did you launch volplugin? This sounds like one of two things:
Review the /sys is mapped right and that your rbd version in your container matches the one on your host kernel (nothing much we can do about this, especially on redhat).
If that doesn’t work for you I will review soon, but I just saw this and need to leave.
Trying contiv volume plugin for ceph/rbd support for docker. Ceph/rbd works perfectly fine independent of docker. Once integrated with docker, after creating the global and tenant configurations, when a container is launched with contiv volume driver and the policy, it errors out with the message "rbd: map failed: (6) No such device or address". It looks like it is executing rbd map command without waiting for or checking if the volume got created. Detailed log from volplugin is provided below.
For some reason , it is not executing the rbd create volume command. May be it is timing out fast. I have set the Timeout option in global configuration to 10. The doc says it is in minutes. But, I do not see rbd create volume run in the background. Is there any other way to set the timeout driver option in minutes?
Using contiv volume plugin git clone with docker 1.12 with ceph jewel.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/contiv/volplugin/issues/470, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AABJ6_pa8aZlc7eJJGJUTEHq4t5wX5Znks5q9Q6TgaJpZM4KwRof.
Thanks Erik for the response. However, i found the cause of the problem. contiv createvolume command is creating rbd create volume, but with some new kernel features such as exclusive-lock, object-map, fast-diff, deep-flatten which are not supported yet. So, i had to intercept the code to disable these features for each volume instance that contiv created, before it maps the vol image to the block device. I tried adding a function for rbd disable feature in the code, but, that is returning exit status 1 from within contiv..
interesting. Can you give me a more concrete example of what is going on and what you had to do to get it to the point you're at now? Pastes, anything would be a huge help.
In my 3 node docker/ceph POC environment, the kernel rbd does not support these features exclusive lock, object map, fast-diff, and deep-flatten. It supports only layering feature. I am using ubuntu 16.04LTS with kernel version 4.4.xxx By default, ceph rbd creates images with all these features. As my kernel rbd does not support these features, rbd map from contiv was failing. I tried fixing by disabling these features per image basis for which contiv plugin does not seem to have the option to pass parameters to ceph/rbd. So, i had to fix it by disabling all these features in my whole POC environment in ceph.conf.
contiv-experimental
